- Logistic Regression: If we need a probabilistic output (80% change of event happening), data is linearly separable, and we want a fast and interpretable baseline model. Avoid if relationship are non-linear.
- Support Vector Machines: If we have a small to medium dataset with high dimensionality and we need a very accurate boundary between class. SVMs work best if the classes are well-separated. We should use it if we care more about the decision boundary than probabilities. It should be avoided if the dataset is large, or if we need probabilities.
- Decision Trees: If we need interpretability as rules (if feature is X, then output is Y), and if the data has a lot of non-linear relationship and categorical feature. It should be avoided for high accuracy and complex problems. Furthermore, decision trees tend to overfit (which can be minimized using Random Forest, even if this may be slower to train, especially on large datasets).
- Neural Network: Use it only if there is a large dataset, otherwise they’ll just overfit or underfit, and if the problem is complex and nonlinear. When we have unstructured data, such as images, text, audio and video. They are difficult to interpret, and slow to train.