Data Science & Machine Learning Supervised Machine Learning Flashcards

24 sample cards from the Supervised Machine Learning deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. What are support vectors in an SVM?

   The training points closest to the decision boundary (on or within the margin) that determine the position and orientation of the hyperplane.

2. What is the kernel trick in SVMs?

   It implicitly maps data into a higher-dimensional space using a kernel function (e.g., RBF, polynomial) to find a linear separator without computing the transformation explicitly.

3. In SVM, what does the regularization parameter C control?

   The trade-off between maximizing the margin and minimizing misclassification: large C → less regularization (narrow margin, fewer errors); small C → more regularization (wider margin, more tolerance for errors).

4. How does a decision tree decide where to split a node?

   It chooses the feature and threshold that best reduce impurity, measured by Gini impurity or entropy/information gain (classification) or variance reduction (regression).

5. What is the formula for Gini impurity?

   Gini = 1 − Σ pᵢ², where pᵢ is the proportion of class i in the node. It is 0 for a pure node.

6. Why are decision trees prone to overfitting, and how is it controlled?

   Deep trees can memorize the training data. It is controlled by pruning or hyperparameters like max_depth, min_samples_split, min_samples_leaf, and max_leaf_nodes.

7. What is bagging (bootstrap aggregating)?

   An ensemble method that trains many models on bootstrap (random with-replacement) samples of the data and aggregates their predictions (voting/averaging) to reduce variance.

8. How does a Random Forest extend basic bagging of decision trees?

   In addition to bootstrap sampling, it considers only a random subset of features at each split, decorrelating the trees and further reducing variance.

9. What is the difference between bagging and boosting in terms of how models are trained?

   Bagging trains models independently in parallel to reduce variance; boosting trains models sequentially, each correcting the errors of the previous, to reduce bias.

10. How does AdaBoost work?

    It trains weak learners sequentially, increasing the weights of misclassified samples each round so later learners focus on hard cases, then combines them as a weighted vote.

11. What does Gradient Boosting do at each stage?

    It fits a new weak learner to the residual errors (negative gradient of the loss) of the current ensemble, then adds it scaled by a learning rate to improve predictions.

12. In a confusion matrix, what are TP, TN, FP, and FN?

    TP = correctly predicted positive; TN = correctly predicted negative; FP = negative predicted as positive (Type I error); FN = positive predicted as negative (Type II error).

13. What is the formula for accuracy and when is it misleading?

    Accuracy = (TP + TN) / (TP + TN + FP + FN). It is misleading on imbalanced datasets where a trivial majority-class predictor scores high.

14. Define precision and recall with their formulas.

    Precision = TP / (TP + FP) (of predicted positives, how many are correct). Recall = TP / (TP + FN) (of actual positives, how many were found).

15. What is the F1 score and why is it used?

    F1 = 2·(precision·recall)/(precision+recall), the harmonic mean of precision and recall; it balances the two, useful for imbalanced classes.

16. What does the ROC curve plot and what does AUC represent?

    ROC plots True Positive Rate vs False Positive Rate across thresholds. AUC (area under the curve) is the probability the model ranks a random positive above a random negative; 1.0 is perfect, 0.5 is random.

17. Give the formulas for MAE and MSE in regression.

    MAE = (1/n)·Σ|yᵢ − ŷᵢ|; MSE = (1/n)·Σ(yᵢ − ŷᵢ)². MSE penalizes large errors more heavily.

18. What is RMSE and why is it often preferred over MSE for reporting?

    RMSE = √MSE. It is in the same units as the target variable, making it more interpretable than MSE.

19. What does the R² (coefficient of determination) metric measure?

    The proportion of variance in the target explained by the model: R² = 1 − (SS_residual / SS_total). 1 is perfect; 0 means no better than predicting the mean; it can be negative.

20. What is k-fold cross-validation?

    The data is split into k folds; the model is trained on k−1 folds and validated on the remaining fold, repeated k times so each fold is the validation set once; results are averaged.

21. Why use cross-validation instead of a single train/test split?

    It gives a more robust, lower-variance estimate of model performance by using all data for both training and validation, reducing dependence on one particular split.

22. What is stratified k-fold cross-validation and when is it important?

    It preserves the class distribution (proportions) within each fold; important for imbalanced classification so every fold is representative.

23. What is the difference between grid search and random search for hyperparameter tuning?

    Grid search exhaustively tries all combinations in a defined grid; random search samples random combinations, often finding good values faster in high-dimensional spaces.

24. Why must hyperparameter tuning be evaluated with cross-validation or a validation set rather than the test set?

    Tuning on the test set leaks information and overfits hyperparameters to it, giving an optimistically biased estimate; the test set must stay untouched for final unbiased evaluation.

What this deck covers

Supervised Machine Learning flashcards FAQ