GATE DA & AI Engineering Machine Learning Flashcards

24 sample cards from the Machine Learning deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. How are decision trees prone to overfitting and how is it controlled?

   Deep, fully grown trees overfit by memorizing noise; controlled via pruning, max depth, minimum samples per leaf, or minimum impurity decrease.

2. Define the bias-variance trade-off.

   It is the tension between bias (error from overly simple assumptions, causing underfitting) and variance (error from sensitivity to training data, causing overfitting); reducing one tends to increase the other.

3. Write the bias-variance decomposition of expected squared error.

   $$\mathbb{E}[(y-\hat{f}(\vec{x}))^{2}] = \text{Bias}^{2}[\hat{f}(\vec{x})] + \text{Var}[\hat{f}(\vec{x})] + \sigma^{2}$$ where $\sigma^{2}$ is irreducible noise.

4. In the bias-variance trade-off, what characterizes underfitting versus overfitting?

   Underfitting: high bias, low variance, poor on both train and test. Overfitting: low bias, high variance, good on train but poor on test.

5. What is the purpose of cross-validation?

   To estimate a model's generalization performance and tune hyperparameters by repeatedly training and validating on different data partitions, using data more efficiently than a single split.

6. Describe k-fold cross-validation.

   Data is split into $k$ equal folds; the model trains on $k-1$ folds and validates on the remaining one, rotating so each fold is validation once; the $k$ scores are averaged.

7. What is Leave-One-Out Cross-Validation (LOOCV)?

   A special case of $k$-fold where $k = n$: each single observation is held out as the validation set once. It is nearly unbiased but computationally expensive and high-variance.

8. What is stratified k-fold cross-validation and when is it used?

   A variant that preserves the class proportions in each fold; used for classification, especially with imbalanced classes, to keep folds representative.

9. What is a Multi-Layer Perceptron (MLP)?

   A feed-forward neural network with one or more hidden layers of nonlinearly-activated neurons fully connected between layers, capable of learning nonlinear mappings.

10. Why are nonlinear activation functions essential in an MLP?

    Without them, stacked linear layers collapse into a single linear transformation; nonlinearities (e.g. ReLU, sigmoid, tanh) let the network approximate complex nonlinear functions (universal approximation).

11. What does 'feed-forward' mean in a feed-forward neural network?

    Information flows in one direction, from input through hidden layers to output, with no cycles or feedback connections (a directed acyclic graph).

12. What algorithm trains MLPs and what does it compute?

    Backpropagation with gradient descent; it applies the chain rule to compute the gradient of the loss with respect to every weight, which is used to update the weights.

13. What is the goal of clustering algorithms?

    Unsupervised grouping of data points so that points in the same cluster are more similar to each other than to points in other clusters, without using labels.

14. Describe the K-Means algorithm's iterative steps.

    1) Initialize $k$ centroids; 2) assign each point to its nearest centroid; 3) recompute each centroid as the mean of assigned points; repeat steps 2-3 until assignments stabilize.

15. What objective does K-Means minimize?

    The within-cluster sum of squares (inertia): $$J = \sum_{i=1}^{k}\sum_{\vec{x}\in C_i}\lVert\vec{x}-\vec{\mu}_i\rVert^{2}$$

16. How does K-Medoids differ from K-Means?

    K-Medoids uses actual data points (medoids) as cluster centers and minimizes sum of pairwise dissimilarities, making it more robust to outliers and usable with arbitrary distance metrics; K-Means uses mean centroids and Euclidean distance.

17. What is the 'elbow method' for choosing $k$ in K-Means?

    Plot within-cluster sum of squares versus $k$; the 'elbow' point, where adding more clusters yields diminishing reduction in WCSS, indicates a good $k$.

18. What are the two main approaches to hierarchical clustering?

    Agglomerative (bottom-up): start with each point as its own cluster and merge. Divisive (top-down): start with one cluster containing all points and split.

19. What is a dendrogram in hierarchical clustering?

    A tree diagram showing the sequence of merges/splits and the distances at which they occur; cutting it at a chosen height yields a given number of clusters.

20. Name three linkage criteria used in agglomerative hierarchical clustering.

    Single linkage (minimum pairwise distance), complete linkage (maximum pairwise distance), and average linkage (mean pairwise distance); Ward's method (minimizes variance increase) is also common.

21. What is a key advantage of hierarchical clustering over K-Means?

    It does not require pre-specifying the number of clusters and produces a full hierarchy (dendrogram), though it is computationally more expensive (typically $O(n^{2}\log n)$ or $O(n^{3})$).

22. Compare generative versus discriminative classifiers with examples.

    Generative models the joint $P(\vec{x},y)$ / class-conditional densities (e.g. Naive Bayes, LDA); discriminative models the boundary or $P(y\mid\vec{x})$ directly (e.g. logistic regression, SVM).

23. Why does feature scaling matter for KNN, SVM, and K-Means but not for decision trees?

    KNN, SVM, and K-Means rely on distance/inner-product magnitudes, so unscaled features dominate; decision trees split on thresholds per feature and are invariant to monotonic feature scaling.

24. What is one major limitation of K-Means clustering?

    It requires choosing $k$ in advance, assumes spherical equally-sized clusters, is sensitive to initialization and outliers, and may converge to a local optimum (mitigated by k-means++ initialization).

What this deck covers

Machine Learning flashcards FAQ