GATE DA & AI Engineering Machine Learning Syllabus

Machine Learning syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Machine Learning in GATE DA & AI Engineering, not a summary of it.

1. 1

   Supervised Learning

   2 topics

   • Regression
     • Simple Linear Regression
     • Multiple Linear Regression
     • Ridge Regression
   • Classification Problems
     • Logistic Regression
     • K-Nearest Neighbour
     • Naive Bayes Classifier
     • Linear Discriminant Analysis
     • Support Vector Machine
     • Decision Trees
     • Bias-Variance Trade-off
     • Cross-validation Methods
     • Mul􏰀-Layer Perceptron
     • Feed-Forward Neural Network
2. 2

   Unsupervised Learning

   1 topic

   • Clustering Algorithms
     • K-Means/K-Medoid
     • Hierarchical Clustering
     • Dimensionality Reduction

Machine Learning flashcards for GATE DA & AI Engineering

25 of 61 cards from the Machine Learning deck — real questions with worked answers.

1. What is regression in machine learning?

   Regression is a supervised learning task that models the relationship between input features and a continuous-valued target, predicting a real number $y \in \mathbb{R}$ from inputs $\vec{x}$.

2. How does regression differ from classification?

   Regression predicts a continuous output ($y \in \mathbb{R}$), whereas classification predicts a discrete categorical label from a finite set of classes.

3. What is the model equation for simple linear regression?

   $$y = \beta_0 + \beta_1 x + \varepsilon$$ where $\beta_0$ is the intercept, $\beta_1$ is the slope, and $\varepsilon$ is the error term.

4. What cost function does ordinary least squares (OLS) minimize?

   The residual sum of squares: $$J(\beta) = \sum_{i=1}^{n}\left(y_i - \hat{y}_i\right)^{2}$$

5. Give the closed-form OLS estimate of the slope $\beta_1$ in simple linear regression.

   $$\beta_1 = \frac{\sum_{i=1}^{n}(x_i-\bar{x})(y_i-\bar{y})}{\sum_{i=1}^{n}(x_i-\bar{x})^{2}}$$ and $\beta_0 = \bar{y} - \beta_1\bar{x}$.

6. What is the multiple linear regression model in matrix form?

   $$\vec{y} = X\vec{\beta} + \vec{\varepsilon}$$ where $X$ is the $n\times(p+1)$ design matrix including a column of ones for the intercept.

7. State the normal equations / closed-form solution for multiple linear regression.

   $$\hat{\vec{\beta}} = (X^{T}X)^{-1}X^{T}\vec{y}$$

8. What does the coefficient of determination $R^{2}$ measure?

   $R^{2}$ measures the proportion of variance in $y$ explained by the model: $$R^{2} = 1 - \frac{\sum_i (y_i-\hat{y}_i)^{2}}{\sum_i (y_i-\bar{y})^{2}}$$

9. What problem does ridge regression address and how?

   Ridge addresses multicollinearity and overfitting by adding an $L_2$ penalty on coefficient magnitudes, shrinking them toward zero (but not exactly to zero).

10. State the ridge regression objective function.

    $$J(\vec{\beta}) = \sum_{i=1}^{n}(y_i - \vec{x}_i^{T}\vec{\beta})^{2} + \lambda \sum_{j=1}^{p}\beta_j^{2}$$ where $\lambda \geq 0$ is the regularization strength.

11. Give the closed-form solution for ridge regression coefficients.

    $$\hat{\vec{\beta}}_{ridge} = (X^{T}X + \lambda I)^{-1}X^{T}\vec{y}$$

12. How does ridge ($L_2$) regularization differ from lasso ($L_1$) in terms of coefficients?

    Ridge ($L_2$) shrinks coefficients smoothly toward but rarely exactly zero (no feature selection); lasso ($L_1$) can drive coefficients exactly to zero, performing feature selection.

13. What is a classification problem?

    A supervised task of assigning an input $\vec{x}$ to one of $K$ discrete classes; binary if $K=2$, multiclass if $K>2$.

14. What function does logistic regression use to model the probability of the positive class?

    The sigmoid (logistic) function: $$P(y=1\mid \vec{x}) = \sigma(\vec{w}^{T}\vec{x}+b) = \frac{1}{1+e^{-(\vec{w}^{T}\vec{x}+b)}}$$

15. What loss function is minimized when training logistic regression?

    The binary cross-entropy (log loss): $$J = -\frac{1}{n}\sum_{i=1}^{n}\left[y_i\log\hat{p}_i + (1-y_i)\log(1-\hat{p}_i)\right]$$

16. What is the log-odds (logit) expressed by logistic regression?

    $$\log\!\left(\frac{p}{1-p}\right) = \vec{w}^{T}\vec{x} + b$$ i.e. the log-odds is a linear function of the inputs.

17. How does the K-Nearest Neighbour (KNN) algorithm classify a new point?

    It finds the $k$ closest training points (by a distance metric) and assigns the majority class among those neighbours (for regression, it averages their values).

18. Why is KNN called a lazy / non-parametric learner?

    It stores all training data and does no explicit training phase; all computation (distance calculation) is deferred to prediction time, and it makes no assumption about the data distribution.

19. What distance metric is most commonly used in KNN, and give its formula.

    Euclidean distance: $$d(\vec{x},\vec{x}') = \sqrt{\sum_{j=1}^{p}(x_j - x'_j)^{2}}$$

20. What is the effect of choosing a very small versus very large $k$ in KNN?

    Small $k$ gives a flexible, low-bias but high-variance model (noisy, overfits); large $k$ gives a smoother, high-bias but low-variance model (may underfit).

21. What assumption gives the Naive Bayes classifier its name?

    It naively assumes all features are conditionally independent given the class label.

22. State the classification rule used by Naive Bayes.

    $$\hat{y} = \arg\max_{c}\; P(c)\prod_{j=1}^{p}P(x_j\mid c)$$ choosing the class maximizing the posterior via Bayes' theorem.

23. Write Bayes' theorem as used in Naive Bayes.

    $$P(c\mid \vec{x}) = \frac{P(\vec{x}\mid c)\,P(c)}{P(\vec{x})}$$

24. What is Laplace (additive) smoothing in Naive Bayes and why is it used?

    It adds a constant $\alpha$ (often 1) to counts: $P(x_j\mid c)=\frac{N_{jc}+\alpha}{N_c + \alpha V}$, preventing zero probabilities for feature values unseen in training.

25. What is the goal of Linear Discriminant Analysis (LDA)?

    To find a linear projection that maximizes between-class separation while minimizing within-class scatter, used for classification and dimensionality reduction.

See more Machine Learning flashcards →

Planning Machine Learning for GATE DA & AI Engineering

Machine Learning (GATE DA & AI Engineering) FAQ