Examius

🇮🇳 Data Science & Machine Learning · subject

Data Science & Machine Learning Supervised Machine Learning Syllabus

The Supervised Machine Learning syllabus for Data Science & Machine Learning in full: 5 chapters, 17 topics and 35 sub-topics, every one listed below, plus 51 Supervised Machine Learning flashcards with answers.

5Chapters
17Topics
35Sub-topics
51Flashcards

Supervised Machine Learning syllabus: all 5 chapters

Expand any chapter to see its topics and sub-topics.

  1. Machine Learning Foundations

     3 topics
    • Types of Learning
      • Supervised vs unsupervised
      • Reinforcement learning overview
    • The ML Pipeline
      • Train-test split
      • Bias-variance tradeoff
      • Overfitting and underfitting
    • scikit-learn API Design
      • Estimator, fit and predict
      • Transformers and pipelines

  2. Regression Algorithms

     3 topics
    • Linear Regression
      • Ordinary least squares
      • Assumptions of linear regression
    • Polynomial Regression
      • Feature expansion
    • Regularized Regression
      • Ridge (L2)
      • Lasso (L1)
      • Elastic Net

  3. Classification Algorithms

     4 topics
    • Logistic Regression
      • Sigmoid and decision boundary
      • Multiclass with softmax
    • K-Nearest Neighbors
      • Distance metrics
      • Choosing k
    • Naive Bayes
      • Gaussian and multinomial variants
    • Support Vector Machines
      • Margin and support vectors
      • Kernel trick

  4. Tree-Based and Ensemble Methods

     3 topics
    • Decision Trees
      • Gini and entropy
      • Pruning
    • Bagging and Random Forest
      • Bootstrap aggregation
      • Feature importance
    • Boosting
      • AdaBoost
      • Gradient Boosting
      • XGBoost and LightGBM

  5. Model Evaluation and Tuning

     4 topics
    • Classification Metrics
      • Accuracy, precision and recall
      • F1-score and confusion matrix
      • ROC-AUC
    • Regression Metrics
      • MAE, MSE and RMSE
      • R-squared
    • Cross-Validation
      • K-fold and stratified CV
    • Hyperparameter Tuning
      • Grid search
      • Randomized search

Data Science & Machine Learning Supervised Machine Learning flashcards — 51 questions with answers

  1. What are the three main types of machine learning based on the feedback signal available during training?

    Supervised learning (labeled data), unsupervised learning (unlabeled data, find structure), and reinforcement learning (learn from rewards/penalties via interaction).

  2. In supervised learning, what is the difference between a classification and a regression task?

    Classification predicts a discrete categorical label; regression predicts a continuous numeric value.

  3. What distinguishes supervised learning from unsupervised learning?

    Supervised learning uses input-output pairs (labeled data) to learn a mapping; unsupervised learning uses only inputs (unlabeled) to find patterns like clusters or structure.

  4. List the typical stages of the machine learning pipeline in order.

    1) Data collection, 2) Data cleaning/preprocessing, 3) Feature engineering, 4) Train/test split, 5) Model selection & training, 6) Evaluation, 7) Hyperparameter tuning, 8) Deployment/monitoring.

  5. Why must feature scaling (fit) be done only on the training set, then applied to the test set?

    To prevent data leakage; fitting the scaler on the full dataset leaks test-set statistics into training, giving overly optimistic, biased evaluation.

  6. In scikit-learn's API, what do the fit(), predict(), and transform() methods do?

    fit() learns parameters from data; predict() outputs predictions for estimators; transform() applies a learned transformation for transformers.

  7. What is the purpose of a scikit-learn Pipeline?

    It chains preprocessing steps and a final estimator into one object, ensuring transformations are fit only on training folds, preventing leakage and simplifying cross-validation/tuning.

  8. In scikit-learn, what is the difference between fit_transform() and transform()?

    fit_transform() learns parameters and applies the transform (used on training data); transform() only applies previously learned parameters (used on validation/test data).

  9. What is the hypothesis function (model equation) for simple linear regression?

    ŷ = β₀ + β₁x, where β₀ is the intercept and β₁ is the slope/coefficient.

  10. What cost function does ordinary least squares (OLS) linear regression minimize?

    The Mean Squared Error / sum of squared residuals: minimize Σ(yᵢ − ŷᵢ)².

  11. What are the key assumptions of linear regression?

    Linearity, independence of errors, homoscedasticity (constant error variance), normally distributed errors, and little multicollinearity among predictors.

  12. What is polynomial regression and is it a linear model?

    It fits a polynomial (e.g., y = β₀ + β₁x + β₂x² + ...) by adding powers of features. It is still linear in the parameters, so it is a linear model fit with linear methods.

  13. What problem arises from using a high-degree polynomial in polynomial regression?

    Overfitting — the model fits noise in the training data, has high variance, and generalizes poorly to new data.

  14. How does Ridge regression (L2) modify the linear regression cost function?

    It adds an L2 penalty: minimize RSS + α·Σβⱼ². This shrinks coefficients toward zero but does not set them exactly to zero.

  15. How does Lasso regression (L1) differ from Ridge in its effect on coefficients?

    Lasso adds an L1 penalty (α·Σ|βⱼ|) which can shrink coefficients exactly to zero, performing automatic feature selection; Ridge only shrinks them close to zero.

  16. What is Elastic Net regression?

    A regularized regression combining both L1 (Lasso) and L2 (Ridge) penalties, controlled by a mixing ratio; useful with correlated features and for feature selection.

  17. In regularized regression, what does the hyperparameter alpha (λ) control?

    The strength of regularization: larger alpha increases penalty (more shrinkage, higher bias, lower variance); alpha = 0 reduces to ordinary least squares.

  18. What function does logistic regression use to map outputs to probabilities?

    The sigmoid (logistic) function: σ(z) = 1 / (1 + e^(−z)), which outputs values between 0 and 1.

  19. Is logistic regression a regression or classification algorithm, and what does it predict?

    It is a classification algorithm; it models the probability of class membership and applies a threshold (default 0.5) to assign a class.

  20. What loss function is minimized when training logistic regression?

    Log loss (binary cross-entropy): −Σ[y·log(p) + (1−y)·log(1−p)].

Show the other 31 Supervised Machine Learning flashcards

  1. How does the K-Nearest Neighbors (KNN) algorithm make a prediction?

    It finds the K closest training points (by a distance metric) to the query and predicts the majority class (classification) or average value (regression).

  2. Why is KNN called a lazy / instance-based learner?

    It does no explicit training; it stores all training data and defers computation to prediction time, comparing the query against stored instances.

  3. How does the choice of K affect a KNN model's bias-variance tradeoff?

    Small K → low bias, high variance (sensitive to noise); large K → higher bias, lower variance (smoother decision boundary).

  4. What core assumption gives Naive Bayes its name?

    The 'naive' assumption that all features are conditionally independent given the class label.

  5. What theorem is Naive Bayes based on, and what is its formula?

    Bayes' theorem: P(class|features) = P(features|class)·P(class) / P(features).

  6. Name the common Naive Bayes variants and when each is used.

    Gaussian NB (continuous features assumed normal), Multinomial NB (count data like word frequencies), Bernoulli NB (binary/boolean features).

  7. What is the goal of a Support Vector Machine (SVM) for classification?

    To find the hyperplane that maximizes the margin — the distance between the decision boundary and the nearest data points (support vectors) of each class.

  8. What are support vectors in an SVM?

    The training points closest to the decision boundary (on or within the margin) that determine the position and orientation of the hyperplane.

  9. What is the kernel trick in SVMs?

    It implicitly maps data into a higher-dimensional space using a kernel function (e.g., RBF, polynomial) to find a linear separator without computing the transformation explicitly.

  10. In SVM, what does the regularization parameter C control?

    The trade-off between maximizing the margin and minimizing misclassification: large C → less regularization (narrow margin, fewer errors); small C → more regularization (wider margin, more tolerance for errors).

  11. How does a decision tree decide where to split a node?

    It chooses the feature and threshold that best reduce impurity, measured by Gini impurity or entropy/information gain (classification) or variance reduction (regression).

  12. What is the formula for Gini impurity?

    Gini = 1 − Σ pᵢ², where pᵢ is the proportion of class i in the node. It is 0 for a pure node.

  13. Why are decision trees prone to overfitting, and how is it controlled?

    Deep trees can memorize the training data. It is controlled by pruning or hyperparameters like max_depth, min_samples_split, min_samples_leaf, and max_leaf_nodes.

  14. What is bagging (bootstrap aggregating)?

    An ensemble method that trains many models on bootstrap (random with-replacement) samples of the data and aggregates their predictions (voting/averaging) to reduce variance.

  15. How does a Random Forest extend basic bagging of decision trees?

    In addition to bootstrap sampling, it considers only a random subset of features at each split, decorrelating the trees and further reducing variance.

  16. What is the difference between bagging and boosting in terms of how models are trained?

    Bagging trains models independently in parallel to reduce variance; boosting trains models sequentially, each correcting the errors of the previous, to reduce bias.

  17. How does AdaBoost work?

    It trains weak learners sequentially, increasing the weights of misclassified samples each round so later learners focus on hard cases, then combines them as a weighted vote.

  18. What does Gradient Boosting do at each stage?

    It fits a new weak learner to the residual errors (negative gradient of the loss) of the current ensemble, then adds it scaled by a learning rate to improve predictions.

  19. In a confusion matrix, what are TP, TN, FP, and FN?

    TP = correctly predicted positive; TN = correctly predicted negative; FP = negative predicted as positive (Type I error); FN = positive predicted as negative (Type II error).

  20. What is the formula for accuracy and when is it misleading?

    Accuracy = (TP + TN) / (TP + TN + FP + FN). It is misleading on imbalanced datasets where a trivial majority-class predictor scores high.

  21. Define precision and recall with their formulas.

    Precision = TP / (TP + FP) (of predicted positives, how many are correct). Recall = TP / (TP + FN) (of actual positives, how many were found).

  22. What is the F1 score and why is it used?

    F1 = 2·(precision·recall)/(precision+recall), the harmonic mean of precision and recall; it balances the two, useful for imbalanced classes.

  23. What does the ROC curve plot and what does AUC represent?

    ROC plots True Positive Rate vs False Positive Rate across thresholds. AUC (area under the curve) is the probability the model ranks a random positive above a random negative; 1.0 is perfect, 0.5 is random.

  24. Give the formulas for MAE and MSE in regression.

    MAE = (1/n)·Σ|yᵢ − ŷᵢ|; MSE = (1/n)·Σ(yᵢ − ŷᵢ)². MSE penalizes large errors more heavily.

  25. What is RMSE and why is it often preferred over MSE for reporting?

    RMSE = √MSE. It is in the same units as the target variable, making it more interpretable than MSE.

  26. What does the R² (coefficient of determination) metric measure?

    The proportion of variance in the target explained by the model: R² = 1 − (SS_residual / SS_total). 1 is perfect; 0 means no better than predicting the mean; it can be negative.

  27. What is k-fold cross-validation?

    The data is split into k folds; the model is trained on k−1 folds and validated on the remaining fold, repeated k times so each fold is the validation set once; results are averaged.

  28. Why use cross-validation instead of a single train/test split?

    It gives a more robust, lower-variance estimate of model performance by using all data for both training and validation, reducing dependence on one particular split.

  29. What is stratified k-fold cross-validation and when is it important?

    It preserves the class distribution (proportions) within each fold; important for imbalanced classification so every fold is representative.

  30. What is the difference between grid search and random search for hyperparameter tuning?

    Grid search exhaustively tries all combinations in a defined grid; random search samples random combinations, often finding good values faster in high-dimensional spaces.

  31. Why must hyperparameter tuning be evaluated with cross-validation or a validation set rather than the test set?

    Tuning on the test set leaks information and overfits hyperparameters to it, giving an optimistically biased estimate; the test set must stay untouched for final unbiased evaluation.

Data Science & Machine Learning Supervised Machine Learning FAQ

What is in the Data Science & Machine Learning Supervised Machine Learning syllabus?

Data Science & Machine Learning Supervised Machine Learning is split into 5 chapters: Machine Learning Foundations, Regression Algorithms, Classification Algorithms, Tree-Based and Ensemble Methods and Model Evaluation and Tuning. Together they contain 17 topics and 35 sub-topics.

How many chapters are in Data Science & Machine Learning Supervised Machine Learning?

5 chapters, containing 17 topics and 35 sub-topics. The largest chapter is Classification Algorithms, with 4 topics.

Are there flashcards for Data Science & Machine Learning Supervised Machine Learning?

Yes — a 51-card Supervised Machine Learning deck, printed in full in the flashcards section of this page and available in the Examius app with spaced repetition. The free plan covers one exam of your choice, including its flashcards; Examius Pro, a one-time purchase, unlocks every other exam and deck.