Data Science & Machine Learning Supervised Machine Learning Syllabus

Supervised Machine Learning syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Supervised Machine Learning in Data Science & Machine Learning, not a summary of it.

1. 1

   Machine Learning Foundations

   3 topics

   • Types of Learning
     • Supervised vs unsupervised
     • Reinforcement learning overview
   • The ML Pipeline
     • Train-test split
     • Bias-variance tradeoff
     • Overfitting and underfitting
   • scikit-learn API Design
     • Estimator, fit and predict
     • Transformers and pipelines
2. 2

   Regression Algorithms

   3 topics

   • Linear Regression
     • Ordinary least squares
     • Assumptions of linear regression
   • Polynomial Regression
     • Feature expansion
   • Regularized Regression
     • Ridge (L2)
     • Lasso (L1)
     • Elastic Net
3. 3

   Classification Algorithms

   4 topics

   • Logistic Regression
     • Sigmoid and decision boundary
     • Multiclass with softmax
   • K-Nearest Neighbors
     • Distance metrics
     • Choosing k
   • Naive Bayes
     • Gaussian and multinomial variants
   • Support Vector Machines
     • Margin and support vectors
     • Kernel trick
4. 4

   Tree-Based and Ensemble Methods

   3 topics

   • Decision Trees
     • Gini and entropy
     • Pruning
   • Bagging and Random Forest
     • Bootstrap aggregation
     • Feature importance
   • Boosting
     • AdaBoost
     • Gradient Boosting
     • XGBoost and LightGBM
5. 5

   Model Evaluation and Tuning

   4 topics

   • Classification Metrics
     • Accuracy, precision and recall
     • F1-score and confusion matrix
     • ROC-AUC
   • Regression Metrics
     • MAE, MSE and RMSE
     • R-squared
   • Cross-Validation
     • K-fold and stratified CV
   • Hyperparameter Tuning
     • Grid search
     • Randomized search

Supervised Machine Learning flashcards for Data Science & Machine Learning

22 of 51 cards from the Supervised Machine Learning deck — real questions with worked answers.

1. What are the three main types of machine learning based on the feedback signal available during training?

   Supervised learning (labeled data), unsupervised learning (unlabeled data, find structure), and reinforcement learning (learn from rewards/penalties via interaction).

2. In supervised learning, what is the difference between a classification and a regression task?

   Classification predicts a discrete categorical label; regression predicts a continuous numeric value.

3. What distinguishes supervised learning from unsupervised learning?

   Supervised learning uses input-output pairs (labeled data) to learn a mapping; unsupervised learning uses only inputs (unlabeled) to find patterns like clusters or structure.

4. List the typical stages of the machine learning pipeline in order.

   1) Data collection, 2) Data cleaning/preprocessing, 3) Feature engineering, 4) Train/test split, 5) Model selection & training, 6) Evaluation, 7) Hyperparameter tuning, 8) Deployment/monitoring.

5. Why must feature scaling (fit) be done only on the training set, then applied to the test set?

   To prevent data leakage; fitting the scaler on the full dataset leaks test-set statistics into training, giving overly optimistic, biased evaluation.

6. In scikit-learn's API, what do the fit(), predict(), and transform() methods do?

   fit() learns parameters from data; predict() outputs predictions for estimators; transform() applies a learned transformation for transformers.

7. What is the purpose of a scikit-learn Pipeline?

   It chains preprocessing steps and a final estimator into one object, ensuring transformations are fit only on training folds, preventing leakage and simplifying cross-validation/tuning.

8. In scikit-learn, what is the difference between fit_transform() and transform()?

   fit_transform() learns parameters and applies the transform (used on training data); transform() only applies previously learned parameters (used on validation/test data).

9. What is the hypothesis function (model equation) for simple linear regression?

   ŷ = β₀ + β₁x, where β₀ is the intercept and β₁ is the slope/coefficient.

10. What cost function does ordinary least squares (OLS) linear regression minimize?

    The Mean Squared Error / sum of squared residuals: minimize Σ(yᵢ − ŷᵢ)².

11. What are the key assumptions of linear regression?

    Linearity, independence of errors, homoscedasticity (constant error variance), normally distributed errors, and little multicollinearity among predictors.

12. What is polynomial regression and is it a linear model?

    It fits a polynomial (e.g., y = β₀ + β₁x + β₂x² + ...) by adding powers of features. It is still linear in the parameters, so it is a linear model fit with linear methods.

13. What problem arises from using a high-degree polynomial in polynomial regression?

    Overfitting — the model fits noise in the training data, has high variance, and generalizes poorly to new data.

14. How does Ridge regression (L2) modify the linear regression cost function?

    It adds an L2 penalty: minimize RSS + α·Σβⱼ². This shrinks coefficients toward zero but does not set them exactly to zero.

15. How does Lasso regression (L1) differ from Ridge in its effect on coefficients?

    Lasso adds an L1 penalty (α·Σ|βⱼ|) which can shrink coefficients exactly to zero, performing automatic feature selection; Ridge only shrinks them close to zero.

16. What is Elastic Net regression?

    A regularized regression combining both L1 (Lasso) and L2 (Ridge) penalties, controlled by a mixing ratio; useful with correlated features and for feature selection.

17. In regularized regression, what does the hyperparameter alpha (λ) control?

    The strength of regularization: larger alpha increases penalty (more shrinkage, higher bias, lower variance); alpha = 0 reduces to ordinary least squares.

18. What function does logistic regression use to map outputs to probabilities?

    The sigmoid (logistic) function: σ(z) = 1 / (1 + e^(−z)), which outputs values between 0 and 1.

19. Is logistic regression a regression or classification algorithm, and what does it predict?

    It is a classification algorithm; it models the probability of class membership and applies a threshold (default 0.5) to assign a class.

20. What loss function is minimized when training logistic regression?

    Log loss (binary cross-entropy): −Σ[y·log(p) + (1−y)·log(1−p)].

21. How does the K-Nearest Neighbors (KNN) algorithm make a prediction?

    It finds the K closest training points (by a distance metric) to the query and predicts the majority class (classification) or average value (regression).

22. Why is KNN called a lazy / instance-based learner?

    It does no explicit training; it stores all training data and defers computation to prediction time, comparing the query against stored instances.

See more Supervised Machine Learning flashcards →

Planning Supervised Machine Learning for Data Science & Machine Learning

Supervised Machine Learning (Data Science & Machine Learning) FAQ