Data Science Machine Learning Syllabus

Machine Learning syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Machine Learning in Data Science, not a summary of it.

1. 1

   Introduction to Machine Learning

   2 topics

   • Overview of Machine Learning
     • Definition of Machine Learning
     • Types of Machine Learning
     • Applications of Machine Learning
     • History of Machine Learning
   • Basic Concepts
     • Supervised Learning
     • Unsupervised Learning
     • Reinforcement Learning
     • Semi-Supervised Learning
2. 2

   Mathematics for Machine Learning

   4 topics

   • Linear Algebra
     • Vectors and Matrices
     • Matrix Operations
     • Eigenvalues and Eigenvectors
     • Singular Value Decomposition
   • Calculus
     • Differentiation
     • Integration
     • Partial Derivatives
     • Gradient Descent
   • Probability and Statistics
     • Probability Theory
     • Random Variables
     • Distributions
     • Bayesian Statistics
   • Linear Regression
     • Simple Linear Regression
     • Multiple Linear Regression
     • Assumptions of Linear Regression
     • Regularization Techniques
3. 3

   Data Preprocessing

   3 topics

   • Data Cleaning
     • Handling Missing Values
     • Handling Outliers
     • Data Imputation
   • Feature Engineering
     • Feature Selection
     • Feature Extraction
     • Feature Scaling
   • Data Transformation
     • Normalization
     • Standardization
     • Encoding Categorical Variables
4. 4

   Supervised Learning

   2 topics

   • Classification
     • Logistic Regression
     • K-Nearest Neighbors
     • Support Vector Machines
     • Decision Trees
     • Random Forests
     • Gradient Boosting Machines
     • Naive Bayes
   • Regression
     • Linear Regression
     • Polynomial Regression
     • Ridge Regression
     • Lasso Regression
     • Elastic Net
5. 5

   Unsupervised Learning

   3 topics

   • Clustering
     • K-Means Clustering
     • Hierarchical Clustering
     • DBSCAN
     • Gaussian Mixture Models
   • Dimensionality Reduction
     • Principal Component Analysis
     • t-Distributed Stochastic Neighbor Embedding
     • Linear Discriminant Analysis
     • Independent Component Analysis
   • Association Rule Learning
     • Apriori Algorithm
     • Eclat Algorithm
6. 6

   Reinforcement Learning

   2 topics

   • Basics of Reinforcement Learning
     • Markov Decision Processes
     • Policy Learning
     • Value Learning
   • Advanced sub-topics
     • Q-Learning
     • Deep Q-Networks
     • Policy Gradient Methods
7. 7

   Model Evaluation and Optimization

   2 topics

   • Model Evaluation
     • Confusion Matrix
     • Precision, Recall, and F1 Score
     • ROC and AUC
     • Cross-Validation
   • Model Optimization
     • Hyperparameter Tuning
     • Grid Search
     • Random Search
     • Bayesian Optimization
8. 8

   Deep Learning

   4 topics

   • Neural Networks
     • Perceptrons
     • Feedforward Neural Networks
     • Activation Functions
     • Backpropagation
   • Convolutional Neural Networks
     • Convolutional Layers
     • Pooling Layers
     • Architectures (e.g., AlexNet, VGG, ResNet)
   • Recurrent Neural Networks
     • Vanilla RNNs
     • Long Short-Term Memory (LSTM)
     • Gated Recurrent Units (GRU)
   • Advanced sub-topics in Deep Learning
     • Generative Adversarial Networks (GANs)
     • Autoencoders
     • Transfer Learning
9. 9

   Natural Language Processing

   3 topics

   • Text Preprocessing
     • Tokenization
     • Stemming and Lemmatization
     • Stop Words Removal
   • Text Representation
     • Bag of Words
     • TF-IDF
     • Word Embeddings (Word2Vec, GloVe)
   • NLP Tasks
     • Text Classification
     • Sentiment Analysis
     • Named Entity Recognition
     • Machine Translation
10. 10

    Time Series Analysis

    2 topics

    • Basics of Time Series
      • Time Series Components
      • Stationarity
      • Autocorrelation
    • Time Series Models
      • ARIMA
      • SARIMA
      • Exponential Smoothing
      • Prophet
11. 11

    Deployment and Production

    2 topics

    • Model Deployment
      • Saving and Loading Models
      • APIs for Model Serving
      • Cloud Services (AWS, GCP, Azure)
    • Monitoring and Maintenance
      • Model Performance Monitoring
      • A/B Testing
      • Retraining Models

Machine Learning flashcards for Data Science

25 of 56 cards from the Machine Learning deck — real questions with worked answers.

1. What is Machine Learning?

   A subfield of artificial intelligence in which systems learn patterns from data to make predictions or decisions without being explicitly programmed for the task. Formally (Tom Mitchell): a program learns from experience $E$ with respect to task $T$ and performance measure $P$ if its performance at $T$, measured by $P$, improves with $E$.

2. What are the three main types of machine learning?

   Supervised learning (labeled data, learns input-to-output mapping), unsupervised learning (unlabeled data, finds structure/patterns), and reinforcement learning (an agent learns via rewards from interacting with an environment).

3. What is the difference between supervised classification and regression?

   Both are supervised. Classification predicts a discrete categorical label (e.g., spam/not-spam), while regression predicts a continuous numerical value (e.g., house price).

4. Define the bias-variance tradeoff.

   Expected test error decomposes as $\text{Error} = \text{Bias}^{2} + \text{Variance} + \sigma^{2}$. High bias (underfitting) means the model is too simple; high variance (overfitting) means it is too sensitive to training data. Reducing one tends to increase the other.

5. What is overfitting and how is it typically detected?

   Overfitting occurs when a model learns noise/specifics of the training data, giving low training error but high test error. It is detected when training performance is much better than validation/test performance.

6. What is the difference between a parameter and a hyperparameter?

   A parameter is learned from data during training (e.g., weights $w$). A hyperparameter is set before training and controls the learning process (e.g., learning rate $\alpha$, number of trees, regularization strength $\lambda$).

7. What is a dot product of two vectors $\vec{a}$ and $\vec{b}$, and what does it equal geometrically?

   $\vec{a} \cdot \vec{b} = \sum_{i=1}^{n} a_i b_i = \lVert \vec{a} \rVert \, \lVert \vec{b} \rVert \cos\theta$, where $\theta$ is the angle between them. It is zero when the vectors are orthogonal.

8. What does it mean for a matrix to be invertible, and what is the condition involving its determinant?

   A square matrix $A$ is invertible (non-singular) if there exists $A^{-1}$ such that $AA^{-1} = I$. This holds if and only if $\det(A) \neq 0$.

9. What are eigenvalues and eigenvectors of a matrix $A$?

   A nonzero vector $\vec{v}$ is an eigenvector of $A$ with eigenvalue $\lambda$ if $A\vec{v} = \lambda \vec{v}$. They describe directions that are only scaled (not rotated) by the linear transformation.

10. Define the $L_1$ and $L_2$ norms of a vector $\vec{x}$.

    $L_1$ norm: $\lVert \vec{x} \rVert_1 = \sum_{i=1}^{n} |x_i|$ (Manhattan). $L_2$ norm: $\lVert \vec{x} \rVert_2 = \sqrt{\sum_{i=1}^{n} x_i^{2}}$ (Euclidean).

11. What is the derivative used for in machine learning optimization?

    The derivative (gradient) gives the slope/direction of steepest increase of a function. Gradient descent moves in the opposite direction of the gradient to minimize a loss function.

12. What is the gradient of a multivariable function $f(x_1, \dots, x_n)$?

    $\nabla f = \left( \frac{\partial f}{\partial x_1}, \frac{\partial f}{\partial x_2}, \dots, \frac{\partial f}{\partial x_n} \right)$ — the vector of partial derivatives, pointing in the direction of steepest ascent.

13. State the chain rule for the composition $f(g(x))$.

    $\frac{d}{dx} f(g(x)) = f'(g(x)) \cdot g'(x)$. It is the foundation of backpropagation in neural networks.

14. What is the gradient descent parameter update rule?

    $\theta := \theta - \alpha \nabla_{\theta} J(\theta)$, where $\alpha$ is the learning rate and $J(\theta)$ is the cost function. The parameters are moved opposite to the gradient to reduce the cost.

15. State Bayes' theorem.

    $P(A \mid B) = \dfrac{P(B \mid A)\,P(A)}{P(B)}$, where $P(A)$ is the prior, $P(B \mid A)$ the likelihood, and $P(A \mid B)$ the posterior.

16. Define expected value (mean) and variance of a random variable $X$.

    Mean: $\mu = E[X] = \sum_i x_i P(x_i)$ (or $\int x f(x)\,dx$). Variance: $\sigma^{2} = E[(X - \mu)^{2}] = E[X^{2}] - (E[X])^{2}$.

17. What is the probability density function of a normal (Gaussian) distribution?

    $f(x) = \dfrac{1}{\sigma \sqrt{2\pi}} \, e^{-\frac{(x - \mu)^{2}}{2\sigma^{2}}}$, parameterized by mean $\mu$ and standard deviation $\sigma$.

18. What is the difference between covariance and correlation?

    Covariance $\text{Cov}(X,Y) = E[(X-\mu_X)(Y-\mu_Y)]$ measures joint variability in original units. Correlation $\rho = \dfrac{\text{Cov}(X,Y)}{\sigma_X \sigma_Y}$ is the normalized, unitless version bounded in $[-1, 1]$.

19. What does a p-value represent in hypothesis testing?

    The probability of observing a result at least as extreme as the one obtained, assuming the null hypothesis $H_0$ is true. A small p-value (e.g., $< 0.05$) leads to rejecting $H_0$.

20. What is the hypothesis (model) equation for simple linear regression?

    $\hat{y} = \beta_0 + \beta_1 x$, where $\beta_0$ is the intercept and $\beta_1$ is the slope. For multiple features: $\hat{y} = \beta_0 + \sum_{j=1}^{p} \beta_j x_j$.

21. What cost function does linear regression minimize, and what is its formula?

    Mean Squared Error (MSE): $J(\beta) = \dfrac{1}{n} \sum_{i=1}^{n} (y_i - \hat{y}_i)^{2}$. Ordinary Least Squares finds the coefficients that minimize this.

22. What is the closed-form normal equation solution for linear regression?

    $\hat{\beta} = (X^{T} X)^{-1} X^{T} y$, which directly computes the coefficients minimizing the sum of squared errors without iteration.

23. What does the coefficient of determination $R^{2}$ measure?

    The proportion of variance in the target explained by the model: $R^{2} = 1 - \dfrac{\sum (y_i - \hat{y}_i)^{2}}{\sum (y_i - \bar{y})^{2}}$. It ranges from $-\infty$ to $1$, where $1$ is a perfect fit.

24. What are the key assumptions of linear regression?

    Linearity (relationship is linear), independence of errors, homoscedasticity (constant error variance), normality of residuals, and little or no multicollinearity among predictors.

25. What are common strategies for handling missing data?

    Deletion (drop rows/columns), imputation (fill with mean/median/mode), model-based imputation (e.g., KNN or regression), or using a missingness indicator flag. The choice depends on the missingness mechanism (MCAR, MAR, MNAR).

See more Machine Learning flashcards →

Planning Machine Learning for Data Science

Machine Learning (Data Science) FAQ