Machine Learning Core Concepts Syllabus

Core Concepts syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Core Concepts in Machine Learning, not a summary of it.

1. 1

   Data Preprocessing

   3 topics

   • Data Cleaning
     • Handling Missing Values
     • Handling Outliers
     • Data Imputation
   • Feature Engineering
     • Feature Selection
     • Feature Extraction
     • Feature Scaling
   • Data Transformation
     • Normalization
     • Standardization
     • Encoding Categorical Variables
2. 2

   Supervised Learning

   2 topics

   • Classification
     • Logistic Regression
     • K-Nearest Neighbors
     • Support Vector Machines
     • Decision Trees
     • Random Forests
     • Gradient Boosting Machines
     • Naive Bayes
   • Regression
     • Linear Regression
     • Polynomial Regression
     • Ridge Regression
     • Lasso Regression
     • Elastic Net
3. 3

   Unsupervised Learning

   3 topics

   • Clustering
     • K-Means Clustering
     • Hierarchical Clustering
     • DBSCAN
     • Gaussian Mixture Models
   • Dimensionality Reduction
     • Principal Component Analysis
     • t-Distributed Stochastic Neighbor Embedding
     • Linear Discriminant Analysis
     • Independent Component Analysis
   • Association Rule Learning
     • Apriori Algorithm
     • Eclat Algorithm
4. 4

   Reinforcement Learning

   2 topics

   • Basics of Reinforcement Learning
     • Markov Decision Processes
     • Policy Learning
     • Value Learning
   • Advanced sub-topics
     • Q-Learning
     • Deep Q-Networks
     • Policy Gradient Methods
5. 5

   Model Evaluation and Optimization

   2 topics

   • Model Evaluation
     • Confusion Matrix
     • Precision, Recall, and F1 Score
     • ROC and AUC
     • Cross-Validation
   • Model Optimization
     • Hyperparameter Tuning
     • Grid Search
     • Random Search
     • Bayesian Optimization

Core Concepts flashcards for Machine Learning

22 of 51 cards from the Core Concepts deck — real questions with worked answers.

1. What is data cleaning in a machine learning pipeline?

   Data cleaning is the process of detecting and correcting (or removing) corrupt, inaccurate, incomplete, or irrelevant records from a dataset — handling missing values, duplicates, outliers, and inconsistent formatting — so the data is reliable for modeling.

2. Name three common strategies for handling missing values in a dataset.

   (1) Deletion — drop rows or columns with missing data; (2) Imputation — fill with a statistic such as the mean, median, or mode; (3) Model-based imputation — predict the missing value using algorithms like KNN or regression.

3. How is the Interquartile Range (IQR) used to detect outliers?

   Compute $IQR = Q_3 - Q_1$. A value is flagged as an outlier if it lies below $Q_1 - 1.5 \cdot IQR$ or above $Q_3 + 1.5 \cdot IQR$.

4. What is feature engineering?

   Feature engineering is the process of using domain knowledge to create, transform, or select input variables (features) that make machine learning algorithms work better — improving model accuracy by exposing the underlying structure of the data.

5. What is the difference between feature selection and feature extraction?

   Feature selection chooses a subset of the existing original features (e.g., filter, wrapper, embedded methods). Feature extraction creates new features by combining or transforming the originals (e.g., PCA), producing a new, usually lower-dimensional, representation.

6. Compare one-hot encoding and label encoding for categorical variables.

   Label encoding assigns each category an integer (introduces an artificial ordinal relationship, suited to ordinal data or tree models). One-hot encoding creates a binary indicator column per category (no false ordering, but increases dimensionality), suited to nominal data and linear models.

7. State the formula for min-max normalization (feature scaling to $[0,1]$).

   $$x' = \frac{x - x_{\min}}{x_{\max} - x_{\min}}$$

8. State the formula for standardization (z-score normalization) and what it produces.

   $$z = \frac{x - \mu}{\sigma}$$ It rescales a feature to have mean $\mu = 0$ and standard deviation $\sigma = 1$.

9. Why might you apply a log transformation to a feature?

   A log transformation, e.g. $x' = \log(x)$, compresses large values and reduces right-skew, making a heavily skewed distribution more symmetric (closer to normal) and stabilizing variance, which helps models that assume normality or are sensitive to scale.

10. What is the difference between supervised and unsupervised learning?

    Supervised learning trains on labeled data (input-output pairs) to predict a target — e.g., classification and regression. Unsupervised learning finds structure in unlabeled data without target outputs — e.g., clustering, dimensionality reduction, and association rule learning.

11. Define classification in machine learning.

    Classification is a supervised learning task that predicts a discrete categorical label (class) for an input — e.g., spam vs. not-spam. The model learns a decision boundary separating classes.

12. What is the logistic (sigmoid) function used in logistic regression, and what is its output range?

    $$\sigma(z) = \frac{1}{1 + e^{-z}}$$ Its output lies in $(0,1)$ and is interpreted as a probability of the positive class.

13. Write the binary cross-entropy (log loss) used to train a classifier.

    $$L = -\frac{1}{N}\sum_{i=1}^{N}\left[ y_i \log(\hat{y}_i) + (1 - y_i)\log(1 - \hat{y}_i) \right]$$ where $y_i$ is the true label and $\hat{y}_i$ the predicted probability.

14. How does the k-Nearest Neighbors (KNN) algorithm classify a new point?

    It finds the $k$ training points closest to the query (by a distance metric such as Euclidean), then assigns the majority class among those $k$ neighbors. It is a lazy, instance-based, non-parametric method.

15. What is the core idea behind a Support Vector Machine (SVM)?

    An SVM finds the hyperplane that maximizes the margin — the distance between the decision boundary and the nearest data points (support vectors) of each class — giving the most robust separation. The kernel trick lets it separate non-linearly separable data.

16. What does the kernel trick allow an SVM to do?

    The kernel trick computes inner products in a high-dimensional feature space without explicitly mapping the data there, e.g. via the RBF kernel $K(x,x') = \exp(-\gamma \lVert x - x' \rVert^2)$, enabling non-linear decision boundaries efficiently.

17. Define precision and recall.

    $$\text{Precision} = \frac{TP}{TP + FP}, \qquad \text{Recall} = \frac{TP}{TP + FN}$$ Precision is the fraction of predicted positives that are correct; recall is the fraction of actual positives correctly identified.

18. Write the formula for the $F_1$ score and explain why it is used.

    $$F_1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$$ It is the harmonic mean of precision and recall, balancing the two — useful for imbalanced classes where accuracy is misleading.

19. What is a confusion matrix?

    A table comparing predicted vs. actual classes. For binary classification it has four cells: True Positives (TP), False Positives (FP), True Negatives (TN), and False Negatives (FN), from which accuracy, precision, recall, and other metrics are computed.

20. What do the ROC curve and AUC measure?

    The ROC curve plots True Positive Rate against False Positive Rate across classification thresholds. AUC (Area Under the Curve) summarizes it: $AUC = 0.5$ is random guessing, $AUC = 1.0$ is a perfect classifier — it measures ranking/separability independent of threshold.

21. Define regression in machine learning.

    Regression is a supervised learning task that predicts a continuous numeric output (e.g., price, temperature) from input features, by modeling the relationship between inputs and a real-valued target.

22. Write the simple linear regression model equation.

    $$\hat{y} = \beta_0 + \beta_1 x$$ where $\beta_0$ is the intercept and $\beta_1$ is the slope (coefficient) of the feature $x$.

See more Core Concepts flashcards →

Planning Core Concepts for Machine Learning

Core Concepts (Machine Learning) FAQ