Data Science & Machine Learning Unsupervised Learning and Feature Engineering Syllabus

Unsupervised Learning and Feature Engineering syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Unsupervised Learning and Feature Engineering in Data Science & Machine Learning, not a summary of it.

1. 1

   Clustering Techniques

   3 topics

   • K-Means Clustering
     • Centroid initialization
     • Elbow method and silhouette score
   • Hierarchical Clustering
     • Agglomerative approach
     • Dendrograms
   • Density-Based Clustering
     • DBSCAN
2. 2

   Dimensionality Reduction

   3 topics

   • Principal Component Analysis
     • Explained variance
     • Components selection
   • t-SNE and UMAP
     • Nonlinear embedding for visualization
   • Association Rule Mining
     • Apriori algorithm
     • Support, confidence and lift
3. 3

   Feature Engineering

   4 topics

   • Feature Scaling
     • Standardization and normalization
     • MinMax and Robust scalers
   • Encoding Categorical Variables
     • One-hot encoding
     • Label and target encoding
   • Feature Creation and Selection
     • Binning and interaction features
     • Filter, wrapper and embedded methods
   • Handling Imbalanced Data
     • SMOTE and oversampling
     • Class weights

Unsupervised Learning and Feature Engineering flashcards for Data Science & Machine Learning

22 of 51 cards from the Unsupervised Learning and Feature Engineering deck — real questions with worked answers.

1. What is the objective function that K-Means clustering minimizes?

   The within-cluster sum of squares (WCSS), also called inertia: the sum of squared Euclidean distances between each point and its assigned cluster centroid, summed over all clusters.

2. Describe the two iterative steps of the K-Means (Lloyd's) algorithm.

   1) Assignment step: assign each point to the nearest centroid. 2) Update step: recompute each centroid as the mean of the points assigned to it. Repeat until assignments no longer change (convergence).

3. What is the 'elbow method' used for in K-Means?

   It selects the optimal number of clusters K by plotting WCSS (inertia) against K and choosing the K at the 'elbow' point, where adding more clusters yields diminishing reductions in WCSS.

4. What problem does K-Means++ solve, and how?

   It improves centroid initialization to avoid poor local optima. It picks the first centroid randomly, then chooses each subsequent centroid with probability proportional to its squared distance from the nearest existing centroid, spreading initial centers apart.

5. Name three key assumptions/limitations of K-Means clustering.

   It assumes clusters are spherical and roughly equal in size, requires K to be specified in advance, is sensitive to outliers and feature scaling, and only finds convex (linearly separable) clusters.

6. What does the silhouette score measure and what is its range?

   It measures how well a point fits its own cluster versus the nearest neighboring cluster. It ranges from -1 to +1; values near +1 indicate well-clustered points, near 0 indicate overlapping clusters, and negative values indicate likely misassignment.

7. What is the silhouette coefficient formula for a single point?

   s = (b - a) / max(a, b), where a is the mean distance to other points in the same cluster (intra-cluster) and b is the mean distance to points in the nearest other cluster (inter-cluster).

8. What is the difference between agglomerative and divisive hierarchical clustering?

   Agglomerative is bottom-up: each point starts as its own cluster and pairs are merged successively. Divisive is top-down: all points start in one cluster that is recursively split.

9. What is a dendrogram in hierarchical clustering?

   A tree diagram that records the sequence of merges (or splits) and the distance at which they occur. Cutting it at a chosen height yields a particular number of clusters.

10. List the four common linkage criteria in agglomerative clustering.

    Single linkage (minimum pairwise distance), complete linkage (maximum pairwise distance), average linkage (mean pairwise distance), and Ward's linkage (minimizes increase in within-cluster variance).

11. What is the 'chaining effect' and which linkage causes it?

    Chaining is the tendency to form long, straggly clusters where points are linked through a chain of close neighbors. It is caused by single linkage, which uses the minimum distance between clusters.

12. What is the typical time complexity of standard agglomerative hierarchical clustering?

    O(n^3) time (or O(n^2 log n) with optimized implementations) and O(n^2) space, which makes it impractical for very large datasets.

13. What do the two core parameters epsilon (eps) and minPts mean in DBSCAN?

    eps is the radius of the neighborhood around a point; minPts is the minimum number of points (including the point itself) required within that radius for a point to be a core point.

14. Define core point, border point, and noise point in DBSCAN.

    A core point has at least minPts neighbors within eps. A border point has fewer than minPts neighbors but lies within eps of a core point. A noise point is neither core nor border and is treated as an outlier.

15. Name two major advantages of DBSCAN over K-Means.

    DBSCAN does not require specifying the number of clusters in advance, can find arbitrarily shaped (non-convex) clusters, and identifies outliers as noise rather than forcing them into clusters.

16. What is a key weakness of DBSCAN regarding cluster density?

    DBSCAN uses a single global eps/minPts, so it struggles with clusters of widely varying densities; HDBSCAN was developed to address this by handling variable density.

17. What is the main goal of Principal Component Analysis (PCA)?

    To reduce dimensionality by projecting data onto a smaller set of orthogonal axes (principal components) that capture the maximum possible variance in the data.

18. How are principal components mathematically derived?

    They are the eigenvectors of the data's covariance matrix (or via SVD of the centered data matrix); the corresponding eigenvalues give the variance explained by each component, ordered from largest to smallest.

19. Why must data be standardized (scaled) before applying PCA?

    PCA is variance-based, so features with larger scales/units would dominate the principal components. Standardizing to zero mean and unit variance ensures each feature contributes fairly.

20. What does 'explained variance ratio' tell you in PCA?

    It is the proportion of total variance captured by each principal component (eigenvalue divided by the sum of all eigenvalues), used to decide how many components to retain.

21. Is PCA a linear or non-linear technique, and what does that imply?

    PCA is linear: it finds linear combinations of features. It cannot capture non-linear structure, so manifold methods like t-SNE, UMAP, or kernel PCA are needed for non-linear relationships.

22. What does the acronym t-SNE stand for?

    t-distributed Stochastic Neighbor Embedding.

See more Unsupervised Learning and Feature Engineering flashcards →

Planning Unsupervised Learning and Feature Engineering for Data Science & Machine Learning

Unsupervised Learning and Feature Engineering (Data Science & Machine Learning) FAQ