Data Science & Machine Learning Unsupervised Learning and Feature Engineering Syllabus

Is PCA a linear or non-linear technique, and what does that imply?

PCA is linear: it finds linear combinations of features. It cannot capture non-linear structure, so manifold methods like t-SNE, UMAP, or kernel PCA are needed for non-linear relationships.

What does the acronym t-SNE stand for?

t-distributed Stochastic Neighbor Embedding.

What is the role of the 'perplexity' parameter in t-SNE?

Perplexity roughly controls the effective number of nearest neighbors considered for each point, balancing attention between local and global structure; typical values range from about 5 to 50.

Why is t-SNE generally used only for visualization, not as preprocessing for other models?

t-SNE preserves local neighborhoods but distorts global distances and densities, is computationally expensive, is non-deterministic, and lacks a stable transform for new (out-of-sample) data.

What does UMAP stand for and what mathematical foundation does it use?

Uniform Manifold Approximation and Projection. It is based on manifold learning and topological data analysis (fuzzy simplicial sets / Riemannian geometry).

Name two practical advantages of UMAP over t-SNE.

UMAP is significantly faster and scales better to large datasets, and it better preserves global structure; it can also transform new data points after fitting.

In t-SNE, why is a Student's t-distribution used in the low-dimensional space?

Its heavier tails alleviate the 'crowding problem' by allowing moderately distant points to be placed farther apart in the embedding, preventing clusters from collapsing together.

What are the three key metrics in association rule mining: support, confidence, and lift?

Support = fraction of transactions containing the itemset. Confidence = P(B|A), the fraction of transactions with A that also contain B. Lift = confidence / support(B), measuring how much more likely B is given A versus independence.

How is lift interpreted in association rule mining?

Lift > 1 means A and B occur together more than expected (positive correlation); lift = 1 means independence; lift < 1 means they occur together less than expected (negative correlation).

What is the Apriori principle (downward closure property)?

If an itemset is frequent, all of its subsets are also frequent; equivalently, if an itemset is infrequent, all its supersets are infrequent. This lets Apriori prune the search space.

How does the FP-Growth algorithm differ from Apriori?

FP-Growth avoids candidate generation by compressing the dataset into an FP-tree and mining frequent patterns recursively, making it generally faster than Apriori, which repeatedly scans the database to generate and test candidates.

What is the formula for standardization (Z-score scaling)?

z = (x - mu) / sigma, where mu is the feature mean and sigma is its standard deviation. The result has mean 0 and standard deviation 1.

What is the formula for Min-Max normalization to the range [0, 1]?

x' = (x - x_min) / (x_max - x_min). It rescales features to a fixed range but is sensitive to outliers.

When should you prefer RobustScaler over StandardScaler?

When the data contains significant outliers. RobustScaler centers using the median and scales using the IQR (interquartile range), making it robust to extreme values.

Name three machine learning algorithms that require feature scaling and one that does not.

Require scaling: K-Means, KNN, SVM, PCA, gradient-descent-based models (logistic/linear regression, neural networks). Do not require it: tree-based models like decision trees, Random Forest, and gradient-boosted trees.

What is one-hot encoding and what is its main drawback?

It converts each category of a nominal variable into a separate binary (0/1) column. Its main drawback is the curse of dimensionality: high-cardinality features create many sparse columns.

What is the difference between label encoding and one-hot encoding?

Label encoding maps each category to an integer (suitable for ordinal data); one-hot creates binary columns per category (suitable for nominal data). Label encoding on nominal data can falsely imply an ordering.

What is target (mean) encoding and what risk does it introduce?

It replaces each category with the mean of the target variable for that category. The main risk is target leakage and overfitting, mitigated by cross-validation folds, smoothing, or adding noise.

What is the 'dummy variable trap' and how is it avoided?

It is perfect multicollinearity arising when one-hot columns are linearly dependent (they sum to 1). It is avoided by dropping one category (drop_first=True), leaving k-1 columns for k categories.

What encoding is appropriate for ordinal categorical variables, and why?

Ordinal encoding, which maps categories to integers that preserve their inherent order (e.g., low=1, medium=2, high=3), because the order carries meaningful information for the model.

What is the difference between feature selection and feature extraction?

Feature selection chooses a subset of the original features (keeping them interpretable); feature extraction creates new features by transforming/combining originals (e.g., PCA), changing the feature space.

Name the three categories of feature selection methods.

Filter methods (use statistical scores like correlation or chi-square, independent of any model), wrapper methods (use a model to evaluate subsets, e.g., recursive feature elimination), and embedded methods (selection built into training, e.g., Lasso, tree importances).

How does Lasso (L1) regularization perform feature selection?

The L1 penalty shrinks some coefficients exactly to zero, effectively removing those features from the model, producing a sparse and automatically feature-selected solution.

What is recursive feature elimination (RFE)?

A wrapper method that fits a model, ranks features by importance, removes the least important feature(s), and repeats the process recursively until the desired number of features remains.

Give two examples of feature creation (feature engineering) techniques.

Examples: creating interaction or polynomial terms, extracting date components (day, month, weekday) from timestamps, binning continuous variables, and computing ratios or aggregations of existing features.

What is the curse of dimensionality?

As the number of features grows, data becomes sparse in the high-dimensional space, distances between points become less meaningful, and models require exponentially more data to generalize, harming performance.

What does SMOTE do to handle imbalanced data?

SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic minority-class examples by interpolating between a minority sample and its k nearest minority-class neighbors, rather than simply duplicating samples.

Why is accuracy a poor metric for imbalanced classification?

With a highly skewed class distribution, a model can achieve high accuracy by predicting only the majority class while completely failing on the minority class. Precision, recall, F1, and AUC-PR are more informative.

Contrast random oversampling and random undersampling for class imbalance.

Random oversampling duplicates minority-class samples (risking overfitting and larger datasets); random undersampling removes majority-class samples (risking loss of useful information and smaller datasets).

What is class weighting as a strategy for imbalanced data?

Assigning higher misclassification cost (weight) to the minority class in the loss function so the model penalizes minority-class errors more heavily, without resampling the data; supported via parameters like class_weight='balanced'.

Which evaluation curve is preferred over ROC for highly imbalanced datasets, and why?

The Precision-Recall curve (and its area, AUC-PR), because it focuses on the positive (minority) class performance and is not misleadingly optimistic when true negatives dominate, unlike the ROC curve.