Data Science & Machine Learning Unsupervised Learning and Feature Engineering Flashcards

24 sample cards from the Unsupervised Learning and Feature Engineering deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. What are the three key metrics in association rule mining: support, confidence, and lift?

   Support = fraction of transactions containing the itemset. Confidence = P(B|A), the fraction of transactions with A that also contain B. Lift = confidence / support(B), measuring how much more likely B is given A versus independence.

2. How is lift interpreted in association rule mining?

   Lift > 1 means A and B occur together more than expected (positive correlation); lift = 1 means independence; lift < 1 means they occur together less than expected (negative correlation).

3. What is the Apriori principle (downward closure property)?

   If an itemset is frequent, all of its subsets are also frequent; equivalently, if an itemset is infrequent, all its supersets are infrequent. This lets Apriori prune the search space.

4. How does the FP-Growth algorithm differ from Apriori?

   FP-Growth avoids candidate generation by compressing the dataset into an FP-tree and mining frequent patterns recursively, making it generally faster than Apriori, which repeatedly scans the database to generate and test candidates.

5. What is the formula for standardization (Z-score scaling)?

   z = (x - mu) / sigma, where mu is the feature mean and sigma is its standard deviation. The result has mean 0 and standard deviation 1.

6. What is the formula for Min-Max normalization to the range [0, 1]?

   x' = (x - x_min) / (x_max - x_min). It rescales features to a fixed range but is sensitive to outliers.

7. When should you prefer RobustScaler over StandardScaler?

   When the data contains significant outliers. RobustScaler centers using the median and scales using the IQR (interquartile range), making it robust to extreme values.

8. Name three machine learning algorithms that require feature scaling and one that does not.

   Require scaling: K-Means, KNN, SVM, PCA, gradient-descent-based models (logistic/linear regression, neural networks). Do not require it: tree-based models like decision trees, Random Forest, and gradient-boosted trees.

9. What is one-hot encoding and what is its main drawback?

   It converts each category of a nominal variable into a separate binary (0/1) column. Its main drawback is the curse of dimensionality: high-cardinality features create many sparse columns.

10. What is the difference between label encoding and one-hot encoding?

    Label encoding maps each category to an integer (suitable for ordinal data); one-hot creates binary columns per category (suitable for nominal data). Label encoding on nominal data can falsely imply an ordering.

11. What is target (mean) encoding and what risk does it introduce?

    It replaces each category with the mean of the target variable for that category. The main risk is target leakage and overfitting, mitigated by cross-validation folds, smoothing, or adding noise.

12. What is the 'dummy variable trap' and how is it avoided?

    It is perfect multicollinearity arising when one-hot columns are linearly dependent (they sum to 1). It is avoided by dropping one category (drop_first=True), leaving k-1 columns for k categories.

13. What encoding is appropriate for ordinal categorical variables, and why?

    Ordinal encoding, which maps categories to integers that preserve their inherent order (e.g., low=1, medium=2, high=3), because the order carries meaningful information for the model.

14. What is the difference between feature selection and feature extraction?

    Feature selection chooses a subset of the original features (keeping them interpretable); feature extraction creates new features by transforming/combining originals (e.g., PCA), changing the feature space.

15. Name the three categories of feature selection methods.

    Filter methods (use statistical scores like correlation or chi-square, independent of any model), wrapper methods (use a model to evaluate subsets, e.g., recursive feature elimination), and embedded methods (selection built into training, e.g., Lasso, tree importances).

16. How does Lasso (L1) regularization perform feature selection?

    The L1 penalty shrinks some coefficients exactly to zero, effectively removing those features from the model, producing a sparse and automatically feature-selected solution.

17. What is recursive feature elimination (RFE)?

    A wrapper method that fits a model, ranks features by importance, removes the least important feature(s), and repeats the process recursively until the desired number of features remains.

18. Give two examples of feature creation (feature engineering) techniques.

    Examples: creating interaction or polynomial terms, extracting date components (day, month, weekday) from timestamps, binning continuous variables, and computing ratios or aggregations of existing features.

19. What is the curse of dimensionality?

    As the number of features grows, data becomes sparse in the high-dimensional space, distances between points become less meaningful, and models require exponentially more data to generalize, harming performance.

20. What does SMOTE do to handle imbalanced data?

    SMOTE (Synthetic Minority Over-sampling Technique) generates synthetic minority-class examples by interpolating between a minority sample and its k nearest minority-class neighbors, rather than simply duplicating samples.

21. Why is accuracy a poor metric for imbalanced classification?

    With a highly skewed class distribution, a model can achieve high accuracy by predicting only the majority class while completely failing on the minority class. Precision, recall, F1, and AUC-PR are more informative.

22. Contrast random oversampling and random undersampling for class imbalance.

    Random oversampling duplicates minority-class samples (risking overfitting and larger datasets); random undersampling removes majority-class samples (risking loss of useful information and smaller datasets).

23. What is class weighting as a strategy for imbalanced data?

    Assigning higher misclassification cost (weight) to the minority class in the loss function so the model penalizes minority-class errors more heavily, without resampling the data; supported via parameters like class_weight='balanced'.

24. Which evaluation curve is preferred over ROC for highly imbalanced datasets, and why?

    The Precision-Recall curve (and its area, AUC-PR), because it focuses on the positive (minority) class performance and is not misleadingly optimistic when true negatives dominate, unlike the ROC curve.

What this deck covers

Unsupervised Learning and Feature Engineering flashcards FAQ