Machine Learning Python Programming Flashcards

24 sample cards from the Python Programming deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. What is the difference between Lasso (L1) and Ridge (L2) regularization?

   Ridge adds an $L2$ penalty $\lambda \sum_j \beta_j^{2}$, shrinking coefficients smoothly toward zero. Lasso adds an $L1$ penalty $\lambda \sum_j |\beta_j|$, which can drive some coefficients exactly to zero, performing feature selection.

2. What does the coefficient of determination $R^2$ measure in regression?

   $R^2$ measures the proportion of variance in the target explained by the model: $$R^2 = 1 - \frac{\sum_i (y_i - \hat{y}_i)^2}{\sum_i (y_i - \bar{y})^2}$$ where $1$ is a perfect fit and $0$ means no better than predicting the mean.

3. What function does logistic regression use to map outputs to probabilities, and what is its formula?

   The sigmoid (logistic) function: $$\sigma(z) = \frac{1}{1 + e^{-z}}$$ where $z = \beta_0 + \beta^\top x$. It squashes any real value into the range $(0, 1)$, interpreted as a class probability.

4. Define precision, recall, and the F1 score for binary classification.

   $\text{Precision} = \frac{TP}{TP + FP}$ (correctness of positive predictions); $\text{Recall} = \frac{TP}{TP + FN}$ (coverage of actual positives); $F_1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$, their harmonic mean.

5. What is a confusion matrix in classification?

   A table comparing predicted vs. actual classes. For binary classification it has four cells: True Positives (TP), True Negatives (TN), False Positives (FP, Type I error), and False Negatives (FN, Type II error).

6. How does the k-Nearest Neighbors (KNN) algorithm classify a new point?

   It finds the $k$ closest training points (typically by Euclidean distance $d = \sqrt{\sum_i (x_i - q_i)^2}$) and assigns the majority class among those neighbors. It is a lazy, non-parametric, instance-based method.

7. What does a Support Vector Machine (SVM) try to find, and what is the kernel trick?

   An SVM finds the hyperplane that maximizes the margin between classes (distance to the nearest support vectors). The kernel trick maps data into a higher-dimensional space via a kernel function (e.g. RBF) to make non-linearly-separable data separable without explicit transformation.

8. How does the K-Means clustering algorithm work?

   It partitions data into $k$ clusters by iterating: (1) assign each point to the nearest centroid, (2) recompute each centroid as the mean of its assigned points; repeat until assignments stabilize. It minimizes within-cluster sum of squares (inertia): $\sum_{k}\sum_{x \in C_k}\lVert x - \mu_k \rVert^2$.

9. What is the elbow method for choosing the number of clusters in K-Means?

   Plot the within-cluster sum of squares (inertia) against the number of clusters $k$. The 'elbow' point, where the curve's rate of decrease sharply levels off, suggests a good value of $k$ that balances compactness and simplicity.

10. How does DBSCAN differ from K-Means clustering?

    K-Means requires a preset number of clusters and finds spherical, similarly sized clusters. DBSCAN is density-based, using parameters eps and min_samples to find arbitrarily shaped clusters of varying size, automatically determining cluster count and labeling sparse points as noise/outliers.

11. What is the difference between supervised and unsupervised learning, with example tasks?

    Supervised learning uses labeled data to learn a mapping from inputs to known targets (e.g. regression, classification). Unsupervised learning finds structure in unlabeled data (e.g. clustering, dimensionality reduction) without target outputs.

12. What is Principal Component Analysis (PCA) and what does it maximize?

    PCA is a linear dimensionality-reduction technique that projects data onto orthogonal principal components—directions that maximize retained variance. The components are the eigenvectors of the covariance matrix, ordered by their eigenvalues (variance explained).

13. Why is dimensionality reduction useful, and name one nonlinear method besides PCA.

    It reduces the number of features to combat the curse of dimensionality, lower computation, remove noise/redundancy, and enable visualization. A nonlinear method is t-SNE (or UMAP), which preserves local neighborhood structure for visualizing high-dimensional data in 2D/3D.

14. What is a perceptron / artificial neuron, and what does it compute?

    A neuron computes a weighted sum of its inputs plus a bias, then applies an activation function: $$a = f\!\left(\sum_{i} w_i x_i + b\right)$$ producing one output value passed to the next layer.

15. Name three common neural-network activation functions and their formulas.

    Sigmoid: $\sigma(z) = \frac{1}{1+e^{-z}}$; Hyperbolic tangent: $\tanh(z) = \frac{e^{z}-e^{-z}}{e^{z}+e^{-z}}$; ReLU: $\mathrm{ReLU}(z) = \max(0, z)$.

16. What is backpropagation in neural networks?

    Backpropagation is the algorithm that computes gradients of the loss with respect to every weight by applying the chain rule backward through the network, layer by layer. These gradients are then used by gradient descent to update the weights: $w \leftarrow w - \eta \frac{\partial L}{\partial w}$.

17. What role does the learning rate $\eta$ play in gradient descent, and what happens if it is too high or too low?

    $\eta$ scales each weight update $w \leftarrow w - \eta \nabla L$. If too high, training may overshoot the minimum and diverge or oscillate; if too low, training converges very slowly and may get stuck in poor local regions.

18. What is the difference between TensorFlow and Keras?

    TensorFlow is Google's low-level, scalable deep-learning framework operating on tensors and computation graphs. Keras is a high-level, user-friendly API (now tf.keras, integrated into TensorFlow) for quickly building and training neural networks with concise code.

19. What is a tensor in TensorFlow, and how do its rank/shape relate to a scalar, vector, and matrix?

    A tensor is a multi-dimensional array of a single dtype. Rank 0 is a scalar, rank 1 is a vector, rank 2 is a matrix, and rank $\geq 3$ are higher-dimensional tensors; the shape is the tuple giving the size along each axis.

20. In Keras, what is the difference between the Sequential and Functional APIs?

    The Sequential API stacks layers in a simple linear pipeline (one input, one output). The Functional API connects layers as a graph, supporting multiple inputs/outputs, shared layers, and branching (non-linear architectures).

21. What are the typical steps to build and train a model with Keras?

    1) Define the architecture (e.g. Sequential with Dense layers); 2) compile(optimizer, loss, metrics) to configure training; 3) fit(X, y, epochs, batch_size) to train; 4) evaluate(X_test, y_test) to assess; 5) predict(X) for inference.

22. What is the difference between overfitting and underfitting, and one remedy for each?

    Overfitting: the model learns noise and fits training data too closely, so it generalizes poorly (high variance); remedy: regularization, dropout, or more data. Underfitting: the model is too simple to capture the pattern (high bias); remedy: a more complex model or more/better features.

23. What is the bias-variance tradeoff?

    Expected error decomposes into bias (error from overly simplistic assumptions) plus variance (error from sensitivity to training data) plus irreducible noise. Reducing one tends to increase the other; the goal is to minimize total generalization error by balancing model complexity.

24. What is a loss function commonly used for multi-class classification in neural networks, and its formula?

    Categorical cross-entropy: $$L = -\sum_{i=1}^{C} y_i \log(\hat{y}_i)$$ where $C$ is the number of classes, $y_i$ is the one-hot true label, and $\hat{y}_i$ is the predicted probability (typically from a softmax output layer).

What this deck covers

Python Programming flashcards FAQ