Machine Learning Python Programming Syllabus

In NumPy, what is the difference between a.reshape(2,3) and a.T?

reshape(2,3) returns a view of the same data arranged into a 2x3 shape (total element count must match), while a.T returns the transpose, swapping axes so that element $(i,j)$ becomes $(j,i)$.

In Pandas, what is the difference between a Series and a DataFrame?

A Series is a one-dimensional labeled array (a single column with an index). A DataFrame is a two-dimensional labeled table of rows and columns, where each column is a Series sharing a common row index.

In Pandas, what is the difference between .loc and .iloc?

.loc selects by label/name of rows and columns (inclusive of the end label in slices), while .iloc selects by integer position (end-exclusive like normal Python slicing).

How do you handle missing values in a Pandas DataFrame df?

df.isna() detects missing (NaN) values; df.dropna() removes rows/columns containing them; df.fillna(value) replaces them with a constant, mean, forward-fill (method='ffill'), or other strategy.

What does the Pandas groupby operation follow as its conceptual model?

The split-apply-combine model: split the data into groups by key(s), apply an aggregation/transformation/filter to each group, then combine the results into a new Series or DataFrame, e.g. df.groupby('col')['x'].mean().

In Matplotlib, what is the difference between the pyplot interface and the object-oriented (Axes) interface?

The pyplot interface (plt.plot, plt.title) operates on an implicit current figure/axes and is quick for simple plots. The object-oriented interface creates explicit Figure and Axes objects via fig, ax = plt.subplots() and calls methods like ax.plot, giving finer control over multiple subplots.

In Matplotlib, which functions create a line plot, a scatter plot, a bar chart, and a histogram?

plt.plot() draws line plots, plt.scatter() draws scatter plots, plt.bar() draws bar charts, and plt.hist() draws histograms of a distribution.

What is Seaborn and how does it relate to Matplotlib?

Seaborn is a high-level statistical data visualization library built on top of Matplotlib. It provides attractive default styles and concise functions for statistical plots and integrates directly with Pandas DataFrames.

In Seaborn, what kind of plot does sns.heatmap() typically display, and a common use case?

sns.heatmap() displays a matrix of values as a color-encoded grid. A common use case is visualizing a correlation matrix, e.g. sns.heatmap(df.corr(), annot=True), to spot strongly related features.

What is the difference between Seaborn's sns.boxplot and sns.violinplot?

A boxplot summarizes a distribution via median, quartiles, and outliers. A violinplot shows the same summary plus a mirrored kernel density estimate of the full distribution's shape, revealing multimodality a boxplot would hide.

In Scikit-Learn, what is the standard estimator API pattern (the three core methods)?

fit(X, y) learns parameters from training data; predict(X) produces predictions on new data; and transform(X) (for transformers) outputs modified features. fit_transform combines fitting and transforming in one call.

What is the purpose of train_test_split in Scikit-Learn?

It randomly partitions a dataset into separate training and testing subsets (e.g. test_size=0.2) so a model is trained on one portion and evaluated on unseen data, giving an unbiased estimate of generalization performance.

Why and how is feature scaling done with StandardScaler in Scikit-Learn?

StandardScaler standardizes each feature to zero mean and unit variance using $z = \frac{x - \mu}{\sigma}$. Scaling is needed for distance- and gradient-based algorithms (KNN, SVM, neural nets) so features on larger scales do not dominate.

What is k-fold cross-validation?

The data is split into $k$ equal folds; the model is trained on $k-1$ folds and validated on the remaining fold, repeated so each fold serves once as validation. The $k$ scores are averaged for a robust performance estimate that reduces dependence on a single split.

What is the simple linear regression model equation and what does it predict?

$$\hat{y} = \beta_0 + \beta_1 x$$ It predicts a continuous target $\hat{y}$ as a linear function of input $x$, where $\beta_0$ is the intercept and $\beta_1$ is the slope (coefficient).

What cost function does ordinary linear regression minimize?

Mean Squared Error (the sum/mean of squared residuals): $$J(\beta) = \frac{1}{n}\sum_{i=1}^{n}\left(y_i - \hat{y}_i\right)^2$$ minimized to find the best-fit coefficients.

What is the difference between Lasso (L1) and Ridge (L2) regularization?

Ridge adds an $L2$ penalty $\lambda \sum_j \beta_j^{2}$, shrinking coefficients smoothly toward zero. Lasso adds an $L1$ penalty $\lambda \sum_j |\beta_j|$, which can drive some coefficients exactly to zero, performing feature selection.

What does the coefficient of determination $R^2$ measure in regression?

$R^2$ measures the proportion of variance in the target explained by the model: $$R^2 = 1 - \frac{\sum_i (y_i - \hat{y}_i)^2}{\sum_i (y_i - \bar{y})^2}$$ where $1$ is a perfect fit and $0$ means no better than predicting the mean.

What function does logistic regression use to map outputs to probabilities, and what is its formula?

The sigmoid (logistic) function: $$\sigma(z) = \frac{1}{1 + e^{-z}}$$ where $z = \beta_0 + \beta^\top x$. It squashes any real value into the range $(0, 1)$, interpreted as a class probability.

Define precision, recall, and the F1 score for binary classification.

$\text{Precision} = \frac{TP}{TP + FP}$ (correctness of positive predictions); $\text{Recall} = \frac{TP}{TP + FN}$ (coverage of actual positives); $F_1 = 2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}$, their harmonic mean.

What is a confusion matrix in classification?

A table comparing predicted vs. actual classes. For binary classification it has four cells: True Positives (TP), True Negatives (TN), False Positives (FP, Type I error), and False Negatives (FN, Type II error).

How does the k-Nearest Neighbors (KNN) algorithm classify a new point?

It finds the $k$ closest training points (typically by Euclidean distance $d = \sqrt{\sum_i (x_i - q_i)^2}$) and assigns the majority class among those neighbors. It is a lazy, non-parametric, instance-based method.

What does a Support Vector Machine (SVM) try to find, and what is the kernel trick?

An SVM finds the hyperplane that maximizes the margin between classes (distance to the nearest support vectors). The kernel trick maps data into a higher-dimensional space via a kernel function (e.g. RBF) to make non-linearly-separable data separable without explicit transformation.

How does the K-Means clustering algorithm work?

It partitions data into $k$ clusters by iterating: (1) assign each point to the nearest centroid, (2) recompute each centroid as the mean of its assigned points; repeat until assignments stabilize. It minimizes within-cluster sum of squares (inertia): $\sum_{k}\sum_{x \in C_k}\lVert x - \mu_k \rVert^2$.

What is the elbow method for choosing the number of clusters in K-Means?

Plot the within-cluster sum of squares (inertia) against the number of clusters $k$. The 'elbow' point, where the curve's rate of decrease sharply levels off, suggests a good value of $k$ that balances compactness and simplicity.

How does DBSCAN differ from K-Means clustering?

K-Means requires a preset number of clusters and finds spherical, similarly sized clusters. DBSCAN is density-based, using parameters eps and min_samples to find arbitrarily shaped clusters of varying size, automatically determining cluster count and labeling sparse points as noise/outliers.

What is the difference between supervised and unsupervised learning, with example tasks?

Supervised learning uses labeled data to learn a mapping from inputs to known targets (e.g. regression, classification). Unsupervised learning finds structure in unlabeled data (e.g. clustering, dimensionality reduction) without target outputs.

What is Principal Component Analysis (PCA) and what does it maximize?

PCA is a linear dimensionality-reduction technique that projects data onto orthogonal principal components—directions that maximize retained variance. The components are the eigenvectors of the covariance matrix, ordered by their eigenvalues (variance explained).

Why is dimensionality reduction useful, and name one nonlinear method besides PCA.

It reduces the number of features to combat the curse of dimensionality, lower computation, remove noise/redundancy, and enable visualization. A nonlinear method is t-SNE (or UMAP), which preserves local neighborhood structure for visualizing high-dimensional data in 2D/3D.

What is a perceptron / artificial neuron, and what does it compute?

A neuron computes a weighted sum of its inputs plus a bias, then applies an activation function: $$a = f\!\left(\sum_{i} w_i x_i + b\right)$$ producing one output value passed to the next layer.

Name three common neural-network activation functions and their formulas.

Sigmoid: $\sigma(z) = \frac{1}{1+e^{-z}}$; Hyperbolic tangent: $\tanh(z) = \frac{e^{z}-e^{-z}}{e^{z}+e^{-z}}$; ReLU: $\mathrm{ReLU}(z) = \max(0, z)$.

What is backpropagation in neural networks?

Backpropagation is the algorithm that computes gradients of the loss with respect to every weight by applying the chain rule backward through the network, layer by layer. These gradients are then used by gradient descent to update the weights: $w \leftarrow w - \eta \frac{\partial L}{\partial w}$.

What role does the learning rate $\eta$ play in gradient descent, and what happens if it is too high or too low?

$\eta$ scales each weight update $w \leftarrow w - \eta \nabla L$. If too high, training may overshoot the minimum and diverge or oscillate; if too low, training converges very slowly and may get stuck in poor local regions.

What is the difference between TensorFlow and Keras?

TensorFlow is Google's low-level, scalable deep-learning framework operating on tensors and computation graphs. Keras is a high-level, user-friendly API (now tf.keras, integrated into TensorFlow) for quickly building and training neural networks with concise code.

What is a tensor in TensorFlow, and how do its rank/shape relate to a scalar, vector, and matrix?

A tensor is a multi-dimensional array of a single dtype. Rank 0 is a scalar, rank 1 is a vector, rank 2 is a matrix, and rank $\geq 3$ are higher-dimensional tensors; the shape is the tuple giving the size along each axis.

In Keras, what is the difference between the Sequential and Functional APIs?

The Sequential API stacks layers in a simple linear pipeline (one input, one output). The Functional API connects layers as a graph, supporting multiple inputs/outputs, shared layers, and branching (non-linear architectures).

What are the typical steps to build and train a model with Keras?

1) Define the architecture (e.g. Sequential with Dense layers); 2) compile(optimizer, loss, metrics) to configure training; 3) fit(X, y, epochs, batch_size) to train; 4) evaluate(X_test, y_test) to assess; 5) predict(X) for inference.

What is the difference between overfitting and underfitting, and one remedy for each?

Overfitting: the model learns noise and fits training data too closely, so it generalizes poorly (high variance); remedy: regularization, dropout, or more data. Underfitting: the model is too simple to capture the pattern (high bias); remedy: a more complex model or more/better features.

What is the bias-variance tradeoff?

Expected error decomposes into bias (error from overly simplistic assumptions) plus variance (error from sensitivity to training data) plus irreducible noise. Reducing one tends to increase the other; the goal is to minimize total generalization error by balancing model complexity.

What is a loss function commonly used for multi-class classification in neural networks, and its formula?

Categorical cross-entropy: $$L = -\sum_{i=1}^{C} y_i \log(\hat{y}_i)$$ where $C$ is the number of classes, $y_i$ is the one-hot true label, and $\hat{y}_i$ is the predicted probability (typically from a softmax output layer).