Machine Learning Introduction to Machine Learning Syllabus

State the gradient descent parameter update rule.

$$\theta_j := \theta_j - \alpha \frac{\partial}{\partial \theta_j} J(\theta)$$ where $\alpha$ is the learning rate controlling the step size.

What role does the learning rate $\alpha$ play in gradient descent?

It controls the step size of each update. If $\alpha$ is too large, the algorithm may overshoot or diverge; if too small, convergence is very slow.

What is unsupervised learning?

Unsupervised learning finds hidden patterns, structure, or groupings in input data that has no labeled outputs, discovering the inherent organization of the data on its own.

What are the main task types in unsupervised learning?

Clustering (grouping similar data points), dimensionality reduction (compressing features while preserving structure), and association rule mining (finding relationships among variables).

Name three common unsupervised learning algorithms.

k-means clustering, hierarchical clustering, DBSCAN, principal component analysis (PCA), and the Apriori algorithm for association rules.

Describe the k-means clustering algorithm in steps.

1) Choose $k$ and initialize $k$ centroids. 2) Assign each point to its nearest centroid. 3) Recompute each centroid as the mean of its assigned points. 4) Repeat steps 2-3 until assignments no longer change (convergence).

What objective does k-means minimize?

It minimizes the within-cluster sum of squares (inertia): $$J = \sum_{i=1}^{k} \sum_{x \in C_i} \lVert x - \mu_i \rVert^{2}$$ where $\mu_i$ is the centroid of cluster $C_i$.

What is the purpose of dimensionality reduction, and name one method?

To reduce the number of features while retaining as much important information as possible, easing computation and visualization and reducing overfitting. Principal Component Analysis (PCA) is a common method.

What is the Euclidean distance between two points $p$ and $q$ in $n$ dimensions?

$$d(p, q) = \sqrt{\sum_{i=1}^{n} (p_i - q_i)^{2}}$$

What is reinforcement learning (RL)?

Reinforcement learning is a paradigm where an agent learns to make sequential decisions by interacting with an environment, taking actions to maximize cumulative reward over time through trial and error.

Name the core components of a reinforcement learning problem.

Agent, environment, state ($s$), action ($a$), reward ($r$), and policy ($\pi$). Often also a value function and a model of the environment.

What is a policy $\pi$ in reinforcement learning?

A policy is the agent's strategy: a mapping from states to actions, $\pi(a \mid s)$, that defines which action the agent takes (or the probability of taking it) in each state.

What is the goal of an RL agent expressed as cumulative discounted reward?

To maximize the expected return $$G_t = \sum_{k=0}^{\infty} \gamma^{k} r_{t+k+1}$$ where $\gamma \in [0,1]$ is the discount factor weighting future rewards.

What does the discount factor $\gamma$ control in reinforcement learning?

It controls how much future rewards are valued relative to immediate ones. $\gamma$ near $0$ makes the agent myopic (short-sighted); $\gamma$ near $1$ makes it value long-term rewards.

What is the exploration vs. exploitation trade-off in RL?

Exploration means trying new actions to discover their rewards; exploitation means choosing the best-known action for immediate gain. The agent must balance both to maximize long-term reward.

What is the Q-learning update rule (Bellman update)?

$$Q(s,a) \leftarrow Q(s,a) + \alpha \left[ r + \gamma \max_{a'} Q(s', a') - Q(s,a) \right]$$ where $\alpha$ is the learning rate and $\gamma$ the discount factor.

What mathematical framework formally models reinforcement learning problems?

The Markov Decision Process (MDP), defined by the tuple $(S, A, P, R, \gamma)$: states, actions, transition probabilities, reward function, and discount factor.

What is the Markov property in an MDP?

The future state depends only on the current state and action, not on the full history: $$P(s_{t+1} \mid s_t, a_t) = P(s_{t+1} \mid s_t, a_t, s_{t-1}, \dots)$$

What is semi-supervised learning?

Semi-supervised learning trains on a dataset containing a small amount of labeled data together with a large amount of unlabeled data, using the unlabeled data to improve learning beyond what the labels alone allow.

Why is semi-supervised learning useful in practice?

Labeling data is often expensive and time-consuming, while unlabeled data is abundant and cheap. Semi-supervised learning leverages the plentiful unlabeled data to boost accuracy with limited labeling effort.

Name two common techniques used in semi-supervised learning.

Self-training (pseudo-labeling), co-training, generative models, and graph-based label propagation are common semi-supervised techniques.

What key assumptions allow semi-supervised learning to work?

The cluster/continuity assumption (nearby points likely share a label), the low-density separation assumption (decision boundaries lie in low-density regions), and the manifold assumption (data lies on a lower-dimensional manifold).

Compare supervised and unsupervised learning by data and goal.

Supervised learning uses labeled data to learn an input-to-output mapping for prediction. Unsupervised learning uses unlabeled data to discover hidden structure or groupings. The key difference is the presence or absence of target labels.

What is overfitting in machine learning?

Overfitting occurs when a model learns the training data too closely, including its noise, so it performs well on training data but poorly on new, unseen data (poor generalization).

What is underfitting, and how does it relate to bias?

Underfitting occurs when a model is too simple to capture the underlying pattern, performing poorly on both training and test data. It corresponds to high bias.

State the bias-variance decomposition of expected prediction error.

$$\text{Error} = \text{Bias}^{2} + \text{Variance} + \text{Irreducible Error}$$ High bias causes underfitting; high variance causes overfitting.

Why is data split into training, validation, and test sets?

The training set fits the model, the validation set tunes hyperparameters and selects models, and the test set gives an unbiased estimate of generalization performance on unseen data.

How does deep learning relate to machine learning?

Deep learning is a subset of machine learning that uses multilayer (deep) neural networks to automatically learn hierarchical feature representations from large amounts of data, reducing the need for manual feature engineering.

Give one example task for each: classification, regression, clustering, and reinforcement learning.

Classification: email spam detection. Regression: predicting house prices. Clustering: customer segmentation. Reinforcement learning: training a program to play a game like chess or Go.