Data Science & Machine Learning Mathematics and Statistics for Machine Learning Syllabus

What role does the learning rate play in gradient descent?

It controls the step size. Too large can overshoot or diverge; too small leads to very slow convergence. It balances speed and stability.

What distinguishes batch, stochastic (SGD), and mini-batch gradient descent?

Batch uses the entire dataset per update (stable but slow). Stochastic uses one sample per update (fast, noisy). Mini-batch uses a small subset, balancing speed and stability — the most common in practice.

What is a convex function and why does convexity matter in optimization?

A function is convex if the line segment between any two points on its graph lies on or above the graph. For convex functions, any local minimum is also the global minimum, guaranteeing gradient descent finds the optimum.

Define the mean, median, and mode.

Mean is the arithmetic average (sum ÷ count). Median is the middle value when data is ordered. Mode is the most frequently occurring value.

When is the median preferred over the mean as a measure of central tendency?

The median is preferred for skewed distributions or data with outliers, because it is robust and not distorted by extreme values, unlike the mean.

What are variance and standard deviation?

Variance is the average of the squared deviations from the mean (σ² = Σ(xᵢ − μ)²/N). Standard deviation is the square root of variance, expressing dispersion in the same units as the data.

What is the difference between population variance and sample variance?

Population variance divides by N. Sample variance divides by (n − 1) — Bessel's correction — to give an unbiased estimate of the population variance.

What is the range and the interquartile range (IQR)?

Range = maximum − minimum. IQR = Q3 − Q1, the spread of the middle 50% of the data. IQR is robust to outliers, unlike the range.

What is the coefficient of variation (CV)?

CV = (standard deviation / mean) × 100%. It is a unitless relative measure of dispersion, useful for comparing variability between datasets with different units or scales.

What is skewness and what do positive and negative skew indicate?

Skewness measures asymmetry of a distribution. Positive (right) skew has a long right tail with mean > median; negative (left) skew has a long left tail with mean < median.

What is kurtosis?

Kurtosis measures the 'tailedness' / peakedness of a distribution. High kurtosis (leptokurtic) means heavy tails and more outliers; low kurtosis (platykurtic) means light tails; a normal distribution is mesokurtic (kurtosis = 3).

What is covariance and what does its sign indicate?

Covariance measures how two variables vary together: Cov(X,Y) = E[(X−μₓ)(Y−μᵧ)]. A positive value means they tend to increase together; negative means one rises as the other falls. Its magnitude depends on units.

What is the Pearson correlation coefficient and its range?

r = Cov(X,Y) / (σₓσᵧ), a normalized covariance measuring linear association. It ranges from −1 (perfect negative) to +1 (perfect positive), with 0 meaning no linear relationship.

Why is 'correlation does not imply causation' an important caution?

Two variables can be correlated due to coincidence, a confounding third variable, or reverse causation — correlation alone cannot establish that one variable causes changes in the other.

State the three axioms / basic rules of probability.

1) Probability of any event is between 0 and 1. 2) The probability of the sample space (certain event) is 1. 3) For mutually exclusive events, P(A∪B) = P(A) + P(B).

What is conditional probability?

P(A|B) = P(A∩B)/P(B), the probability of A occurring given that B has occurred (requires P(B) > 0).

What does it mean for two events to be independent?

Events A and B are independent if the occurrence of one does not affect the other: P(A∩B) = P(A)·P(B), equivalently P(A|B) = P(A).

State Bayes' Theorem.

P(A|B) = [P(B|A)·P(A)] / P(B). It updates the prior probability P(A) into the posterior P(A|B) using the likelihood P(B|A) and evidence P(B).

In Bayes' Theorem, name the prior, likelihood, and posterior.

Prior = P(A): belief before evidence. Likelihood = P(B|A): probability of evidence given the hypothesis. Posterior = P(A|B): updated belief after seeing the evidence.

What is the difference between a discrete and a continuous random variable?

A discrete random variable takes countable distinct values (e.g., dice rolls), described by a probability mass function (PMF). A continuous one takes any value in an interval, described by a probability density function (PDF).

What are the expected value and variance of a random variable?

Expected value E[X] = Σ x·P(x) (discrete) or ∫ x·f(x)dx (continuous) — the long-run average. Variance Var(X) = E[(X − E[X])²] = E[X²] − (E[X])² measures its spread.

Describe the key properties of the normal (Gaussian) distribution.

It is symmetric and bell-shaped, defined by mean μ and standard deviation σ, with mean = median = mode. By the empirical rule, ~68% of data lies within 1σ, ~95% within 2σ, and ~99.7% within 3σ of the mean.

What is the Bernoulli distribution versus the Binomial distribution?

Bernoulli models a single trial with two outcomes (success/failure) with probability p. Binomial models the number of successes in n independent Bernoulli trials, with mean np and variance np(1−p).

What does the Poisson distribution model, and what is its mean and variance?

The Poisson distribution models the number of events occurring in a fixed interval of time or space given a constant average rate λ. Its mean and variance are both equal to λ.

What does the Central Limit Theorem (CLT) state?

The sampling distribution of the sample mean approaches a normal distribution as sample size n grows, regardless of the population's distribution, with mean μ and standard error σ/√n (typically n ≥ 30).

What is the difference between a population parameter and a sample statistic?

A parameter is a fixed numerical characteristic of the entire population (e.g., μ, σ). A statistic is a value computed from a sample (e.g., x̄, s) used to estimate the parameter.

What is the standard error of the mean?

The standard error is the standard deviation of the sampling distribution of the mean, SE = σ/√n. It quantifies how much the sample mean is expected to vary from the true population mean and decreases as n increases.

What are the null hypothesis (H₀) and alternative hypothesis (H₁) in hypothesis testing?

H₀ states there is no effect or no difference (the default assumption). H₁ states there is an effect or difference. Testing seeks evidence to reject H₀ in favor of H₁.

What is a p-value and how is it interpreted with a significance level α?

The p-value is the probability of observing data at least as extreme as the sample, assuming H₀ is true. If p ≤ α (commonly 0.05), reject H₀; if p > α, fail to reject H₀. A small p-value does not measure effect size.

Distinguish Type I and Type II errors.

A Type I error (false positive) is rejecting a true H₀; its probability is α. A Type II error (false negative) is failing to reject a false H₀; its probability is β, and power = 1 − β.

When do you use a t-test versus a z-test, and what is a chi-square test for?

Use a z-test when population variance is known or n is large; a t-test when the population variance is unknown and n is small. A chi-square test assesses associations between categorical variables or goodness-of-fit. ANOVA compares means across three or more groups.

What is a confidence interval, and how does a 95% CI interpret correctly?

A confidence interval is a range (estimate ± margin of error) likely to contain the true parameter. A 95% CI means that if sampling were repeated many times, about 95% of such intervals would contain the true parameter — not that there is a 95% probability the parameter lies in this specific interval.