Data Science Mathematics and Statistics Syllabus

What three conditions must hold for $f$ to be continuous at a point $x=a$?

(1) $f(a)$ is defined; (2) $\lim_{x\to a} f(x)$ exists; (3) $\lim_{x\to a} f(x) = f(a)$.

State the definition of the derivative of $f$ at $x$ as a limit.

$f'(x) = \lim_{h \to 0} \dfrac{f(x+h) - f(x)}{h}$, the instantaneous rate of change (slope of the tangent line).

State the product rule and quotient rule for differentiation.

Product: $(fg)' = f'g + fg'$. Quotient: $\left(\dfrac{f}{g}\right)' = \dfrac{f'g - fg'}{g^{2}}$.

State the chain rule for differentiating a composite function.

If $y = f(g(x))$, then $\dfrac{dy}{dx} = f'(g(x)) \cdot g'(x)$. Equivalently, $\dfrac{dy}{dx} = \dfrac{dy}{du}\cdot\dfrac{du}{dx}$.

State the Fundamental Theorem of Calculus (both parts).

Part 1: If $F(x) = \int_a^x f(t)\,dt$ then $F'(x) = f(x)$. Part 2: $\int_a^b f(x)\,dx = F(b) - F(a)$, where $F$ is any antiderivative of $f$.

What is the difference between a definite and an indefinite integral?

An indefinite integral $\int f(x)\,dx = F(x) + C$ gives a family of antiderivatives. A definite integral $\int_a^b f(x)\,dx$ gives a number equal to the signed area under $f$ from $a$ to $b$.

State the integration by parts formula.

$\int u \, dv = uv - \int v \, du$. It follows from reversing the product rule for differentiation.

What is a partial derivative, and what notation is used?

A partial derivative measures the rate of change of a multivariable function with respect to one variable, holding the others constant: $\dfrac{\partial f}{\partial x}$ (or $f_x$). E.g., for $f(x,y)=x^2 y$, $\dfrac{\partial f}{\partial x} = 2xy$.

What is the gradient of a scalar function $f(x_1,\ldots,x_n)$, and what does it represent?

$\nabla f = \left( \dfrac{\partial f}{\partial x_1}, \ldots, \dfrac{\partial f}{\partial x_n} \right)$. It points in the direction of steepest ascent, and its magnitude is the maximum rate of increase.

What is the divergence of a vector field $\vec{F} = (F_1, F_2, F_3)$?

$\nabla \cdot \vec{F} = \dfrac{\partial F_1}{\partial x} + \dfrac{\partial F_2}{\partial y} + \dfrac{\partial F_3}{\partial z}$. It is a scalar measuring the net outflow (source/sink strength) at a point.

What is the curl of a vector field, and what does it measure?

$\nabla \times \vec{F} = \left( \dfrac{\partial F_3}{\partial y} - \dfrac{\partial F_2}{\partial z},\; \dfrac{\partial F_1}{\partial z} - \dfrac{\partial F_3}{\partial x},\; \dfrac{\partial F_2}{\partial x} - \dfrac{\partial F_1}{\partial y} \right)$. It is a vector measuring the local rotation (circulation) of the field.

What is the Hessian matrix, and how is it used to classify critical points?

The Hessian $H$ contains second-order partials $H_{ij} = \dfrac{\partial^2 f}{\partial x_i \partial x_j}$. At a critical point: positive-definite $H \Rightarrow$ local minimum, negative-definite $\Rightarrow$ local maximum, indefinite $\Rightarrow$ saddle point.

State the gradient descent update rule and the role of the learning rate.

$\theta_{t+1} = \theta_t - \eta \,\nabla J(\theta_t)$, where $\eta > 0$ is the learning rate. Too large $\eta$ may diverge/overshoot; too small $\eta$ converges slowly.

What is the difference between batch, stochastic, and mini-batch gradient descent?

Batch GD uses the full dataset per update (stable, slow). Stochastic GD (SGD) uses one sample per update (fast, noisy). Mini-batch GD uses a small subset, balancing speed and stability.

What is the method of Lagrange multipliers used for, and what condition does it impose?

It optimizes $f(\vec{x})$ subject to a constraint $g(\vec{x}) = 0$. At the optimum, $\nabla f = \lambda \nabla g$ for some multiplier $\lambda$; solve this together with $g(\vec{x})=0$.

What is the difference between a convex and a non-convex optimization problem?

In a convex problem (convex objective over a convex set), any local minimum is the global minimum, making it efficiently solvable. Non-convex problems may have many local minima and saddle points, so solutions are not guaranteed global.

State the three axioms of probability (Kolmogorov).

(1) $P(A) \geq 0$ for any event $A$; (2) $P(\Omega) = 1$ for the sample space $\Omega$; (3) For mutually exclusive events, $P\!\left(\bigcup_i A_i\right) = \sum_i P(A_i)$.

State the formula for conditional probability and the definition of independence.

$P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}$ for $P(B) > 0$. Events $A$ and $B$ are independent if $P(A \cap B) = P(A)P(B)$, equivalently $P(A\mid B) = P(A)$.

State the general addition rule for the probability of $A \cup B$.

$P(A \cup B) = P(A) + P(B) - P(A \cap B)$. For mutually exclusive events, $P(A\cap B)=0$, so $P(A\cup B)=P(A)+P(B)$.

What is the difference between a discrete and a continuous random variable?

A discrete random variable takes countable values, described by a probability mass function (PMF) $P(X=x)$. A continuous random variable takes uncountable values, described by a probability density function (PDF) $f(x)$, where $P(X=x)=0$ for any single point.

What is a cumulative distribution function (CDF), and what properties does it have?

$F(x) = P(X \leq x)$. It is non-decreasing, right-continuous, with $\lim_{x\to -\infty} F(x) = 0$ and $\lim_{x\to \infty} F(x) = 1$.

State the PMF, mean, and variance of a Binomial distribution.

$P(X=k) = \binom{n}{k} p^{k} (1-p)^{n-k}$, for $k=0,\ldots,n$. Mean $\mu = np$, variance $\sigma^{2} = np(1-p)$.

State the PMF, mean, and variance of a Poisson distribution.

$P(X=k) = \dfrac{\lambda^{k} e^{-\lambda}}{k!}$, for $k=0,1,2,\ldots$. Mean $= \lambda$ and variance $= \lambda$. It models counts of rare events over a fixed interval.

State the PDF of the Normal (Gaussian) distribution and what its parameters mean.

$f(x) = \dfrac{1}{\sigma\sqrt{2\pi}} \exp\!\left( -\dfrac{(x-\mu)^{2}}{2\sigma^{2}} \right)$, where $\mu$ is the mean (center) and $\sigma^2$ is the variance (spread).

What is the standard normal distribution and the z-score formula?

The standard normal $Z \sim \mathcal{N}(0,1)$ has mean $0$ and variance $1$. The z-score $z = \dfrac{x - \mu}{\sigma}$ standardizes a value to express it in standard-deviation units from the mean.

Define the expectation of a discrete and a continuous random variable.

Discrete: $\mathbb{E}[X] = \sum_x x\, P(X=x)$. Continuous: $\mathbb{E}[X] = \int_{-\infty}^{\infty} x\, f(x)\, dx$. It is the long-run average value.

State the definition of variance and the computational shortcut formula.

$\operatorname{Var}(X) = \mathbb{E}\!\left[(X - \mu)^{2}\right] = \mathbb{E}[X^{2}] - (\mathbb{E}[X])^{2}$. The standard deviation is $\sigma = \sqrt{\operatorname{Var}(X)}$.

State the linearity of expectation and the variance scaling rule.

$\mathbb{E}[aX + bY] = a\,\mathbb{E}[X] + b\,\mathbb{E}[Y]$ (always). For variance, $\operatorname{Var}(aX + b) = a^{2}\operatorname{Var}(X)$, and $\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)$ only if $X,Y$ are independent.

State Bayes' Theorem.

$P(A \mid B) = \dfrac{P(B \mid A)\, P(A)}{P(B)}$, where $P(A)$ is the prior, $P(B\mid A)$ the likelihood, $P(B)$ the evidence, and $P(A\mid B)$ the posterior.

How is the evidence $P(B)$ computed via the law of total probability in Bayes' Theorem?

For a partition $\{A_i\}$ of the sample space, $P(B) = \sum_i P(B \mid A_i)\, P(A_i)$. This normalizing denominator ensures the posterior sums to $1$.

State the (weak) Law of Large Numbers.

For i.i.d. random variables with mean $\mu$, the sample mean converges to the true mean as $n\to\infty$: $\bar{X}_n = \frac{1}{n}\sum_{i=1}^{n} X_i \xrightarrow{P} \mu$. Empirical averages stabilize around the expected value with more data.

State the Central Limit Theorem and why it matters.

For i.i.d. variables with mean $\mu$ and variance $\sigma^2$, the sample mean is approximately normal for large $n$: $\bar{X}_n \approx \mathcal{N}\!\left(\mu, \dfrac{\sigma^{2}}{n}\right)$, regardless of the original distribution. It justifies using normal-based inference (confidence intervals, hypothesis tests).