Machine Learning Mathematics Syllabus

What is the power rule for derivatives: $\frac{d}{dx}x^{n} = ?$

$\dfrac{d}{dx}x^{n} = n\,x^{n-1}$.

State the chain rule for $\frac{d}{dx}f(g(x))$.

$\dfrac{d}{dx}f(g(x)) = f'(g(x))\cdot g'(x)$.

State the product rule for differentiation of $f(x)g(x)$.

$(fg)' = f'g + fg'$.

What is the gradient $\nabla f$ of a scalar function $f(x_1,\dots,x_n)$, and what does it point toward?

$\nabla f = \left(\dfrac{\partial f}{\partial x_1}, \dots, \dfrac{\partial f}{\partial x_n}\right)$; it points in the direction of steepest ascent.

What is the derivative of the natural exponential and natural logarithm?

$\dfrac{d}{dx}e^{x} = e^{x}$ and $\dfrac{d}{dx}\ln x = \dfrac{1}{x}$.

State the power rule for integration: $\int x^{n}\,dx = ?$ (for $n \neq -1$).

$\int x^{n}\,dx = \dfrac{x^{n+1}}{n+1} + C$.

State the Fundamental Theorem of Calculus (evaluation form).

If $F' = f$, then $\displaystyle\int_a^b f(x)\,dx = F(b) - F(a)$.

What is the formula for integration by parts?

$\displaystyle\int u\,dv = uv - \int v\,du$.

At an interior point, what is the first-order necessary condition for $f$ to have a local extremum?

The gradient vanishes: $\nabla f = \vec{0}$ (a stationary/critical point); $f'(x)=0$ in one dimension.

How does the second-derivative test classify a critical point in one variable where $f'(x_0)=0$?

If $f''(x_0) > 0$ it is a local minimum; if $f''(x_0) < 0$ a local maximum; if $f''(x_0) = 0$ the test is inconclusive.

What is the gradient descent update rule with learning rate $\eta$?

$\vec{x}_{t+1} = \vec{x}_t - \eta\,\nabla f(\vec{x}_t)$.

What property of a function guarantees that any local minimum is also a global minimum?

Convexity — for a convex function over a convex set, every local minimum is a global minimum.

In constrained optimization, what method introduces multipliers to handle equality constraints?

The method of Lagrange multipliers: optimize $\mathcal{L}(\vec{x},\lambda) = f(\vec{x}) - \lambda\,g(\vec{x})$, setting $\nabla\mathcal{L} = \vec{0}$.

State the three Kolmogorov axioms of probability.

(1) $P(A) \geq 0$; (2) $P(\Omega) = 1$; (3) for mutually exclusive events, $P(A\cup B) = P(A) + P(B)$.

State Bayes' theorem.

$P(A\mid B) = \dfrac{P(B\mid A)\,P(A)}{P(B)}$.

When does $P(A\cap B) = P(A)\,P(B)$ hold?

When events $A$ and $B$ are independent.

State the general addition rule for $P(A\cup B)$.

$P(A\cup B) = P(A) + P(B) - P(A\cap B)$.

What are the definitions of expectation $E[X]$ for a discrete random variable and its variance?

$E[X] = \sum_i x_i\,P(x_i)$ and $\operatorname{Var}(X) = E[(X-E[X])^{2}] = E[X^{2}] - (E[X])^{2}$.

What is the probability density function of the normal distribution $\mathcal{N}(\mu,\sigma^2)$?

$f(x) = \dfrac{1}{\sigma\sqrt{2\pi}}\,\exp\!\left(-\dfrac{(x-\mu)^{2}}{2\sigma^{2}}\right)$.

How do the mean, median, and mode compare, and which is most robust to outliers?

Mean is the arithmetic average, median the middle value, mode the most frequent value. The median is most robust to outliers; the mean is most sensitive.

What is the difference between population variance and sample variance formulas?

Population: $\sigma^{2} = \dfrac{1}{N}\sum (x_i-\mu)^{2}$. Sample (Bessel's correction): $s^{2} = \dfrac{1}{n-1}\sum (x_i-\bar{x})^{2}$.

Define the standard deviation in terms of variance.

$\sigma = \sqrt{\operatorname{Var}(X)}$ — the square root of the variance, in the same units as the data.

What does the Central Limit Theorem state?

The distribution of the sample mean of $n$ i.i.d. variables approaches a normal distribution as $n\to\infty$, with mean $\mu$ and variance $\dfrac{\sigma^{2}}{n}$, regardless of the population's distribution.

In hypothesis testing, what is a p-value?

The probability of observing a result at least as extreme as the data, assuming the null hypothesis $H_0$ is true. A small p-value (e.g. $< 0.05$) leads to rejecting $H_0$.

Distinguish a Type I error from a Type II error.

Type I ($\alpha$): rejecting a true null hypothesis (false positive). Type II ($\beta$): failing to reject a false null hypothesis (false negative).

What is De Morgan's law for set complements?

$(A\cup B)^{c} = A^{c}\cap B^{c}$ and $(A\cap B)^{c} = A^{c}\cup B^{c}$.

How many elements are in the power set of a set $S$ with $|S| = n$?

$2^{n}$ — the power set contains all subsets of $S$.

What are the formulas for the number of permutations and combinations of $n$ items taken $k$ at a time?

Permutations: $P(n,k) = \dfrac{n!}{(n-k)!}$. Combinations: $\binom{n}{k} = \dfrac{n!}{k!\,(n-k)!}$.

State the binomial theorem.

$(x+y)^{n} = \sum_{k=0}^{n}\binom{n}{k}x^{n-k}y^{k}$.

In graph theory, what does the Handshaking Lemma state?

The sum of all vertex degrees equals twice the number of edges: $\sum_{v} \deg(v) = 2|E|$; consequently the number of odd-degree vertices is even.

Compare the time complexity and use cases of bisection versus Newton's method for root finding.

Bisection: guaranteed linear convergence, needs a sign-change bracket $[a,b]$ where $f(a)f(b)<0$, robust but slow. Newton's method: $x_{n+1} = x_n - \dfrac{f(x_n)}{f'(x_n)}$, quadratic convergence near the root but needs the derivative and may diverge.

How does Newton's method update an estimate of a root of $f(x)=0$?

$x_{n+1} = x_n - \dfrac{f(x_n)}{f'(x_n)}$, converging quadratically near a simple root.

Compare direct (e.g. LU/Gaussian elimination) versus iterative (e.g. Jacobi, Gauss-Seidel) linear system solvers.

Direct solvers (Gaussian elimination, LU) give an exact solution in $O(n^{3})$ and suit dense systems. Iterative solvers (Jacobi, Gauss-Seidel, conjugate gradient) refine an approximation, scale better for large sparse systems, but require convergence conditions (e.g. diagonal dominance).

What does LU decomposition factor a matrix into, and why is it useful?

$A = LU$, a lower-triangular $L$ times an upper-triangular $U$. It lets you solve $A\vec{x}=\vec{b}$ efficiently via forward/back substitution, reusing the factorization for multiple right-hand sides.

State the trapezoidal rule for numerical integration over $[a,b]$ (single interval).

$\displaystyle\int_a^b f(x)\,dx \approx \dfrac{b-a}{2}\big(f(a) + f(b)\big)$.

State Simpson's rule for numerical integration over $[a,b]$ and how its accuracy compares to the trapezoidal rule.

$\displaystyle\int_a^b f(x)\,dx \approx \dfrac{b-a}{6}\Big(f(a) + 4f\!\left(\tfrac{a+b}{2}\right) + f(b)\Big)$. It fits a parabola and is more accurate (error $O(h^{4})$) than the trapezoidal rule (error $O(h^{2})$).

How does stochastic gradient descent (SGD) differ from batch gradient descent?

Batch GD computes the gradient over the entire dataset per update (accurate, slow, costly). SGD updates using one sample (or a mini-batch) per step — faster and able to escape shallow minima, but with noisier, higher-variance updates.

What advantage do momentum-based optimizers (e.g. Adam) provide over plain gradient descent?

They accumulate a velocity from past gradients (and Adam adapts per-parameter learning rates via first and second moment estimates), accelerating convergence in ravines, damping oscillations, and handling sparse or poorly scaled gradients.

What is the convexity condition for a twice-differentiable function in terms of its Hessian?

The function is convex if and only if its Hessian matrix $H = \nabla^{2}f$ is positive semidefinite ($H \succeq 0$) everywhere on the domain.