Machine Learning Mathematics Flashcards

24 sample cards from the Mathematics deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. When does $P(A\cap B) = P(A)\,P(B)$ hold?

   When events $A$ and $B$ are independent.

2. State the general addition rule for $P(A\cup B)$.

   $P(A\cup B) = P(A) + P(B) - P(A\cap B)$.

3. What are the definitions of expectation $E[X]$ for a discrete random variable and its variance?

   $E[X] = \sum_i x_i\,P(x_i)$ and $\operatorname{Var}(X) = E[(X-E[X])^{2}] = E[X^{2}] - (E[X])^{2}$.

4. What is the probability density function of the normal distribution $\mathcal{N}(\mu,\sigma^2)$?

   $f(x) = \dfrac{1}{\sigma\sqrt{2\pi}}\,\exp\!\left(-\dfrac{(x-\mu)^{2}}{2\sigma^{2}}\right)$.

5. How do the mean, median, and mode compare, and which is most robust to outliers?

   Mean is the arithmetic average, median the middle value, mode the most frequent value. The median is most robust to outliers; the mean is most sensitive.

6. What is the difference between population variance and sample variance formulas?

   Population: $\sigma^{2} = \dfrac{1}{N}\sum (x_i-\mu)^{2}$. Sample (Bessel's correction): $s^{2} = \dfrac{1}{n-1}\sum (x_i-\bar{x})^{2}$.

7. Define the standard deviation in terms of variance.

   $\sigma = \sqrt{\operatorname{Var}(X)}$ — the square root of the variance, in the same units as the data.

8. What does the Central Limit Theorem state?

   The distribution of the sample mean of $n$ i.i.d. variables approaches a normal distribution as $n\to\infty$, with mean $\mu$ and variance $\dfrac{\sigma^{2}}{n}$, regardless of the population's distribution.

9. In hypothesis testing, what is a p-value?

   The probability of observing a result at least as extreme as the data, assuming the null hypothesis $H_0$ is true. A small p-value (e.g. $< 0.05$) leads to rejecting $H_0$.

10. Distinguish a Type I error from a Type II error.

    Type I ($\alpha$): rejecting a true null hypothesis (false positive). Type II ($\beta$): failing to reject a false null hypothesis (false negative).

11. What is De Morgan's law for set complements?

    $(A\cup B)^{c} = A^{c}\cap B^{c}$ and $(A\cap B)^{c} = A^{c}\cup B^{c}$.

12. How many elements are in the power set of a set $S$ with $|S| = n$?

    $2^{n}$ — the power set contains all subsets of $S$.

13. What are the formulas for the number of permutations and combinations of $n$ items taken $k$ at a time?

    Permutations: $P(n,k) = \dfrac{n!}{(n-k)!}$. Combinations: $\binom{n}{k} = \dfrac{n!}{k!\,(n-k)!}$.

14. State the binomial theorem.

    $(x+y)^{n} = \sum_{k=0}^{n}\binom{n}{k}x^{n-k}y^{k}$.

15. In graph theory, what does the Handshaking Lemma state?

    The sum of all vertex degrees equals twice the number of edges: $\sum_{v} \deg(v) = 2|E|$; consequently the number of odd-degree vertices is even.

16. Compare the time complexity and use cases of bisection versus Newton's method for root finding.

    Bisection: guaranteed linear convergence, needs a sign-change bracket $[a,b]$ where $f(a)f(b)<0$, robust but slow. Newton's method: $x_{n+1} = x_n - \dfrac{f(x_n)}{f'(x_n)}$, quadratic convergence near the root but needs the derivative and may diverge.

17. How does Newton's method update an estimate of a root of $f(x)=0$?

    $x_{n+1} = x_n - \dfrac{f(x_n)}{f'(x_n)}$, converging quadratically near a simple root.

18. Compare direct (e.g. LU/Gaussian elimination) versus iterative (e.g. Jacobi, Gauss-Seidel) linear system solvers.

    Direct solvers (Gaussian elimination, LU) give an exact solution in $O(n^{3})$ and suit dense systems. Iterative solvers (Jacobi, Gauss-Seidel, conjugate gradient) refine an approximation, scale better for large sparse systems, but require convergence conditions (e.g. diagonal dominance).

19. What does LU decomposition factor a matrix into, and why is it useful?

    $A = LU$, a lower-triangular $L$ times an upper-triangular $U$. It lets you solve $A\vec{x}=\vec{b}$ efficiently via forward/back substitution, reusing the factorization for multiple right-hand sides.

20. State the trapezoidal rule for numerical integration over $[a,b]$ (single interval).

    $\displaystyle\int_a^b f(x)\,dx \approx \dfrac{b-a}{2}\big(f(a) + f(b)\big)$.

21. State Simpson's rule for numerical integration over $[a,b]$ and how its accuracy compares to the trapezoidal rule.

    $\displaystyle\int_a^b f(x)\,dx \approx \dfrac{b-a}{6}\Big(f(a) + 4f\!\left(\tfrac{a+b}{2}\right) + f(b)\Big)$. It fits a parabola and is more accurate (error $O(h^{4})$) than the trapezoidal rule (error $O(h^{2})$).

22. How does stochastic gradient descent (SGD) differ from batch gradient descent?

    Batch GD computes the gradient over the entire dataset per update (accurate, slow, costly). SGD updates using one sample (or a mini-batch) per step — faster and able to escape shallow minima, but with noisier, higher-variance updates.

23. What advantage do momentum-based optimizers (e.g. Adam) provide over plain gradient descent?

    They accumulate a velocity from past gradients (and Adam adapts per-parameter learning rates via first and second moment estimates), accelerating convergence in ravines, damping oscillations, and handling sparse or poorly scaled gradients.

24. What is the convexity condition for a twice-differentiable function in terms of its Hessian?

    The function is convex if and only if its Hessian matrix $H = \nabla^{2}f$ is positive semidefinite ($H \succeq 0$) everywhere on the domain.

What this deck covers

Mathematics flashcards FAQ