Data Science Mathematics and Statistics Flashcards

24 sample cards from the Mathematics and Statistics deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. What is the gradient of a scalar function $f(x_1,\ldots,x_n)$, and what does it represent?

   $\nabla f = \left( \dfrac{\partial f}{\partial x_1}, \ldots, \dfrac{\partial f}{\partial x_n} \right)$. It points in the direction of steepest ascent, and its magnitude is the maximum rate of increase.

2. What is the divergence of a vector field $\vec{F} = (F_1, F_2, F_3)$?

   $\nabla \cdot \vec{F} = \dfrac{\partial F_1}{\partial x} + \dfrac{\partial F_2}{\partial y} + \dfrac{\partial F_3}{\partial z}$. It is a scalar measuring the net outflow (source/sink strength) at a point.

3. What is the curl of a vector field, and what does it measure?

   $\nabla \times \vec{F} = \left( \dfrac{\partial F_3}{\partial y} - \dfrac{\partial F_2}{\partial z},\; \dfrac{\partial F_1}{\partial z} - \dfrac{\partial F_3}{\partial x},\; \dfrac{\partial F_2}{\partial x} - \dfrac{\partial F_1}{\partial y} \right)$. It is a vector measuring the local rotation (circulation) of the field.

4. What is the Hessian matrix, and how is it used to classify critical points?

   The Hessian $H$ contains second-order partials $H_{ij} = \dfrac{\partial^2 f}{\partial x_i \partial x_j}$. At a critical point: positive-definite $H \Rightarrow$ local minimum, negative-definite $\Rightarrow$ local maximum, indefinite $\Rightarrow$ saddle point.

5. State the gradient descent update rule and the role of the learning rate.

   $\theta_{t+1} = \theta_t - \eta \,\nabla J(\theta_t)$, where $\eta > 0$ is the learning rate. Too large $\eta$ may diverge/overshoot; too small $\eta$ converges slowly.

6. What is the difference between batch, stochastic, and mini-batch gradient descent?

   Batch GD uses the full dataset per update (stable, slow). Stochastic GD (SGD) uses one sample per update (fast, noisy). Mini-batch GD uses a small subset, balancing speed and stability.

7. What is the method of Lagrange multipliers used for, and what condition does it impose?

   It optimizes $f(\vec{x})$ subject to a constraint $g(\vec{x}) = 0$. At the optimum, $\nabla f = \lambda \nabla g$ for some multiplier $\lambda$; solve this together with $g(\vec{x})=0$.

8. What is the difference between a convex and a non-convex optimization problem?

   In a convex problem (convex objective over a convex set), any local minimum is the global minimum, making it efficiently solvable. Non-convex problems may have many local minima and saddle points, so solutions are not guaranteed global.

9. State the three axioms of probability (Kolmogorov).

   (1) $P(A) \geq 0$ for any event $A$; (2) $P(\Omega) = 1$ for the sample space $\Omega$; (3) For mutually exclusive events, $P\!\left(\bigcup_i A_i\right) = \sum_i P(A_i)$.

10. State the formula for conditional probability and the definition of independence.

    $P(A \mid B) = \dfrac{P(A \cap B)}{P(B)}$ for $P(B) > 0$. Events $A$ and $B$ are independent if $P(A \cap B) = P(A)P(B)$, equivalently $P(A\mid B) = P(A)$.

11. State the general addition rule for the probability of $A \cup B$.

    $P(A \cup B) = P(A) + P(B) - P(A \cap B)$. For mutually exclusive events, $P(A\cap B)=0$, so $P(A\cup B)=P(A)+P(B)$.

12. What is the difference between a discrete and a continuous random variable?

    A discrete random variable takes countable values, described by a probability mass function (PMF) $P(X=x)$. A continuous random variable takes uncountable values, described by a probability density function (PDF) $f(x)$, where $P(X=x)=0$ for any single point.

13. What is a cumulative distribution function (CDF), and what properties does it have?

    $F(x) = P(X \leq x)$. It is non-decreasing, right-continuous, with $\lim_{x\to -\infty} F(x) = 0$ and $\lim_{x\to \infty} F(x) = 1$.

14. State the PMF, mean, and variance of a Binomial distribution.

    $P(X=k) = \binom{n}{k} p^{k} (1-p)^{n-k}$, for $k=0,\ldots,n$. Mean $\mu = np$, variance $\sigma^{2} = np(1-p)$.

15. State the PMF, mean, and variance of a Poisson distribution.

    $P(X=k) = \dfrac{\lambda^{k} e^{-\lambda}}{k!}$, for $k=0,1,2,\ldots$. Mean $= \lambda$ and variance $= \lambda$. It models counts of rare events over a fixed interval.

16. State the PDF of the Normal (Gaussian) distribution and what its parameters mean.

    $f(x) = \dfrac{1}{\sigma\sqrt{2\pi}} \exp\!\left( -\dfrac{(x-\mu)^{2}}{2\sigma^{2}} \right)$, where $\mu$ is the mean (center) and $\sigma^2$ is the variance (spread).

17. What is the standard normal distribution and the z-score formula?

    The standard normal $Z \sim \mathcal{N}(0,1)$ has mean $0$ and variance $1$. The z-score $z = \dfrac{x - \mu}{\sigma}$ standardizes a value to express it in standard-deviation units from the mean.

18. Define the expectation of a discrete and a continuous random variable.

    Discrete: $\mathbb{E}[X] = \sum_x x\, P(X=x)$. Continuous: $\mathbb{E}[X] = \int_{-\infty}^{\infty} x\, f(x)\, dx$. It is the long-run average value.

19. State the definition of variance and the computational shortcut formula.

    $\operatorname{Var}(X) = \mathbb{E}\!\left[(X - \mu)^{2}\right] = \mathbb{E}[X^{2}] - (\mathbb{E}[X])^{2}$. The standard deviation is $\sigma = \sqrt{\operatorname{Var}(X)}$.

20. State the linearity of expectation and the variance scaling rule.

    $\mathbb{E}[aX + bY] = a\,\mathbb{E}[X] + b\,\mathbb{E}[Y]$ (always). For variance, $\operatorname{Var}(aX + b) = a^{2}\operatorname{Var}(X)$, and $\operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)$ only if $X,Y$ are independent.

21. State Bayes' Theorem.

    $P(A \mid B) = \dfrac{P(B \mid A)\, P(A)}{P(B)}$, where $P(A)$ is the prior, $P(B\mid A)$ the likelihood, $P(B)$ the evidence, and $P(A\mid B)$ the posterior.

22. How is the evidence $P(B)$ computed via the law of total probability in Bayes' Theorem?

    For a partition $\{A_i\}$ of the sample space, $P(B) = \sum_i P(B \mid A_i)\, P(A_i)$. This normalizing denominator ensures the posterior sums to $1$.

23. State the (weak) Law of Large Numbers.

    For i.i.d. random variables with mean $\mu$, the sample mean converges to the true mean as $n\to\infty$: $\bar{X}_n = \frac{1}{n}\sum_{i=1}^{n} X_i \xrightarrow{P} \mu$. Empirical averages stabilize around the expected value with more data.

24. State the Central Limit Theorem and why it matters.

    For i.i.d. variables with mean $\mu$ and variance $\sigma^2$, the sample mean is approximately normal for large $n$: $\bar{X}_n \approx \mathcal{N}\!\left(\mu, \dfrac{\sigma^{2}}{n}\right)$, regardless of the original distribution. It justifies using normal-based inference (confidence intervals, hypothesis tests).

What this deck covers

Mathematics and Statistics flashcards FAQ