GATE Statistics Probability Syllabus

Define the expectation $E[X]$ for discrete and for continuous random variables.

Discrete: $E[X] = \sum_{x} x\,p_X(x)$. Continuous: $E[X] = \int_{-\infty}^{\infty} x\,f_X(x)\,dx$, provided the sum/integral converges absolutely.

State the law of the unconscious statistician (LOTUS) for $E[g(X)]$.

Discrete: $E[g(X)] = \sum_{x} g(x)\,p_X(x)$. Continuous: $E[g(X)] = \int_{-\infty}^{\infty} g(x)\,f_X(x)\,dx$.

State the linearity of expectation property.

$$E[aX + bY + c] = a\,E[X] + b\,E[Y] + c,$$ for constants $a,b,c$ — holds whether or not $X$ and $Y$ are independent.

Define the $k$-th raw moment and the $k$-th central moment of $X$.

Raw moment about origin: $\mu_k' = E[X^{k}]$. Central moment about the mean: $\mu_k = E\big[(X-\mu)^{k}\big]$, where $\mu = E[X]$.

Express the variance in terms of moments, and state its scaling rule.

$$\operatorname{Var}(X) = E[X^{2}] - (E[X])^{2} = \mu_2.$$ Scaling: $\operatorname{Var}(aX+b) = a^{2}\operatorname{Var}(X)$.

Define skewness and kurtosis in terms of central moments.

Skewness $\gamma_1 = \dfrac{\mu_3}{\mu_2^{3/2}} = \dfrac{E[(X-\mu)^3]}{\sigma^3}$. Kurtosis $\beta_2 = \dfrac{\mu_4}{\mu_2^{2}} = \dfrac{E[(X-\mu)^4]}{\sigma^4}$ (excess kurtosis $= \beta_2 - 3$).

Define the moment generating function (MGF) $M_X(t)$ of a random variable $X$.

$$M_X(t) = E\big[e^{tX}\big],$$ defined for $t$ in an open interval around $0$ where the expectation is finite.

How are moments obtained from the MGF $M_X(t)$?

By differentiation at $0$: $E[X^{n}] = M_X^{(n)}(0) = \left.\dfrac{d^{n}}{dt^{n}} M_X(t)\right|_{t=0}$. Equivalently, $M_X(t) = \sum_{n=0}^{\infty} \dfrac{E[X^n]}{n!} t^{n}$.

State the MGF rule for a linear transformation $Y = aX + b$.

$$M_{aX+b}(t) = e^{bt}\,M_X(at).$$

State the MGF property for the sum of independent random variables.

If $X$ and $Y$ are independent, then $M_{X+Y}(t) = M_X(t)\,M_Y(t)$. More generally, the MGF of a sum of independents is the product of their MGFs.

What is the uniqueness property of moment generating functions?

If two random variables have MGFs that are finite and equal on an open interval around $0$, they have the same distribution. Thus an MGF (when it exists) uniquely determines the distribution.

Define the $p$-th quantile (and in particular the median) of a continuous random variable $X$.

The $p$-th quantile $x_p$ satisfies $F_X(x_p) = p$, i.e. $x_p = F_X^{-1}(p)$ for $p\in(0,1)$. The median is the $0.5$ quantile, $F_X(m) = 0.5$.

Give the general definition of the $p$-th quantile valid for any (possibly discrete) distribution.

$$x_p = \inf\{x : F_X(x) \geq p\},$$ the smallest value $x$ for which the CDF reaches at least $p$. This is the generalized inverse $F_X^{-1}$.

What are the quartiles and the interquartile range (IQR)?

First quartile $Q_1 = x_{0.25}$, second $Q_2 = $ median $= x_{0.5}$, third $Q_3 = x_{0.75}$. IQR $= Q_3 - Q_1$, a measure of spread.

State the CDF method (distribution function technique) for finding the distribution of $Y = g(X)$.

Compute $F_Y(y) = P(g(X)\leq y) = P\big(X \in \{x: g(x)\leq y\}\big)$ using $F_X$, then differentiate to get $f_Y(y) = F_Y'(y)$.

State the change-of-variable (Jacobian) formula for the pdf of $Y = g(X)$ when $g$ is monotone and differentiable.

$$f_Y(y) = f_X\big(g^{-1}(y)\big)\left|\frac{d}{dy} g^{-1}(y)\right|,$$ for $y$ in the range of $g$.

State the probability integral transform.

If $X$ is continuous with CDF $F_X$, then $U = F_X(X)$ is uniformly distributed on $(0,1)$. Conversely, if $U\sim\text{Uniform}(0,1)$, then $F_X^{-1}(U)$ has CDF $F_X$ (basis of inverse-transform sampling).

For a non-monotone transformation $Y=g(X)$ with several inverse branches $x_i = g_i^{-1}(y)$, give the pdf formula.

$$f_Y(y) = \sum_{i} f_X\big(g_i^{-1}(y)\big)\left|\frac{d}{dy} g_i^{-1}(y)\right|,$$ summing over all branches mapping into $y$. E.g. for $Y=X^2$, $f_Y(y) = \dfrac{f_X(\sqrt{y}) + f_X(-\sqrt{y})}{2\sqrt{y}}$, $y>0$.

State Markov's inequality and the conditions under which it applies.

For a non-negative random variable $X$ and any $a > 0$, $$P(X \geq a) \leq \frac{E[X]}{a}.$$ Requires $X \geq 0$ and $E[X]$ finite.

State Chebyshev's inequality.

For a random variable with mean $\mu$ and finite variance $\sigma^2$, for any $k>0$, $$P\big(|X - \mu| \geq k\big) \leq \frac{\sigma^{2}}{k^{2}}.$$

Write Chebyshev's inequality in the form involving $k$ standard deviations.

$$P\big(|X - \mu| \geq k\sigma\big) \leq \frac{1}{k^{2}}, \qquad k > 0.$$ Equivalently, $P(|X-\mu| < k\sigma) \geq 1 - \tfrac{1}{k^2}$.

Show how Chebyshev's inequality is derived from Markov's inequality.

Apply Markov's inequality to the non-negative variable $(X-\mu)^2$ with threshold $k^2$: $P\big((X-\mu)^2 \geq k^2\big) \leq \dfrac{E[(X-\mu)^2]}{k^2} = \dfrac{\sigma^2}{k^2}$, and $\{(X-\mu)^2\geq k^2\} = \{|X-\mu|\geq k\}$.

Define a convex function and a concave function (twice-differentiable case).

$g$ is convex on an interval if $g''(x) \geq 0$ there (lies below its chords); $g$ is concave if $g''(x) \leq 0$ (lies above its chords). Equivalently convex: $g(\lambda a + (1-\lambda)b) \leq \lambda g(a) + (1-\lambda)g(b)$.

State Jensen's inequality for a convex function $g$.

If $g$ is convex, then $$g\big(E[X]\big) \leq E\big[g(X)\big].$$ For concave $g$ the inequality reverses: $g(E[X]) \geq E[g(X)]$.

Apply Jensen's inequality to $g(x)=x^2$ and to $g(x)=\tfrac{1}{x}$ (on $x>0$).

Since $x^2$ is convex: $E[X^2] \geq (E[X])^2$ (consistent with $\operatorname{Var}(X)\geq 0$). Since $1/x$ is convex on $x>0$: $E\left[\tfrac{1}{X}\right] \geq \tfrac{1}{E[X]}$.

Compare Markov and Chebyshev inequalities: what does each require and bound?

Markov: needs only $X\geq 0$ and $E[X]$; bounds the upper tail $P(X\geq a)$. Chebyshev: needs finite mean and variance; bounds two-sided deviation $P(|X-\mu|\geq k)$ using $\sigma^2$. Chebyshev is derived from Markov and is generally tighter for deviations from the mean.

What does the weak law of large numbers state, and how does Chebyshev's inequality establish it?

For i.i.d. $X_i$ with mean $\mu$ and finite variance $\sigma^2$, the sample mean $\bar{X}_n \to \mu$ in probability. Chebyshev gives $P(|\bar{X}_n - \mu|\geq\epsilon) \leq \dfrac{\sigma^2}{n\epsilon^2} \to 0$ as $n\to\infty$.

State the relationship between the median and the mean for a right-skewed (positively skewed) unimodal distribution.

For typical right-skewed unimodal distributions: $\text{mode} < \text{median} < \text{mean}$. For left-skewed: $\text{mean} < \text{median} < \text{mode}$. Skewness $\gamma_1 > 0$ indicates right skew.

Give the formula for variance of a sum and contrast the independent case.

$$\operatorname{Var}(X+Y) = \operatorname{Var}(X) + \operatorname{Var}(Y) + 2\operatorname{Cov}(X,Y).$$ If $X,Y$ are independent, $\operatorname{Cov}(X,Y)=0$, so $\operatorname{Var}(X+Y) = \operatorname{Var}(X)+\operatorname{Var}(Y)$.

What is the characteristic function $\varphi_X(t)$, and what advantage does it have over the MGF?

$\varphi_X(t) = E\big[e^{itX}\big]$. Advantage: it always exists for every random variable (since $|e^{itX}|=1$), whereas the MGF may not exist; like the MGF it uniquely determines the distribution.

Distinguish a discrete, continuous, and mixed random variable in terms of the behaviour of the CDF.

Discrete: CDF is a step function (jumps, flat between). Continuous: CDF is continuous (no jumps) and differentiable a.e., admitting a pdf. Mixed: CDF has both jump points (atoms) and continuous portions.