GATE Statistics Standard discrete and continuous univariate distributions Syllabus

If $X_i \sim N(\mu_i, \sigma_i^{2})$ are independent, what is the distribution of $\sum_i a_i X_i$?

$$\sum_i a_i X_i \sim N\!\left(\sum_i a_i \mu_i,\ \sum_i a_i^{2}\sigma_i^{2}\right).$$ Any linear combination of independent normals is normal.

State the pdf, mean, and variance of an Exponential random variable $X \sim \text{Exp}(\lambda)$ (rate parameterization).

$$f(x) = \lambda e^{-\lambda x},\quad x \ge 0,$$ with $E(X) = \frac{1}{\lambda}$ and $\operatorname{Var}(X) = \frac{1}{\lambda^{2}}$.

State the memorylessness property of the Exponential distribution.

$P(X > s+t \mid X > s) = P(X > t)$. It is the only continuous distribution with the memoryless property.

What is the cumulative distribution function (cdf) of $X \sim \text{Exp}(\lambda)$?

$$F(x) = 1 - e^{-\lambda x}, \quad x \ge 0.$$

State the pdf of a Gamma random variable $X \sim \text{Gamma}(\alpha, \beta)$ (shape $\alpha$, rate $\beta$).

$$f(x) = \frac{\beta^{\alpha}}{\Gamma(\alpha)} x^{\alpha-1} e^{-\beta x}, \quad x > 0.$$

What are the mean and variance of $X \sim \text{Gamma}(\alpha,\beta)$ with rate $\beta$?

$E(X) = \frac{\alpha}{\beta}$ and $\operatorname{Var}(X) = \frac{\alpha}{\beta^{2}}$.

How is the Exponential distribution related to the Gamma distribution?

$\text{Exp}(\lambda) = \text{Gamma}(\alpha=1, \beta=\lambda)$. The sum of $n$ i.i.d. $\text{Exp}(\lambda)$ variables is $\text{Gamma}(n, \lambda)$ (an Erlang distribution).

Define the Gamma function $\Gamma(\alpha)$ and give its value at integers and at $\tfrac12$.

$$\Gamma(\alpha) = \int_0^{\infty} x^{\alpha-1}e^{-x}\,dx.$$ For a positive integer $n$, $\Gamma(n) = (n-1)!$; also $\Gamma\!\left(\tfrac12\right) = \sqrt{\pi}$ and $\Gamma(\alpha+1) = \alpha\Gamma(\alpha)$.

State the pdf of a Beta random variable $X \sim \text{Beta}(\alpha,\beta)$.

$$f(x) = \frac{x^{\alpha-1}(1-x)^{\beta-1}}{B(\alpha,\beta)}, \quad 0 < x < 1,$$ where $B(\alpha,\beta) = \frac{\Gamma(\alpha)\Gamma(\beta)}{\Gamma(\alpha+\beta)}$.

What are the mean and variance of $X \sim \text{Beta}(\alpha,\beta)$?

$E(X) = \frac{\alpha}{\alpha+\beta}$ and $\operatorname{Var}(X) = \frac{\alpha\beta}{(\alpha+\beta)^{2}(\alpha+\beta+1)}$.

State the pdf of a Chi-square random variable with $n$ degrees of freedom, $X \sim \chi^{2}_{n}$.

$$f(x) = \frac{1}{2^{n/2}\Gamma(n/2)} x^{n/2 - 1} e^{-x/2}, \quad x > 0.$$ It is $\text{Gamma}(\alpha = n/2, \beta = 1/2)$.

What are the mean and variance of $X \sim \chi^{2}_{n}$?

$E(X) = n$ and $\operatorname{Var}(X) = 2n$.

How does the Chi-square distribution arise from standard normal variables?

If $Z_1,\dots,Z_n$ are i.i.d. $N(0,1)$, then $\sum_{i=1}^{n} Z_i^{2} \sim \chi^{2}_{n}$.

Define a Student's $t$ random variable in terms of standard normal and chi-square variables.

If $Z \sim N(0,1)$ and $V \sim \chi^{2}_{n}$ are independent, then $$T = \frac{Z}{\sqrt{V/n}} \sim t_{n}.$$

What are the mean and variance of a Student's $t$ distribution with $n$ degrees of freedom?

$E(T) = 0$ for $n > 1$ (undefined for $n=1$), and $\operatorname{Var}(T) = \frac{n}{n-2}$ for $n > 2$.

Define an $F$ random variable in terms of two independent chi-square variables.

If $U \sim \chi^{2}_{m}$ and $V \sim \chi^{2}_{n}$ are independent, then $$F = \frac{U/m}{V/n} \sim F_{m,n}.$$

State the relationship between the $t$ and $F$ distributions.

If $T \sim t_{n}$, then $T^{2} \sim F_{1,n}$.

What is the mean of an $F_{m,n}$ random variable?

$E(F) = \frac{n}{n-2}$ for $n > 2$ (it does not depend on the numerator degrees of freedom $m$).

State the pdf of a standard Cauchy random variable and a key fact about its moments.

$$f(x) = \frac{1}{\pi(1+x^{2})}, \quad -\infty < x < \infty.$$ The Cauchy distribution has no finite mean or variance; its mgf does not exist. It equals the ratio of two independent $N(0,1)$ variables and $t_1$.

State the pdf of a Lognormal random variable: $X = e^{Y}$ where $Y \sim N(\mu, \sigma^{2})$.

$$f(x) = \frac{1}{x\sigma\sqrt{2\pi}} \exp\!\left(-\frac{(\ln x - \mu)^{2}}{2\sigma^{2}}\right), \quad x > 0.$$

What are the mean and variance of a Lognormal random variable with parameters $\mu, \sigma^{2}$?

$E(X) = e^{\mu + \sigma^{2}/2}$ and $\operatorname{Var}(X) = \left(e^{\sigma^{2}} - 1\right)e^{2\mu + \sigma^{2}}$.

State the pdf and cdf of a Weibull random variable with shape $k$ and scale $\lambda$.

$$f(x) = \frac{k}{\lambda}\left(\frac{x}{\lambda}\right)^{k-1} e^{-(x/\lambda)^{k}}, \quad x \ge 0,$$ and $F(x) = 1 - e^{-(x/\lambda)^{k}}$. When $k=1$ it reduces to the Exponential.

State the pdf of a Laplace (double exponential) random variable with location $\mu$ and scale $b$.

$$f(x) = \frac{1}{2b}\exp\!\left(-\frac{|x-\mu|}{b}\right), \quad -\infty < x < \infty,$$ with $E(X)=\mu$ and $\operatorname{Var}(X)=2b^{2}$.

What is the moment generating function of $X \sim \text{Exp}(\lambda)$ (rate $\lambda$), and for which $t$ does it exist?

$$M_X(t) = \frac{\lambda}{\lambda - t}, \quad t < \lambda.$$

For $X \sim \text{Gamma}(\alpha,\beta)$ with rate $\beta$, state the moment generating function.

$$M_X(t) = \left(\frac{\beta}{\beta - t}\right)^{\alpha}, \quad t < \beta.$$

Distinguish between the discrete and continuous classes of distributions in terms of their probability functions.

Discrete distributions assign positive probability to individual points via a pmf $P(X=x)$ summing to 1; continuous distributions have a density $f(x)$ with $P(X=x)=0$ for any point and $\int f(x)\,dx = 1$. Probabilities for continuous variables come from integrating the pdf over an interval.

What is the maximum likelihood estimator (MLE) of $p$ for an i.i.d. sample $X_1,\dots,X_n \sim \text{Bernoulli}(p)$?

$\hat{p} = \bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i$, the sample proportion of successes.

What is the MLE of $\lambda$ for an i.i.d. sample from $\text{Poisson}(\lambda)$?

$\hat{\lambda} = \bar{X} = \frac{1}{n}\sum_{i=1}^{n} X_i$, the sample mean.

What is the MLE of $\lambda$ (rate) for an i.i.d. sample from $\text{Exp}(\lambda)$?

$\hat{\lambda} = \frac{1}{\bar{X}} = \frac{n}{\sum_{i=1}^{n} X_i}$.

Compare the relationship between the mean and variance for the Binomial, Poisson, and Geometric distributions.

Binomial: $\operatorname{Var} = np(1-p) < np = E$ (underdispersed relative to Poisson). Poisson: $\operatorname{Var} = E = \lambda$ (equidispersed). Geometric (on $\{1,2,\dots\}$): $\operatorname{Var} = \frac{1-p}{p^{2}} > \frac{1}{p} = E$ (overdispersed).