Sixth Term Examination Paper (STEP) Probability and Statistics Syllabus

State the conditions required for a binomial distribution $X\sim B(n,p)$.

A fixed number $n$ of trials; each trial has two outcomes (success/failure); constant success probability $p$ on every trial; trials are independent.

Give the pmf, mean and variance of $X\sim B(n,p)$.

$P(X=r)=\binom{n}{r}p^r(1-p)^{n-r}$ for $r=0,1,\dots,n$. Mean $E(X)=np$; variance $\operatorname{Var}(X)=np(1-p)$.

State the conditions and the pmf for the Poisson distribution $X\sim \mathrm{Po}(\lambda)$.

Events occur singly, independently, at a constant average rate over a fixed interval. $P(X=r)=\dfrac{e^{-\lambda}\lambda^r}{r!}$ for $r=0,1,2,\dots$, where $\lambda>0$ is the mean number of events.

Give the mean and variance of the Poisson distribution and a notable identity between them.

$E(X)=\lambda$ and $\operatorname{Var}(X)=\lambda$. The mean equals the variance, a characteristic property of the Poisson distribution.

When can the Poisson distribution be used to approximate the binomial, and what parameter is used?

When $n$ is large and $p$ is small (a common rule: $n>50$, $p<0.1$), $B(n,p)\approx \mathrm{Po}(\lambda)$ with $\lambda=np$. The approximation improves as $n\to\infty$, $p\to 0$ with $np$ fixed.

Give the pmf, mean and variance of the geometric distribution (number of trials up to and including the first success).

$P(X=r)=(1-p)^{r-1}p$ for $r=1,2,3,\dots$. Mean $E(X)=\dfrac{1}{p}$; variance $\operatorname{Var}(X)=\dfrac{1-p}{p^2}$.

State the memoryless property of the geometric distribution.

$P(X>m+n\mid X>m)=P(X>n)$. Having already had $m$ failures does not change the distribution of the number of further trials needed; the geometric is the only discrete distribution with this property.

Give the pmf, mean and variance of a discrete uniform distribution on $\{1,2,\dots,n\}$.

$P(X=r)=\dfrac{1}{n}$ for $r=1,\dots,n$. Mean $E(X)=\dfrac{n+1}{2}$; variance $\operatorname{Var}(X)=\dfrac{n^2-1}{12}$.

Define the probability generating function $G_X(t)$ of a non-negative integer random variable and how to recover $E(X)$.

$G_X(t)=E(t^X)=\sum_{r\geq 0}P(X=r)\,t^r$. Then $E(X)=G_X'(1)$ and $\operatorname{Var}(X)=G_X''(1)+G_X'(1)-\big(G_X'(1)\big)^2$, while $P(X=r)=\dfrac{G_X^{(r)}(0)}{r!}$.

Define the moment generating function $M_X(t)$ and state how moments are obtained from it.

$M_X(t)=E(e^{tX})$. The $n$th moment is $E(X^n)=M_X^{(n)}(0)$, the $n$th derivative evaluated at $t=0$. In particular $E(X)=M_X'(0)$ and $E(X^2)=M_X''(0)$.

How does the mgf behave for a sum of independent variables and under a linear transformation?

If $X,Y$ are independent, $M_{X+Y}(t)=M_X(t)\,M_Y(t)$. For constants $a,b$, $M_{aX+b}(t)=e^{bt}M_X(at)$.

For a continuous random variable, define the probability density function $f(x)$ and the conditions it satisfies.

$f(x)\geq 0$ for all $x$, and $\displaystyle\int_{-\infty}^{\infty} f(x)\,dx=1$. Probabilities come from areas: $P(a\leq X\leq b)=\displaystyle\int_a^b f(x)\,dx$, and $P(X=x)=0$ for any single point.

State the relationship between the pdf $f(x)$ and the cdf $F(x)$ of a continuous variable.

$F(x)=\displaystyle\int_{-\infty}^{x} f(u)\,du$ and, where $F$ is differentiable, $f(x)=F'(x)=\dfrac{dF}{dx}$.

Give the formulas for $E(X)$ and $\operatorname{Var}(X)$ of a continuous random variable with pdf $f(x)$.

$E(X)=\displaystyle\int_{-\infty}^{\infty} x f(x)\,dx$ and $\operatorname{Var}(X)=\displaystyle\int_{-\infty}^{\infty}(x-\mu)^2 f(x)\,dx=E(X^2)-\mu^2$, where $E(X^2)=\int x^2 f(x)\,dx$.

How are the median and mode of a continuous random variable found from $f$ and $F$?

The median $m$ satisfies $F(m)=\dfrac{1}{2}$, i.e. $\displaystyle\int_{-\infty}^{m} f(x)\,dx=\tfrac12$. The mode is the value of $x$ maximising the pdf $f(x)$, found where $f'(x)=0$ (a maximum) within the range.

Give the pdf, mean and variance of the continuous uniform distribution on $[a,b]$.

$f(x)=\dfrac{1}{b-a}$ for $a\leq x\leq b$. Mean $E(X)=\dfrac{a+b}{2}$; variance $\operatorname{Var}(X)=\dfrac{(b-a)^2}{12}$.

Give the pdf, cdf, mean and variance of the exponential distribution with rate $\lambda$.

$f(x)=\lambda e^{-\lambda x}$ for $x\geq 0$; $F(x)=1-e^{-\lambda x}$. Mean $E(X)=\dfrac{1}{\lambda}$; variance $\operatorname{Var}(X)=\dfrac{1}{\lambda^2}$. It is memoryless.

Give the pdf of the normal distribution $X\sim N(\mu,\sigma^2)$ and its key shape properties.

$f(x)=\dfrac{1}{\sigma\sqrt{2\pi}}\exp\!\left(-\dfrac{(x-\mu)^2}{2\sigma^2}\right)$. It is symmetric and bell-shaped about $x=\mu$, with mean = median = mode = $\mu$ and variance $\sigma^2$; points of inflection are at $x=\mu\pm\sigma$.

How do you standardise a normal variable, and what is the standard normal distribution?

If $X\sim N(\mu,\sigma^2)$ then $Z=\dfrac{X-\mu}{\sigma}\sim N(0,1)$, the standard normal. Probabilities are read using $\Phi(z)=P(Z\leq z)$, with $\Phi(-z)=1-\Phi(z)$.

State the approximate proportions of a normal distribution within $1$, $2$ and $3$ standard deviations of the mean.

About $68\%$ lies within $\mu\pm\sigma$, about $95\%$ within $\mu\pm 2\sigma$ (more precisely $1.96\sigma$), and about $99.7\%$ within $\mu\pm 3\sigma$ (the empirical/68–95–99.7 rule).

Describe the cdf method for finding the distribution of $Y=g(X)$ for a monotonic increasing $g$.

Compute $F_Y(y)=P(Y\leq y)=P(g(X)\leq y)=P(X\leq g^{-1}(y))=F_X\big(g^{-1}(y)\big)$, then differentiate to get $f_Y(y)=f_X\big(g^{-1}(y)\big)\left|\dfrac{dx}{dy}\right|$.

State the change-of-variable (Jacobian) formula for the pdf of $Y=g(X)$ when $g$ is monotonic.

$f_Y(y)=f_X\big(g^{-1}(y)\big)\left|\dfrac{d}{dy}g^{-1}(y)\right|$. The absolute value of the derivative ensures the resulting density is non-negative.

Distinguish the mean, median and mode as measures of location.

The mean is the arithmetic average $\bar{x}=\dfrac{1}{n}\sum x_i$ (sensitive to outliers). The median is the middle value when ordered (robust to outliers). The mode is the most frequently occurring value. For a symmetric distribution they coincide; for right-skew, mean > median > mode.

Define range, interquartile range, variance and standard deviation as measures of spread.

Range $=$ max $-$ min. Interquartile range $\mathrm{IQR}=Q_3-Q_1$. Variance $s^2=\dfrac{1}{n}\sum(x_i-\bar x)^2$ (or with $n-1$ for a sample estimate). Standard deviation $s=\sqrt{s^2}$, in the same units as the data.

How are outliers commonly identified using the IQR, and what does skewness indicate?

An outlier is typically a value below $Q_1-1.5\,\mathrm{IQR}$ or above $Q_3+1.5\,\mathrm{IQR}$. Positive (right) skew has a long upper tail with mean > median; negative (left) skew has a long lower tail with mean < median.

State the distribution of the sample mean $\bar X$ for a sample of size $n$ from a population with mean $\mu$ and variance $\sigma^2$.

$E(\bar X)=\mu$ and $\operatorname{Var}(\bar X)=\dfrac{\sigma^2}{n}$, so the standard error is $\dfrac{\sigma}{\sqrt{n}}$. If the population is normal, $\bar X\sim N\!\left(\mu,\dfrac{\sigma^2}{n}\right)$ exactly.

State the Central Limit Theorem.

For independent identically distributed variables with mean $\mu$ and finite variance $\sigma^2$, the sample mean $\bar X$ is approximately $N\!\left(\mu,\dfrac{\sigma^2}{n}\right)$ for large $n$, regardless of the population distribution. Equivalently $\dfrac{\bar X-\mu}{\sigma/\sqrt{n}}\to N(0,1)$.

What is an unbiased estimator, and give unbiased estimators of the population mean and variance.

An estimator $\hat\theta$ is unbiased if $E(\hat\theta)=\theta$. The sample mean $\bar X$ is unbiased for $\mu$. The sample variance $s^2=\dfrac{1}{n-1}\sum(X_i-\bar X)^2$ is an unbiased estimator of $\sigma^2$ (the $n-1$ divisor corrects the bias).

Give the formula for a $100(1-\alpha)\%$ confidence interval for a mean with known variance $\sigma^2$.

$\bar x \pm z_{\alpha/2}\dfrac{\sigma}{\sqrt{n}}$, where $z_{\alpha/2}$ is the standard normal critical value (e.g. $1.96$ for $95\%$, $2.576$ for $99\%$). The interval widens with larger confidence and narrows with larger $n$.

In hypothesis testing, define the null and alternative hypotheses, the significance level, and Type I and Type II errors.

$H_0$ is the default claim being tested; $H_1$ is the alternative. The significance level $\alpha$ is $P(\text{reject }H_0\mid H_0\text{ true})$. A Type I error rejects a true $H_0$ (probability $\alpha$); a Type II error fails to reject a false $H_0$ (probability $\beta$); the power is $1-\beta$.

Define the product moment correlation coefficient $r$ and the equation of the least-squares regression line of $y$ on $x$.

$r=\dfrac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}$ with $-1\leq r\leq 1$, measuring linear association. The regression line of $y$ on $x$ is $y-\bar y=b(x-\bar x)$ with gradient $b=\dfrac{S_{xy}}{S_{xx}}$, where $S_{xy}=\sum xy-\dfrac{(\sum x)(\sum y)}{n}$ and $S_{xx}=\sum x^2-\dfrac{(\sum x)^2}{n}$.