GATE DA & AI Engineering Probability and Statistics Syllabus

Define variance and standard deviation of a random variable $X$.

$$\operatorname{Var}(X) = E[(X-\mu)^{2}] = E[X^{2}] - (E[X])^{2}, \qquad \sigma = \sqrt{\operatorname{Var}(X)}$$

Define the covariance of two random variables $X$ and $Y$.

$$\operatorname{Cov}(X,Y) = E[(X-\mu_X)(Y-\mu_Y)] = E[XY] - E[X]E[Y]$$

Define the Pearson correlation coefficient $\rho_{XY}$.

$$\rho_{XY} = \frac{\operatorname{Cov}(X,Y)}{\sigma_X\,\sigma_Y}, \qquad -1 \leq \rho_{XY} \leq 1$$

How do correlation and covariance differ?

Covariance measures the direction of joint variability but is unbounded and scale-dependent; correlation is covariance normalized by the standard deviations, giving a dimensionless value in $[-1,1]$.

Does zero correlation imply independence?

No. Independence implies $\operatorname{Cov}(X,Y)=0$, but zero covariance/correlation does not imply independence (except for jointly normal variables).

What is a random variable?

A random variable is a function that assigns a real number to each outcome in the sample space of a random experiment.

What distinguishes a discrete random variable from a continuous one?

A discrete random variable takes countable values described by a probability mass function (PMF); a continuous one takes values over an interval described by a probability density function (PDF).

What two properties must a probability mass function $p(x)$ satisfy?

$$p(x) \geq 0 \quad \text{for all } x, \qquad \sum_x p(x) = 1$$

Give the PMF, mean, and variance of a discrete uniform distribution over $\{1, 2, \ldots, n\}$.

$P(X=k)=\frac{1}{n}$; mean $\mu = \frac{n+1}{2}$; variance $\sigma^{2} = \frac{n^{2}-1}{12}$.

Give the PMF, mean, and variance of a Bernoulli($p$) random variable.

$P(X=1)=p,\ P(X=0)=1-p$; mean $= p$; variance $= p(1-p)$.

Give the PMF, mean, and variance of a Binomial($n,p$) random variable.

$$P(X=k) = \binom{n}{k} p^{k}(1-p)^{n-k};\quad \mu = np;\quad \sigma^{2} = np(1-p)$$

What two properties must a probability density function $f(x)$ satisfy?

$$f(x) \geq 0 \quad \text{for all } x, \qquad \int_{-\infty}^{\infty} f(x)\,dx = 1$$

How is the probability of an interval found from a continuous PDF $f(x)$?

$$P(a \leq X \leq b) = \int_{a}^{b} f(x)\,dx$$

Give the PDF, mean, and variance of a continuous uniform distribution on $[a,b]$.

$f(x)=\frac{1}{b-a}$ for $a\leq x\leq b$; mean $=\frac{a+b}{2}$; variance $=\frac{(b-a)^{2}}{12}$.

Give the PDF, mean, and variance of an Exponential($\lambda$) distribution.

$f(x)=\lambda e^{-\lambda x}$ for $x\geq 0$; mean $=\frac{1}{\lambda}$; variance $=\frac{1}{\lambda^{2}}$.

What is the memoryless property of the exponential distribution?

$$P(X > s + t \mid X > s) = P(X > t)$$ The exponential (and geometric) distribution is the only continuous distribution with this property.

Give the PMF, mean, and variance of a Poisson($\lambda$) distribution.

$$P(X=k) = \frac{\lambda^{k} e^{-\lambda}}{k!};\quad \mu = \lambda;\quad \sigma^{2} = \lambda$$

Give the PDF of a Normal distribution $N(\mu, \sigma^{2})$.

$$f(x) = \frac{1}{\sigma\sqrt{2\pi}}\, e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}$$

What is the standard normal distribution and how is a normal variable standardized?

The standard normal $N(0,1)$ has mean $0$ and variance $1$. Any $X \sim N(\mu,\sigma^{2})$ is standardized via $$Z = \frac{X - \mu}{\sigma}$$

What is the t-distribution and how does it compare to the standard normal?

The Student's t-distribution is bell-shaped and symmetric about 0 but has heavier tails. As degrees of freedom $\nu \to \infty$ it approaches the standard normal. It is used for small-sample inference about a mean with unknown variance.

How is the chi-squared distribution constructed, and what are its mean and variance?

If $Z_1,\ldots,Z_k$ are independent standard normals, then $\sum_{i=1}^{k} Z_i^{2} \sim \chi^{2}_k$. Mean $= k$; variance $= 2k$.

Define the cumulative distribution function (CDF) of a random variable $X$.

$$F(x) = P(X \leq x)$$ It is non-decreasing, right-continuous, with $F(-\infty)=0$ and $F(\infty)=1$.

How are the PDF and CDF of a continuous random variable related?

$$F(x) = \int_{-\infty}^{x} f(t)\,dt, \qquad f(x) = \frac{dF(x)}{dx}$$

How is the CDF used to compute $P(a < X \leq b)$?

$$P(a < X \leq b) = F(b) - F(a)$$

Define the conditional PDF of $X$ given $Y=y$.

$$f_{X\mid Y}(x \mid y) = \frac{f_{X,Y}(x,y)}{f_Y(y)}, \qquad f_Y(y) > 0$$

State the Central Limit Theorem.

For i.i.d. random variables with mean $\mu$ and finite variance $\sigma^{2}$, the sample mean's standardized sum converges in distribution to standard normal: $$\frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \xrightarrow{d} N(0,1)$$ as $n \to \infty$.

What are the mean and standard deviation (standard error) of the sample mean $\bar{X}_n$?

$$E[\bar{X}_n] = \mu, \qquad \operatorname{SD}(\bar{X}_n) = \frac{\sigma}{\sqrt{n}}$$

What is the linearity of expectation property?

$$E[aX + bY + c] = a\,E[X] + b\,E[Y] + c$$ It holds regardless of whether $X$ and $Y$ are independent.

How does variance behave under scaling and for a sum of independent variables?

$\operatorname{Var}(aX + b) = a^{2}\operatorname{Var}(X)$. For independent $X,Y$: $\operatorname{Var}(X+Y) = \operatorname{Var}(X) + \operatorname{Var}(Y)$.

For a normal distribution, what fraction of data lies within 1, 2, and 3 standard deviations (empirical rule)?

Approximately $68\%$ within $\mu \pm \sigma$, $95\%$ within $\mu \pm 2\sigma$, and $99.7\%$ within $\mu \pm 3\sigma$.