Probability Syllabus

What is the condition for two events $A$ and $B$ to be independent?

They are independent if $P(A \cap B) = P(A)\,P(B)$, equivalently $P(A \mid B) = P(A)$.

How does independence differ from mutual exclusivity?

Mutually exclusive events cannot both occur ($P(A \cap B)=0$); independent events do not influence each other's probability ($P(A \cap B)=P(A)P(B)$). Two events with positive probability cannot be both.

Define a random variable.

A random variable is a function that assigns a real number to each outcome in the sample space of an experiment.

What distinguishes a discrete random variable from a continuous one?

A discrete random variable takes countably many values (e.g., integers); a continuous random variable takes uncountably many values over an interval of real numbers.

What is a probability mass function (PMF) and what two properties must it satisfy?

A PMF $p(x) = P(X = x)$ gives the probability of each value of a discrete random variable. It must satisfy $p(x) \geq 0$ and $\sum_x p(x) = 1$.

What is a probability density function (PDF) and its defining properties?

A PDF $f(x)$ describes a continuous random variable, with $f(x) \geq 0$ and $\int_{-\infty}^{\infty} f(x)\,dx = 1$. Probabilities are areas: $P(a \leq X \leq b) = \int_a^b f(x)\,dx$.

For a continuous random variable, what is $P(X = a)$ for a single point $a$?

$P(X = a) = 0$; probabilities for continuous variables are only nonzero over intervals.

Define the cumulative distribution function (CDF).

$$F(x) = P(X \leq x)$$ It is non-decreasing, right-continuous, with $\lim_{x \to -\infty} F(x) = 0$ and $\lim_{x \to \infty} F(x) = 1$.

Define the expected value of a discrete random variable $X$.

$$E[X] = \sum_{x} x\,P(X = x)$$

Define the expected value of a continuous random variable $X$.

$$E[X] = \int_{-\infty}^{\infty} x\,f(x)\,dx$$

Define the variance of a random variable and give its computational formula.

$$\operatorname{Var}(X) = E\big[(X - \mu)^{2}\big] = E[X^{2}] - (E[X])^{2}$$

How is the standard deviation related to the variance?

The standard deviation is the square root of the variance: $\sigma = \sqrt{\operatorname{Var}(X)}$.

State the linearity of expectation for constants $a, b$ and random variables $X, Y$.

$$E[aX + bY + c] = a\,E[X] + b\,E[Y] + c$$ (holds even if $X$ and $Y$ are dependent).

How does variance behave under $\operatorname{Var}(aX + b)$?

$$\operatorname{Var}(aX + b) = a^{2}\,\operatorname{Var}(X)$$ (the additive constant $b$ has no effect).

What conditions define a binomial experiment?

A fixed number $n$ of independent trials, each with only two outcomes (success/failure) and the same success probability $p$.

State the binomial probability mass function.

$$P(X = k) = \binom{n}{k} p^{k} (1-p)^{n-k}, \quad k = 0, 1, \dots, n$$

Give the mean and variance of a binomial distribution $X \sim \text{Bin}(n, p)$.

$$E[X] = np, \qquad \operatorname{Var}(X) = np(1-p)$$

When is the Poisson distribution used, and what parameter defines it?

It models the number of events occurring in a fixed interval of time or space, given a constant average rate and independence. It is defined by the rate $\lambda$ (mean number of events).

State the Poisson probability mass function.

$$P(X = k) = \frac{\lambda^{k} e^{-\lambda}}{k!}, \quad k = 0, 1, 2, \dots$$

What are the mean and variance of a Poisson distribution?

Both equal $\lambda$: $E[X] = \operatorname{Var}(X) = \lambda$.

Under what condition can the binomial distribution be approximated by the Poisson distribution?

When $n$ is large and $p$ is small, with $\lambda = np$ moderate; then $\text{Bin}(n,p) \approx \text{Poisson}(np)$.

State the probability density function of the normal distribution $N(\mu, \sigma^{2})$.

$$f(x) = \frac{1}{\sigma\sqrt{2\pi}}\,e^{-\frac{(x-\mu)^{2}}{2\sigma^{2}}}$$

What are the parameters of the standard normal distribution, and how do you standardize a value?

The standard normal has $\mu = 0$ and $\sigma = 1$. A value is standardized via the z-score: $z = \dfrac{x - \mu}{\sigma}$.

State the empirical (68–95–99.7) rule for the normal distribution.

About $68\%$ of data falls within $\mu \pm \sigma$, about $95\%$ within $\mu \pm 2\sigma$, and about $99.7\%$ within $\mu \pm 3\sigma$.

State the probability density function of the exponential distribution with rate $\lambda$.

$$f(x) = \lambda e^{-\lambda x}, \quad x \geq 0$$

Give the mean and variance of an exponential distribution with rate $\lambda$.

$$E[X] = \frac{1}{\lambda}, \qquad \operatorname{Var}(X) = \frac{1}{\lambda^{2}}$$

What special property makes the exponential distribution unique among continuous distributions?

The memoryless property: $P(X > s + t \mid X > s) = P(X > t)$; the past waiting time does not affect the future.

What is the relationship between the Poisson and exponential distributions?

If events occur as a Poisson process with rate $\lambda$, the number of events in an interval is Poisson-distributed, while the waiting time between consecutive events is exponentially distributed with rate $\lambda$.

State the (weak) Law of Large Numbers.

As the number of independent, identically distributed trials $n$ grows, the sample mean converges to the expected value: $\bar{X}_n \xrightarrow{P} \mu$ as $n \to \infty$.

State the Central Limit Theorem.

For i.i.d. random variables with mean $\mu$ and finite variance $\sigma^{2}$, the standardized sample mean converges in distribution to a standard normal: $$\frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} \xrightarrow{d} N(0, 1)$$ regardless of the original distribution's shape.

Contrast what the Law of Large Numbers and the Central Limit Theorem each tell us about the sample mean.

The LLN says the sample mean converges to the true mean $\mu$ (a statement about the limit value); the CLT describes the shape and spread of the sample mean's distribution around $\mu$, showing it becomes approximately normal with standard error $\sigma/\sqrt{n}$.