Data Analytics Statistics and Probability for Analytics Syllabus

State the multiplication rule for the probability of $A$ and $B$.

General: $P(A \cap B) = P(A \mid B)\,P(B) = P(B \mid A)\,P(A)$. If $A$ and $B$ are independent, this simplifies to $P(A \cap B) = P(A)\,P(B)$.

What condition defines statistical independence of two events?

Events $A$ and $B$ are independent if $P(A \cap B) = P(A)\,P(B)$, equivalently $P(A \mid B) = P(A)$. Knowing one occurred does not change the probability of the other.

State Bayes' theorem.

$P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}$, where $P(B) = \sum_i P(B \mid A_i)\,P(A_i)$ by the law of total probability. It updates a prior into a posterior using observed evidence.

In Bayes' theorem, identify the prior, likelihood, and posterior.

For $P(A \mid B) = \frac{P(B \mid A) P(A)}{P(B)}$: $P(A)$ is the prior, $P(B \mid A)$ is the likelihood, $P(A \mid B)$ is the posterior, and $P(B)$ is the evidence (marginal likelihood).

Distinguish a discrete random variable from a continuous random variable.

A discrete random variable takes countable values (e.g. counts) described by a probability mass function; a continuous random variable takes values over an interval, described by a probability density function where probabilities are areas under the curve.

What properties must a valid probability mass function (PMF) satisfy?

For all $x$, $0 \leq P(X = x) \leq 1$, and the probabilities sum to one: $\sum_x P(X = x) = 1$.

For a continuous random variable, why is $P(X = a) = 0$, and how are probabilities computed?

For a continuous variable, probability is the area under the density $f(x)$: $P(a \leq X \leq b) = \int_a^b f(x)\,dx$. Any single point has zero width, so $P(X = a) = 0$.

Define the expected value of a discrete random variable.

$E[X] = \mu = \sum_x x\,P(X = x)$. It is the long-run average value, or the probability-weighted mean of all possible outcomes.

Give the definition of the variance of a random variable and its computational formula.

$\operatorname{Var}(X) = E[(X - \mu)^{2}] = E[X^{2}] - (E[X])^{2}$. It measures the expected squared deviation from the mean.

How do the linearity properties give $E[aX + b]$ and $\operatorname{Var}(aX + b)$?

$E[aX + b] = a\,E[X] + b$ and $\operatorname{Var}(aX + b) = a^{2}\operatorname{Var}(X)$. Adding a constant shifts the mean but not the variance; scaling multiplies variance by $a^{2}$.

State the parameters and key properties of the normal distribution.

The normal distribution $N(\mu, \sigma^{2})$ is symmetric and bell-shaped, defined by mean $\mu$ and standard deviation $\sigma$. Mean = median = mode, and it is fully characterized by these two parameters.

State the empirical (68-95-99.7) rule for a normal distribution.

Approximately 68% of values lie within $\mu \pm \sigma$, 95% within $\mu \pm 2\sigma$, and 99.7% within $\mu \pm 3\sigma$.

What is a z-score and how is it computed?

A z-score standardizes a value: $z = \frac{x - \mu}{\sigma}$. It gives the number of standard deviations $x$ lies from the mean, converting to the standard normal $N(0,1)$.

State the PMF, mean, and variance of the binomial distribution.

$P(X = k) = \binom{n}{k} p^{k}(1-p)^{n-k}$, with mean $\mu = np$ and variance $\sigma^{2} = np(1-p)$. It counts successes in $n$ independent trials with success probability $p$.

What four conditions must hold for a binomial experiment?

(1) Fixed number of trials $n$; (2) each trial has two outcomes (success/failure); (3) constant success probability $p$; (4) trials are independent.

State the PMF, mean, and variance of the Poisson distribution.

$P(X = k) = \frac{\lambda^{k} e^{-\lambda}}{k!}$ for $k = 0, 1, 2, \dots$, with mean and variance both equal to $\lambda$. It models counts of rare events over a fixed interval.

When can the Poisson distribution approximate the binomial distribution?

When $n$ is large and $p$ is small (rule of thumb $n \geq 20$, $p \leq 0.05$), the binomial is well approximated by a Poisson with $\lambda = np$.

Give the PDF, mean, and variance of the continuous uniform distribution on $[a, b]$.

$f(x) = \frac{1}{b - a}$ for $a \leq x \leq b$, with mean $\mu = \frac{a + b}{2}$ and variance $\sigma^{2} = \frac{(b - a)^{2}}{12}$.

State the PDF, mean, and variance of the exponential distribution with rate $\lambda$.

$f(x) = \lambda e^{-\lambda x}$ for $x \geq 0$, with mean $\frac{1}{\lambda}$ and variance $\frac{1}{\lambda^{2}}$. It models the waiting time between Poisson events.

What is the memoryless property of the exponential distribution?

$P(X > s + t \mid X > s) = P(X > t)$. The probability of waiting an additional time $t$ does not depend on how long you have already waited.

State the Central Limit Theorem.

For a sample of size $n$ from a population with mean $\mu$ and finite variance $\sigma^{2}$, the sampling distribution of the sample mean $\bar{X}$ approaches $N\!\left(\mu, \frac{\sigma^{2}}{n}\right)$ as $n$ grows, regardless of the population's shape.

According to the CLT, what sample size is typically considered 'large enough' for normality of the sample mean?

A common rule of thumb is $n \geq 30$. For strongly skewed populations a larger $n$ may be needed; for already-normal populations any $n$ works.

What is a sampling distribution?

A sampling distribution is the probability distribution of a statistic (e.g. the sample mean) computed over all possible samples of a given size $n$ from a population. It describes how the statistic varies from sample to sample.

Define the standard error of the mean and give its formula.

The standard error of the mean is the standard deviation of the sampling distribution of $\bar{X}$: $SE = \frac{\sigma}{\sqrt{n}}$ (or $\frac{s}{\sqrt{n}}$ when estimated). It quantifies the precision of $\bar{X}$ as an estimate of $\mu$.

How does increasing the sample size affect the standard error?

Standard error decreases proportionally to $\frac{1}{\sqrt{n}}$. Quadrupling the sample size halves the standard error, giving more precise estimates with diminishing returns.

What is a confidence interval and how is the interpretation of a 95% CI stated correctly?

A confidence interval is a range estimate for a parameter. A 95% CI means that if sampling were repeated many times, about 95% of the constructed intervals would contain the true parameter — not that there is a 95% probability the parameter is in this particular interval.

Write the formula for a confidence interval for a mean when $\sigma$ is known.

$\bar{x} \pm z_{\alpha/2}\,\frac{\sigma}{\sqrt{n}}$, where $z_{\alpha/2}$ is the critical value (e.g. $1.96$ for 95%). The term $z_{\alpha/2}\frac{\sigma}{\sqrt{n}}$ is the margin of error.

When should the t-distribution be used instead of the z-distribution for a confidence interval on a mean?

Use the t-distribution when the population standard deviation $\sigma$ is unknown and estimated by $s$ (especially for small $n$). The interval is $\bar{x} \pm t_{\alpha/2, \, n-1}\,\frac{s}{\sqrt{n}}$, with $n-1$ degrees of freedom.

What distinguishes a point estimate from an interval estimate?

A point estimate is a single value (e.g. $\bar{x}$) used to estimate a parameter; an interval estimate gives a range of plausible values (e.g. a confidence interval) that also conveys the uncertainty of the estimate.

What makes an estimator unbiased, and what does consistency mean?

An estimator $\hat{\theta}$ is unbiased if $E[\hat{\theta}] = \theta$. It is consistent if $\hat{\theta}$ converges in probability to $\theta$ as $n \to \infty$, so estimates improve with more data.

Contrast the null hypothesis and the alternative hypothesis, and describe p-values and significance levels.

The null hypothesis $H_0$ states no effect or no difference (the default, e.g. $\mu = \mu_0$); the alternative $H_a$ states the effect the researcher suspects. The p-value is the probability of observing data as extreme as seen if $H_0$ were true; if it is at most the significance level $\alpha$ (often $0.05$), $H_0$ is rejected.