Statistics & Probability Probability Theory Flashcards

24 sample cards from the Probability Theory deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. What does it mean for three events $A$, $B$, $C$ to be mutually independent?

   They must satisfy pairwise independence AND $P(A \cap B \cap C) = P(A)P(B)P(C)$. Pairwise independence alone is not sufficient for mutual independence.

2. State the Law of Total Probability for a partition $B_1, \dots, B_n$ of the sample space.

   If $B_1,\dots,B_n$ are mutually exclusive and exhaustive with $P(B_i)>0$, then for any event $A$: $$P(A) = \sum_{i=1}^{n} P(A \mid B_i)\,P(B_i).$$

3. State the two-event form of the Law of Total Probability.

   For an event $B$ and its complement $B^{c}$: $$P(A) = P(A \mid B)P(B) + P(A \mid B^{c})P(B^{c}).$$

4. What must a set of events satisfy to form a valid partition of the sample space?

   They must be mutually exclusive (no overlap: $B_i \cap B_j = \varnothing$ for $i \neq j$) and collectively exhaustive (their union is $S$, so $\bigcup_i B_i = S$).

5. What is a tree diagram used for in probability?

   A tree diagram represents a sequence of events with branches; each branch is labeled with a (conditional) probability. Multiplying probabilities along a path gives the probability of that combined outcome, and paths for the same event are added.

6. In a probability tree, how do you find the probability of a specific final outcome?

   Multiply the probabilities along the branches from the root to that outcome. Each branch after the first represents a conditional probability given the previous branches.

7. In a probability tree, what must the probabilities on branches emanating from a single node sum to?

   They must sum to 1, since the branches from a node represent all mutually exclusive possibilities at that stage.

8. Define prior probability.

   A prior probability, $P(H)$, is the probability assigned to a hypothesis or event before observing new evidence — it reflects initial belief based on existing information.

9. Define posterior probability.

   A posterior probability, $P(H \mid E)$, is the updated probability of a hypothesis $H$ after taking into account new evidence $E$. It is computed via Bayes' theorem.

10. State Bayes' Theorem for events $A$ and $B$.

    $$P(A \mid B) = \frac{P(B \mid A)\,P(A)}{P(B)}, \quad P(B) > 0.$$

11. Derive Bayes' Theorem from the definition of conditional probability.

    By definition $P(A\mid B) = \frac{P(A\cap B)}{P(B)}$ and $P(B\mid A) = \frac{P(A\cap B)}{P(A)}$. From the second, $P(A\cap B) = P(B\mid A)P(A)$. Substituting into the first gives $$P(A\mid B) = \frac{P(B\mid A)P(A)}{P(B)}.$$

12. State the expanded form of Bayes' Theorem using the Law of Total Probability in the denominator.

    For a partition $A_1,\dots,A_n$: $$P(A_k \mid B) = \frac{P(B \mid A_k)P(A_k)}{\sum_{i=1}^{n} P(B \mid A_i)P(A_i)}.$$

13. In Bayes' Theorem, identify the prior, likelihood, and posterior in $P(H\mid E) = \frac{P(E\mid H)P(H)}{P(E)}$.

    $P(H)$ is the prior, $P(E\mid H)$ is the likelihood (probability of the evidence given the hypothesis), $P(E)$ is the marginal/evidence, and $P(H\mid E)$ is the posterior.

14. Give a classic application of Bayes' Theorem in medical testing.

    Given a disease with prevalence $P(D)$ and a test with sensitivity $P(+\mid D)$ and false-positive rate $P(+\mid D^{c})$, Bayes' theorem finds the probability a person actually has the disease given a positive test: $$P(D\mid +) = \frac{P(+\mid D)P(D)}{P(+\mid D)P(D) + P(+\mid D^{c})P(D^{c})}.$$

15. What is the base rate fallacy?

    The base rate fallacy is the error of ignoring the prior probability (base rate) of an event and over-relying on specific test/evidence information, leading to overestimation of the posterior probability — especially for rare events.

16. Why can a highly accurate test still yield a low posterior probability for a rare disease (base rate fallacy)?

    When the disease is rare, the large healthy population produces many false positives that can outnumber true positives. Even with high sensitivity, $P(D\mid +)$ stays low because the small prior $P(D)$ dominates the calculation.

17. Define a random variable.

    A random variable is a function that assigns a real number to each outcome in the sample space of a random experiment, typically denoted by a capital letter such as $X$.

18. Distinguish a discrete random variable from a continuous random variable.

    A discrete random variable takes a finite or countably infinite set of distinct values (e.g. counts). A continuous random variable can take any value in an interval of real numbers (e.g. measurements like height or time).

19. Define the probability mass function (PMF) of a discrete random variable.

    The PMF is $p(x) = P(X = x)$, giving the probability that the discrete random variable $X$ equals each value $x$. It must satisfy $p(x) \geq 0$ and $\sum_{x} p(x) = 1$.

20. Define the probability density function (PDF) of a continuous random variable.

    A PDF $f(x)$ satisfies $f(x) \geq 0$ and $\int_{-\infty}^{\infty} f(x)\,dx = 1$. Probabilities are areas: $$P(a \leq X \leq b) = \int_{a}^{b} f(x)\,dx.$$

21. For a continuous random variable, what is $P(X = c)$ for any single value $c$?

    $P(X = c) = 0$. For a continuous variable the probability of any exact single value is zero because it corresponds to zero area under the density curve; only intervals carry positive probability.

22. Compare how probabilities are obtained from a PMF versus a PDF.

    For a discrete variable, probability comes directly from the PMF value $P(X=x)=p(x)$ and is summed over values. For a continuous variable, the PDF value $f(x)$ is not a probability; probability is found by integrating $f(x)$ over an interval.

23. What is the normalization (total probability) condition for a PMF and for a PDF?

    PMF: $\sum_{x} p(x) = 1$. PDF: $\int_{-\infty}^{\infty} f(x)\,dx = 1$. In both cases the total probability over all outcomes equals 1.

24. Define the cumulative distribution function (CDF) of a random variable $X$.

    The CDF is $F(x) = P(X \leq x)$. For a discrete variable $F(x) = \sum_{t \leq x} p(t)$; for a continuous variable $F(x) = \int_{-\infty}^{x} f(t)\,dt$, and $f(x) = F'(x)$ where the derivative exists.

What this deck covers

Probability Theory flashcards FAQ