Statistics & Probability Probability Theory Syllabus

Probability Theory syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Probability Theory in Statistics & Probability, not a summary of it.

1. 1

   Foundations of Probability

   4 topics

   • Sample Spaces and Events
   • Axioms of Probability
   • Classical, Empirical and Subjective Probability
   • Counting Principles
2. 2

   Rules of Probability

   4 topics

   • Addition Rule
   • Multiplication Rule
   • Complement Rule
   • Mutually Exclusive vs Independent Events
3. 3

   Conditional Probability

   4 topics

   • Definition and Notation
   • Independence
   • Law of Total Probability
   • Tree Diagrams
4. 4

   Bayes' Theorem

   4 topics

   • Prior and Posterior Probability
   • Deriving Bayes' Theorem
   • Applications
   • Base Rate Fallacy
5. 5

   Random Variables

   4 topics

   • Discrete vs Continuous Random Variables
   • Probability Mass and Density Functions
   • Cumulative Distribution Function
   • Expected Value and Variance
6. 6

   Joint Distributions

   4 topics

   • Joint and Marginal Distributions
   • Covariance and Correlation
   • Conditional Distributions
   • Independence of Random Variables

Probability Theory flashcards for Statistics & Probability

20 of 50 cards from the Probability Theory deck — real questions with worked answers.

1. What is a sample space in probability theory?

   The sample space, denoted $S$ (or $\Omega$), is the set of all possible outcomes of a random experiment. For example, rolling a die gives $S = \{1,2,3,4,5,6\}$.

2. Define an event in the context of a sample space.

   An event is any subset of the sample space $S$. A single outcome is an elementary (simple) event, while a set of outcomes is a compound event. An event $A$ occurs if the observed outcome lies in $A$.

3. What is the difference between a discrete and a continuous sample space?

   A discrete sample space has a finite or countably infinite number of outcomes (e.g. coin tosses), while a continuous sample space has uncountably many outcomes forming an interval or region (e.g. $S = [0,\infty)$ for waiting times).

4. State the three axioms of probability (Kolmogorov's axioms).

   For a sample space $S$ and events $A$: (1) Non-negativity: $P(A) \geq 0$. (2) Normalization: $P(S) = 1$. (3) Countable additivity: for mutually exclusive events $A_1, A_2, \dots$, $P\left(\bigcup_{i} A_i\right) = \sum_{i} P(A_i)$.

5. Using the probability axioms, what is $P(\varnothing)$, the probability of the empty set?

   $P(\varnothing) = 0$. The impossible event has probability zero, which follows directly from the axioms.

6. What are the bounds on the probability of any event $A$?

   $0 \leq P(A) \leq 1$ for every event $A$. A probability can never be negative or exceed 1.

7. Define classical (theoretical) probability and give its formula.

   Classical probability assumes all outcomes are equally likely. For an event $A$: $$P(A) = \frac{\text{number of favorable outcomes}}{\text{total number of outcomes}} = \frac{|A|}{|S|}.$$

8. Define empirical (experimental) probability and give its formula.

   Empirical probability estimates probability from observed data: $$P(A) = \frac{\text{number of times } A \text{ occurred}}{\text{total number of trials}}.$$ It approaches the true probability as the number of trials grows (Law of Large Numbers).

9. What is subjective probability?

   Subjective probability is a measure of personal belief or degree of confidence that an event will occur, based on judgment or experience rather than on equally likely outcomes or repeated experiments.

10. State the Fundamental Counting Principle (multiplication principle of counting).

    If one task can be done in $m$ ways and a second independent task in $n$ ways, then the two together can be done in $m \times n$ ways. This extends to any number of stages by multiplying the counts.

11. What is the formula for the number of permutations of $n$ distinct objects taken $r$ at a time?

    $$P(n,r) = \frac{n!}{(n-r)!}.$$ Permutations count ordered arrangements, so order matters.

12. What is the formula for the number of combinations of $n$ distinct objects taken $r$ at a time?

    $$C(n,r) = \binom{n}{r} = \frac{n!}{r!\,(n-r)!}.$$ Combinations count selections where order does not matter.

13. When counting arrangements, how do you decide between permutations and combinations?

    Use permutations when order matters (arrangements, rankings, sequences); use combinations when order does not matter (selections, committees, subsets).

14. State the general Addition Rule for two events $A$ and $B$.

    $$P(A \cup B) = P(A) + P(B) - P(A \cap B).$$ Subtracting $P(A \cap B)$ avoids double-counting the overlap.

15. How does the Addition Rule simplify for mutually exclusive events?

    If $A$ and $B$ are mutually exclusive, then $A \cap B = \varnothing$ so $P(A \cap B) = 0$, giving $$P(A \cup B) = P(A) + P(B).$$

16. State the general Multiplication Rule for the probability of $A$ and $B$ both occurring.

    $$P(A \cap B) = P(A)\,P(B \mid A) = P(B)\,P(A \mid B).$$

17. How does the Multiplication Rule simplify for independent events?

    If $A$ and $B$ are independent, then $P(B \mid A) = P(B)$, so $$P(A \cap B) = P(A)\,P(B).$$

18. State the Complement Rule.

    For any event $A$ with complement $A^{c}$ (or $A'$): $$P(A^{c}) = 1 - P(A).$$ The event and its complement together cover the whole sample space.

19. How can the Complement Rule simplify computing 'at least one' probabilities?

    $P(\text{at least one}) = 1 - P(\text{none})$. It is often easier to find the probability that the event never occurs and subtract from 1.

20. Define mutually exclusive (disjoint) events.

    Two events are mutually exclusive if they cannot occur at the same time, i.e. $A \cap B = \varnothing$ and $P(A \cap B) = 0$.

See more Probability Theory flashcards →

Planning Probability Theory for Statistics & Probability

Probability Theory (Statistics & Probability) FAQ