Statistics & Probability Statistical Reasoning & Communication Syllabus

Statistical Reasoning & Communication syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Statistical Reasoning & Communication in Statistics & Probability, not a summary of it.

1. 1

   Interpreting Statistical Results

   4 topics

   • Statistical vs Practical Significance
   • Effect Size
   • Confidence Interval Interpretation
   • Misinterpreting P-Values
2. 2

   Common Pitfalls and Fallacies

   5 topics

   • Simpson's Paradox
   • P-Hacking and Multiple Comparisons
   • Regression to the Mean
   • Ecological Fallacy
   • Correlation-Causation Confusion
3. 3

   Causal Inference

   4 topics

   • Potential Outcomes Framework
   • Confounding and Controls
   • Randomized Controlled Trials
   • Observational Causal Methods
4. 4

   Communicating Statistics

   4 topics

   • Reporting Standards and Transparency
   • Effective Data Visualization
   • Avoiding Misleading Graphics
   • Writing Up Statistical Findings

Statistical Reasoning & Communication flashcards for Statistics & Probability

18 of 52 cards from the Statistical Reasoning & Communication deck — real questions with worked answers.

1. What is the difference between statistical significance and practical significance?

   Statistical significance means an observed effect is unlikely to be due to chance (e.g., $p < \alpha$); practical significance means the effect is large enough to matter in the real world. A result can be statistically significant yet practically trivial, especially with very large samples.

2. Why can a very large sample produce statistically significant but practically meaningless results?

   Because the standard error shrinks as $n$ grows ($SE = \frac{\sigma}{\sqrt{n}}$), even tiny effects yield large test statistics and small $p$-values. Significance reflects detectability, not importance, so a negligible effect size can still be 'significant.'

3. Define effect size and name two common measures.

   Effect size quantifies the magnitude of a phenomenon independent of sample size. Common measures: Cohen's $d$ (standardized mean difference) and Pearson's $r$ (correlation); others include odds ratios and $\eta^{2}$.

4. State the formula for Cohen's $d$ and its conventional benchmarks.

   $$d = \frac{\bar{x}_1 - \bar{x}_2}{s_{pooled}}$$ Conventional benchmarks: $d \approx 0.2$ small, $0.5$ medium, $0.8$ large.

5. How is the pooled standard deviation computed for Cohen's $d$?

   $$s_{pooled} = \sqrt{\frac{(n_1-1)s_1^{2} + (n_2-1)s_2^{2}}{n_1 + n_2 - 2}}$$

6. What does $r^{2}$ (coefficient of determination) represent as an effect size?

   $r^{2}$ is the proportion of variance in the outcome variable that is explained by the predictor. For example, $r = 0.3$ gives $r^{2} = 0.09$, meaning $9\%$ of variance is explained.

7. Give the correct interpretation of a $95\%$ confidence interval.

   If the sampling procedure were repeated many times, about $95\%$ of the constructed intervals would contain the true parameter. It is a statement about the long-run reliability of the method, not the probability that this particular interval contains the parameter.

8. Why is it incorrect to say 'there is a $95\%$ probability the true mean lies in this specific confidence interval'?

   Under the frequentist framework the true parameter is fixed, not random; a computed interval either contains it or does not. The $95\%$ refers to the procedure's long-run coverage across many samples, not to a single realized interval.

9. Write the general form of a confidence interval for a population mean (known $\sigma$).

   $$\bar{x} \pm z_{\alpha/2}\,\frac{\sigma}{\sqrt{n}}$$ where $z_{\alpha/2}$ is the critical value (e.g., $1.96$ for $95\%$).

10. How does confidence interval width relate to sample size and confidence level?

    Width increases with higher confidence level (larger critical value) and decreases as sample size grows (since $SE \propto \frac{1}{\sqrt{n}}$). Quadrupling $n$ roughly halves the width.

11. What is the correct definition of a p-value?

    The p-value is the probability of obtaining a test statistic at least as extreme as the one observed, assuming the null hypothesis is true: $P(\text{data or more extreme} \mid H_0)$.

12. List three common misinterpretations of a p-value.

    (1) That it is the probability the null hypothesis is true, $P(H_0\mid\text{data})$; (2) that $1-p$ is the probability the alternative is true; (3) that a small $p$ measures the size or importance of an effect.

13. Does a non-significant result ($p > \alpha$) prove the null hypothesis is true?

    No. Failing to reject $H_0$ means there is insufficient evidence against it; absence of evidence is not evidence of absence. The result may reflect low power or small sample size rather than a true null.

14. Why does $p = 0.049$ versus $p = 0.051$ not represent a meaningful qualitative difference?

    The $\alpha = 0.05$ threshold is an arbitrary convention. The two p-values reflect nearly identical evidence; treating one as 'significant' and the other as 'not' creates a false dichotomy (the cliff effect).

15. What is Simpson's Paradox?

    A phenomenon where a trend or association that appears in each of several groups reverses or disappears when the groups are combined (aggregated). It arises from a lurking/confounding variable and unequal group sizes.

16. What causes Simpson's Paradox and how is it resolved?

    It is caused by a confounding variable correlated with both the grouping and the outcome, combined with disproportionate subgroup sizes. It is resolved by identifying the correct variable to condition on (often analyzing within subgroups rather than the aggregate).

17. Give the classic real example illustrating Simpson's Paradox.

    The 1973 UC Berkeley admissions data: overall men were admitted at a higher rate than women, but within most individual departments women had equal or higher admission rates. Women applied more to competitive, low-acceptance departments.

18. What is p-hacking?

    P-hacking is the manipulation of data analysis—trying many tests, variables, subgroups, or stopping rules—until a statistically significant ($p < 0.05$) result appears, then reporting only that result. It inflates the false-positive rate.

See more Statistical Reasoning & Communication flashcards →

Planning Statistical Reasoning & Communication for Statistics & Probability

Statistical Reasoning & Communication (Statistics & Probability) FAQ