GATE Statistics Non-parametric Statistics Syllabus

Non-parametric Statistics syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Non-parametric Statistics in GATE Statistics, not a summary of it.

1. 1

   Empirical distribution function and its properties

   2 topics

   • Empirical distribution function
   • Properties of empirical distribution function
2. 2

   Goodness of Fit Tests

   2 topics

   • Chi-square test
   • Kolmogorov-Smirnov test
3. 3

   Sign Test

   1 topic

   • Sign test
4. 4

   Wilcoxon Signed Rank Test

   1 topic

   • Wilcoxon signed rank test
5. 5

   Mann-Whitney U-Test

   1 topic

   • Mann-Whitney U-test
6. 6

   Rank Correlation Coefficients

   2 topics

   • Spearman
   • Kendall

Non-parametric Statistics flashcards for GATE Statistics

25 of 50 cards from the Non-parametric Statistics deck — real questions with worked answers.

1. What is the empirical distribution function (EDF) of a sample $X_1,\dots,X_n$?

   It is the cumulative distribution function of the empirical measure that places mass $\frac{1}{n}$ on each observation: $$F_n(x)=\frac{1}{n}\sum_{i=1}^{n}\mathbf{1}\{X_i\leq x\}.$$ It counts the proportion of sample values $\leq x$.

2. For a fixed $x$, what is the distribution of $n F_n(x)$?

   $n F_n(x)$ follows a Binomial distribution $\text{Bin}(n,F(x))$, since it counts how many of the $n$ i.i.d. observations fall $\leq x$, each with success probability $F(x)$.

3. What are the mean and variance of the empirical distribution function $F_n(x)$ at a fixed point $x$?

   $$E[F_n(x)]=F(x),\qquad \operatorname{Var}[F_n(x)]=\frac{F(x)\big(1-F(x)\big)}{n}.$$ Thus $F_n(x)$ is an unbiased and consistent estimator of $F(x)$.

4. State the Glivenko-Cantelli theorem.

   It states that the EDF converges uniformly almost surely to the true CDF: $$\sup_{x}\big|F_n(x)-F(x)\big|\xrightarrow{a.s.}0\quad\text{as } n\to\infty.$$ It is sometimes called the fundamental theorem of statistics.

5. What is the limiting (asymptotic) distribution of $\sqrt{n}\,\big(F_n(x)-F(x)\big)$ at a fixed $x$?

   By the CLT it is asymptotically normal: $$\sqrt{n}\big(F_n(x)-F(x)\big)\xrightarrow{d}N\!\Big(0,\;F(x)\big(1-F(x)\big)\Big).$$

6. List the key analytic properties of the empirical distribution function $F_n(x)$.

   It is (1) a non-decreasing step function, (2) right-continuous, (3) bounded with $F_n(-\infty)=0$ and $F_n(+\infty)=1$, and (4) it jumps by $\frac{1}{n}$ at each distinct observation (or by $\frac{k}{n}$ at a value of multiplicity $k$).

7. What is the covariance of the EDF at two points, $\operatorname{Cov}\big(F_n(s),F_n(t)\big)$ for $s\leq t$?

   $$\operatorname{Cov}\big(F_n(s),F_n(t)\big)=\frac{F(s)\big(1-F(t)\big)}{n},\qquad s\leq t.$$

8. What general class of hypotheses do non-parametric tests address, and what is their main advantage?

   They test hypotheses without assuming a specific parametric form (e.g. normality) of the underlying distribution. Their main advantage is robustness: validity under weak assumptions (often only continuity), at the cost of some efficiency when parametric assumptions actually hold.

9. What is the test statistic for the Pearson chi-square goodness-of-fit test?

   $$\chi^{2}=\sum_{i=1}^{k}\frac{(O_i-E_i)^{2}}{E_i},$$ where $O_i$ are observed and $E_i$ are expected frequencies in the $k$ categories. Large values indicate poor fit.

10. What are the degrees of freedom for a chi-square goodness-of-fit test with $k$ categories when $m$ parameters are estimated from the data?

    $$df=k-1-m.$$ The $-1$ comes from the constraint that frequencies sum to $n$, and $-m$ from each estimated parameter.

11. What is the degrees of freedom for a chi-square test of independence in an $r\times c$ contingency table?

    $$df=(r-1)(c-1).$$

12. In a chi-square test of independence, how is the expected frequency of cell $(i,j)$ computed?

    $$E_{ij}=\frac{(\text{row } i\text{ total})\times(\text{column } j\text{ total})}{n},$$ assuming independence of the row and column classifications.

13. What is the common rule-of-thumb condition on expected frequencies for the chi-square approximation to be valid?

    Each expected frequency should be at least $5$ (i.e. $E_i\geq 5$). When some cells are too small, categories are pooled, or an exact test is used.

14. What is the null hypothesis of the Kolmogorov-Smirnov one-sample test?

    That the sample comes from a specified continuous distribution $F_0$: $$H_0:\;F(x)=F_0(x)\ \text{for all } x.$$

15. State the Kolmogorov-Smirnov one-sample test statistic.

    It is the maximum absolute deviation between the empirical and hypothesized CDFs: $$D_n=\sup_{x}\big|F_n(x)-F_0(x)\big|.$$ $H_0$ is rejected for large $D_n$.

16. For the one-sample KS test, how is $D_n$ computed in practice from ordered data $x_{(1)}\leq\cdots\leq x_{(n)}$?

    Using $D^{+}=\max_i\big(\tfrac{i}{n}-F_0(x_{(i)})\big)$ and $D^{-}=\max_i\big(F_0(x_{(i)})-\tfrac{i-1}{n}\big)$, then $$D_n=\max(D^{+},D^{-}).$$

17. What is the two-sample Kolmogorov-Smirnov test statistic?

    For samples with EDFs $F_m$ and $G_n$, $$D_{m,n}=\sup_{x}\big|F_m(x)-G_n(x)\big|,$$ testing $H_0$ that both samples come from the same continuous distribution.

18. What is a key advantage of the Kolmogorov-Smirnov statistic regarding the underlying distribution?

    Under $H_0$ (with $F_0$ continuous and fully specified), the distribution of $D_n$ is distribution-free: it does not depend on the particular $F_0$. This lets one tabulate universal critical values.

19. Compare the chi-square goodness-of-fit test and the Kolmogorov-Smirnov test in terms of data type.

    Chi-square works on categorized/binned frequency data and suits discrete or grouped data; KS uses the full continuous EDF without binning, is generally more powerful for continuous distributions, but in its classic form requires a fully specified continuous $F_0$ (no estimated parameters).

20. What hypothesis does the one-sample sign test address, and on what quantity?

    It tests a hypothesized value of the population median $M$ (or the median of differences for paired data): $$H_0:\;M=M_0.$$ It uses only the signs of $X_i-M_0$, ignoring magnitudes.

21. What is the test statistic and its null distribution for the sign test?

    Let $S^{+}$ be the number of positive differences among $n$ non-zero differences. Under $H_0$, $$S^{+}\sim\text{Bin}\!\left(n,\tfrac{1}{2}\right).$$ Ties (zero differences) are discarded and $n$ reduced accordingly.

22. What is the normal approximation used for the sign test for large $n$?

    $$Z=\frac{S^{+}-\tfrac{n}{2}}{\sqrt{n/4}}=\frac{2S^{+}-n}{\sqrt{n}}\;\xrightarrow{d}\;N(0,1),$$ often with a continuity correction of $\pm 0.5$.

23. What assumption does the Wilcoxon signed-rank test require that the sign test does not?

    The Wilcoxon signed-rank test assumes the distribution of the differences is symmetric about its median; it then uses the ranks of the absolute differences, incorporating magnitude information that the sign test ignores.

24. Describe the procedure for computing the Wilcoxon signed-rank statistic.

    (1) Compute differences $d_i$ from the hypothesized median, discard zeros. (2) Rank the $|d_i|$ from smallest to largest. (3) Reattach the original signs. (4) Sum the positive ranks to get $W^{+}$ (and negative ranks $W^{-}$). The test statistic is $W=\min(W^{+},W^{-})$ or $W^{+}$ itself.

25. What are the mean and variance of the Wilcoxon signed-rank statistic $W^{+}$ under $H_0$ (no ties)?

    $$E[W^{+}]=\frac{n(n+1)}{4},\qquad \operatorname{Var}[W^{+}]=\frac{n(n+1)(2n+1)}{24}.$$

See more Non-parametric Statistics flashcards →

Planning Non-parametric Statistics for GATE Statistics

Non-parametric Statistics (GATE Statistics) FAQ