GATE Statistics Non-parametric Statistics Syllabus

What is the test statistic and its null distribution for the sign test?

Let $S^{+}$ be the number of positive differences among $n$ non-zero differences. Under $H_0$, $$S^{+}\sim\text{Bin}\!\left(n,\tfrac{1}{2}\right).$$ Ties (zero differences) are discarded and $n$ reduced accordingly.

What is the normal approximation used for the sign test for large $n$?

$$Z=\frac{S^{+}-\tfrac{n}{2}}{\sqrt{n/4}}=\frac{2S^{+}-n}{\sqrt{n}}\;\xrightarrow{d}\;N(0,1),$$ often with a continuity correction of $\pm 0.5$.

What assumption does the Wilcoxon signed-rank test require that the sign test does not?

The Wilcoxon signed-rank test assumes the distribution of the differences is symmetric about its median; it then uses the ranks of the absolute differences, incorporating magnitude information that the sign test ignores.

Describe the procedure for computing the Wilcoxon signed-rank statistic.

(1) Compute differences $d_i$ from the hypothesized median, discard zeros. (2) Rank the $|d_i|$ from smallest to largest. (3) Reattach the original signs. (4) Sum the positive ranks to get $W^{+}$ (and negative ranks $W^{-}$). The test statistic is $W=\min(W^{+},W^{-})$ or $W^{+}$ itself.

What are the mean and variance of the Wilcoxon signed-rank statistic $W^{+}$ under $H_0$ (no ties)?

$$E[W^{+}]=\frac{n(n+1)}{4},\qquad \operatorname{Var}[W^{+}]=\frac{n(n+1)(2n+1)}{24}.$$

State the large-sample normal approximation for the Wilcoxon signed-rank test.

$$Z=\frac{W^{+}-\dfrac{n(n+1)}{4}}{\sqrt{\dfrac{n(n+1)(2n+1)}{24}}}\;\xrightarrow{d}\;N(0,1).$$

What is the null hypothesis of the Mann-Whitney U-test?

That two independent samples come from the same distribution (equivalently $P(X<Y)=\tfrac{1}{2}$, often stated as equal medians/locations under a shift model): $$H_0:\;F=G.$$

Give the formula for the Mann-Whitney U statistic in terms of the rank sum.

With sample sizes $n_1,n_2$ and $R_1$ the sum of ranks of sample 1 in the combined ranking, $$U_1=R_1-\frac{n_1(n_1+1)}{2},\qquad U_2=n_1n_2-U_1.$$ The test statistic is $U=\min(U_1,U_2)$.

What identity links the two Mann-Whitney statistics $U_1$ and $U_2$?

$$U_1+U_2=n_1 n_2.$$

State the mean and variance of the Mann-Whitney $U$ statistic under $H_0$ (no ties).

$$E[U]=\frac{n_1 n_2}{2},\qquad \operatorname{Var}[U]=\frac{n_1 n_2(n_1+n_2+1)}{12}.$$

What is the relationship between the Mann-Whitney U-test and the Wilcoxon rank-sum test?

They are equivalent tests: the Wilcoxon rank-sum statistic (sum of ranks of one sample) and the Mann-Whitney $U$ differ only by the constant $\frac{n_1(n_1+1)}{2}$, so they yield identical p-values.

What is the parametric counterpart of the Mann-Whitney U-test, and when is U preferred?

Its parametric counterpart is the two-sample (unpaired) $t$-test. The U-test is preferred when normality fails, for ordinal data, or with outliers, since it only uses ranks.

What is the parametric counterpart of the Wilcoxon signed-rank test?

The paired-sample (one-sample) $t$-test. The signed-rank test replaces it when the differences are non-normal but symmetric.

What is Spearman's rank correlation coefficient measuring?

It measures the strength and direction of a monotonic association between two variables, computed as the Pearson correlation applied to the ranks of the data.

Give the shortcut formula for Spearman's rank correlation coefficient $\rho_s$ when there are no tied ranks.

$$\rho_s=1-\frac{6\sum_{i=1}^{n} d_i^{2}}{n(n^{2}-1)},$$ where $d_i$ is the difference between the ranks of the $i$-th pair.

What is the range of Spearman's $\rho_s$, and what do the extreme values mean?

$-1\leq\rho_s\leq 1$. $\rho_s=+1$ indicates a perfect increasing monotonic relationship, $\rho_s=-1$ a perfect decreasing one, and $\rho_s=0$ no monotonic association.

How can the significance of Spearman's $\rho_s$ be tested for large $n$?

Using the approximate $t$ statistic with $n-2$ degrees of freedom: $$t=\rho_s\sqrt{\frac{n-2}{1-\rho_s^{2}}}.$$ Alternatively $Z=\rho_s\sqrt{n-1}\sim N(0,1)$ for large $n$.

What is Kendall's tau ($\tau$) based on?

It is based on counting concordant and discordant pairs of observations. A pair $(i,j)$ is concordant if the orderings of both variables agree, and discordant if they disagree.

Give the formula for Kendall's $\tau$ (tau-a, no ties).

$$\tau=\frac{C-D}{\binom{n}{2}}=\frac{C-D}{\tfrac{1}{2}n(n-1)},$$ where $C$ is the number of concordant pairs and $D$ the number of discordant pairs.

What are the mean and variance of Kendall's $\tau$ under the null hypothesis of independence (no ties)?

$$E[\tau]=0,\qquad \operatorname{Var}[\tau]=\frac{2(2n+5)}{9n(n-1)}.$$ The standardized statistic $Z=\tau/\sqrt{\operatorname{Var}[\tau]}$ is used for large-sample testing.

Compare Spearman's $\rho_s$ and Kendall's $\tau$ as measures of association.

Both are rank-based measures of monotonic association in $[-1,1]$. Kendall's $\tau$ has a direct probabilistic interpretation (difference of concordance/discordance probabilities), is more robust to outliers, and converges faster to normality; Spearman's $\rho_s$ is easier to compute and typically larger in magnitude. Generally $|\tau|\leq|\rho_s|$.

What is Kendall's tau-b designed to handle, compared with tau-a?

Tau-b adjusts for tied ranks by including tie corrections in the denominator: $$\tau_b=\frac{C-D}{\sqrt{(n_0-n_1)(n_0-n_2)}},$$ where $n_0=\tfrac{1}{2}n(n-1)$ and $n_1,n_2$ are tie terms for the two variables. Tau-a assumes no ties.

How are tied observations typically handled when assigning ranks in non-parametric tests?

Tied observations are assigned the average (midrank) of the ranks they would otherwise occupy. This requires variance corrections in tests such as Wilcoxon, Mann-Whitney, and Spearman.

What does it mean for a test statistic to be 'distribution-free'?

Its sampling distribution under $H_0$ does not depend on the specific (continuous) underlying distribution $F$ of the data. This is why rank-based statistics have universal tabulated critical values.

What is the asymptotic relative efficiency (ARE) of the sign test relative to the $t$-test under normality?

$\frac{2}{\pi}\approx 0.637$. The sign test is comparatively inefficient under normality but can be more efficient for heavy-tailed distributions.

What is the asymptotic relative efficiency of the Wilcoxon signed-rank and Mann-Whitney tests relative to the $t$-test under normality?

$\frac{3}{\pi}\approx 0.955$. They lose only about $5\%$ efficiency under normality but can be far more efficient for non-normal, heavy-tailed distributions (ARE never below $0.864$).

For the one-sample KS test, what is the limiting (Kolmogorov) distribution of $\sqrt{n}\,D_n$ under $H_0$?

$$P\big(\sqrt{n}\,D_n\leq t\big)\to K(t)=1-2\sum_{k=1}^{\infty}(-1)^{k-1}e^{-2k^{2}t^{2}},\quad t>0,$$ the Kolmogorov distribution.

In the sign test for paired data, how are observations exactly equal to the hypothesized median (ties/zeros) treated?

They are discarded (excluded), and the effective sample size $n$ is reduced to the number of non-zero differences before applying the binomial distribution.

Which non-parametric test is the rank-based analogue of the one-way ANOVA, extending Mann-Whitney to $k>2$ groups?

The Kruskal-Wallis $H$ test. It generalizes the Mann-Whitney U-test to compare more than two independent samples using combined-sample ranks.

Summarize which non-parametric test to use for: (a) one sample location, (b) two paired samples, (c) two independent samples.

(a) Sign test or Wilcoxon signed-rank test (the latter if differences are symmetric). (b) Wilcoxon signed-rank test on the differences (or sign test). (c) Mann-Whitney U-test (equivalently Wilcoxon rank-sum test).