GATE Statistics Estimation Flashcards

24 sample cards from the Estimation deck

Sampled from the end of the deck, so these are different cards from the ones shown on the syllabus page.

1. For i.i.d. $\text{Bernoulli}(p)$, identify the complete sufficient statistic and justify via exponential family.

   $T=\sum_{i=1}^n X_i$. The Bernoulli pmf is exponential family with natural parameter $\eta=\log\frac{p}{1-p}$, ranging over $\mathbb{R}$ (an open set), so $T$ is complete sufficient for $p$.

2. For i.i.d. $\text{Poisson}(\lambda)$, give the complete sufficient statistic.

   $T=\sum_{i=1}^n X_i$, which is complete sufficient (Poisson is full-rank one-parameter exponential family, $\eta=\log\lambda\in\mathbb{R}$).

3. For i.i.d. $N(\mu,\sigma^2)$ (both unknown), state the complete sufficient statistic.

   $T=\left(\sum_i X_i,\ \sum_i X_i^2\right)$, equivalently $(\bar X, S^2)$. The natural parameter space is a 2-D open set, so $T$ is complete (and minimal) sufficient.

4. Define an unbiased estimator of $g(\theta)$.

   An estimator $W(X)$ is unbiased for $g(\theta)$ if $E_\theta[W(X)] = g(\theta)$ for all $\theta \in \Theta$. Its bias $E_\theta[W]-g(\theta)$ is zero everywhere.

5. Define the UMVUE (Uniformly Minimum Variance Unbiased Estimator).

   $W^*$ is the UMVUE of $g(\theta)$ if it is unbiased and, for every other unbiased estimator $W$ of $g(\theta)$, $$\operatorname{Var}_\theta(W^*) \leq \operatorname{Var}_\theta(W) \quad \text{for all } \theta \in \Theta.$$

6. State the Rao–Blackwell Theorem.

   Let $W$ be an unbiased estimator of $g(\theta)$ and $T$ a sufficient statistic. Define $\phi(T)=E[W\mid T]$. Then $\phi(T)$ is unbiased for $g(\theta)$ and $$\operatorname{Var}_\theta(\phi(T)) \leq \operatorname{Var}_\theta(W)\ \text{for all }\theta,$$ with strict inequality unless $W$ is already a function of $T$.

7. Why must one condition on a sufficient statistic in the Rao–Blackwell process?

   Conditioning on a sufficient statistic ensures $\phi(T)=E[W\mid T]$ does not depend on $\theta$ (so it is a valid statistic/estimator). The sufficiency guarantees the conditional expectation is computable and parameter-free.

8. State the Lehmann–Scheffé Theorem.

   If $T$ is a complete sufficient statistic for $\theta$ and $\phi(T)$ is an unbiased estimator of $g(\theta)$, then $\phi(T)$ is the unique UMVUE of $g(\theta)$ (unique a.s.).

9. Describe the two-step Lehmann–Scheffé method (Rao–Blackwellization route) to find a UMVUE.

   1) Find any unbiased estimator $W$ of $g(\theta)$. 2) Compute $\phi(T)=E[W\mid T]$, where $T$ is complete sufficient. Then $\phi(T)$ is the UMVUE. Completeness guarantees uniqueness.

10. Describe the direct (functional) method to find a UMVUE.

    Find a function $\phi$ of the complete sufficient statistic $T$ such that $E_\theta[\phi(T)] = g(\theta)$ for all $\theta$. By Lehmann–Scheffé, that $\phi(T)$ is the UMVUE. Solving $E[\phi(T)]=g(\theta)$ is often an integral/series equation.

11. State the characterization of UMVUE via uncorrelatedness with unbiased estimators of zero.

    An unbiased estimator $W$ of $g(\theta)$ is the UMVUE if and only if $\operatorname{Cov}_\theta(W,U)=0$ for all $\theta$, for every unbiased estimator $U$ of $0$ (i.e., $E_\theta[U]=0\ \forall\theta$).

12. Why does completeness guarantee uniqueness of the UMVUE in Lehmann–Scheffé?

    If $\phi_1(T)$ and $\phi_2(T)$ are both unbiased for $g(\theta)$, then $E_\theta[\phi_1-\phi_2]=0$ for all $\theta$. Completeness forces $\phi_1=\phi_2$ a.s., so the UMVUE is unique.

13. Find the UMVUE of $\mu$ for i.i.d. $N(\mu,\sigma^2)$.

    $\bar X = \frac{1}{n}\sum_i X_i$. It is unbiased and a function of the complete sufficient statistic, hence the UMVUE of $\mu$.

14. Find the UMVUE of $\sigma^2$ for i.i.d. $N(\mu,\sigma^2)$ with $\mu$ unknown.

    $S^2 = \frac{1}{n-1}\sum_{i=1}^n (X_i-\bar X)^2$. It is unbiased for $\sigma^2$ and a function of the complete sufficient statistic, hence the UMVUE.

15. Find the UMVUE of $\theta$ for i.i.d. $\text{Uniform}(0,\theta)$.

    $\dfrac{n+1}{n}X_{(n)}$. Since $E_\theta[X_{(n)}]=\frac{n}{n+1}\theta$ and $X_{(n)}$ is complete sufficient, $\frac{n+1}{n}X_{(n)}$ is the unbiased UMVUE.

16. Find the UMVUE of $p$ for i.i.d. $\text{Bernoulli}(p)$.

    $\bar X = \frac{1}{n}\sum_i X_i$. It is unbiased for $p$ and a function of the complete sufficient statistic $\sum X_i$, hence the UMVUE.

17. Find the UMVUE of $\lambda$ for i.i.d. $\text{Poisson}(\lambda)$.

    $\bar X = \frac{1}{n}\sum_i X_i$, which is unbiased and a function of the complete sufficient statistic $\sum X_i$, hence the UMVUE of $\lambda$.

18. For i.i.d. $\text{Poisson}(\lambda)$, find the UMVUE of $e^{-\lambda}=P(X=0)$.

    $\left(\dfrac{n-1}{n}\right)^{T}$ where $T=\sum_{i=1}^n X_i$. This equals $E[\mathbf{1}(X_1=0)\mid T]$, a function of the complete sufficient statistic, hence the UMVUE of $e^{-\lambda}$.

19. State the Cramér–Rao Lower Bound (CRLB) for an unbiased estimator of $g(\theta)$.

    Under regularity conditions, for any unbiased estimator $W$ of $g(\theta)$, $$\operatorname{Var}_\theta(W) \geq \frac{[g'(\theta)]^{2}}{I_n(\theta)},$$ where $I_n(\theta)$ is the Fisher information of the sample.

20. Give the formula for Fisher information $I(\theta)$ of a single observation.

    $$I(\theta) = E_\theta\!\left[\left(\frac{\partial}{\partial\theta}\log f(X;\theta)\right)^{2}\right] = -E_\theta\!\left[\frac{\partial^{2}}{\partial\theta^{2}}\log f(X;\theta)\right].$$ For an i.i.d. sample of size $n$, $I_n(\theta)=n\,I(\theta)$.

21. What is the difference between an efficient estimator and the UMVUE?

    An efficient estimator attains the CRLB (variance exactly $=$ bound). Every efficient unbiased estimator is the UMVUE, but a UMVUE need not attain the CRLB — the UMVUE can have minimum variance while still exceeding the CRLB.

22. When does an unbiased estimator attain the CRLB exactly?

    Equality in the CRLB holds iff the score factors as $$\frac{\partial}{\partial\theta}\log f(x;\theta) = a(\theta)\,[W(x)-g(\theta)],$$ i.e., the model is exponential-family and $W$ is the natural (linear) estimator. Then $W$ is efficient.

23. Compare Rao–Blackwell and Lehmann–Scheffé: what does each guarantee?

    Rao–Blackwell (sufficiency only): conditioning on a sufficient statistic gives an unbiased estimator with variance no larger — improvement, but not necessarily best. Lehmann–Scheffé (sufficiency + completeness): yields the unique UMVUE. Completeness is the extra ingredient that upgrades 'improved' to 'best and unique'.

24. Summarize the logical chain linking sufficiency, completeness, and UMVUE.

    Sufficiency $\to$ data reduction (Rao–Blackwell improves estimators). Minimal sufficiency $\to$ maximal reduction. Completeness $+$ sufficiency $\to$ unique UMVUE (Lehmann–Scheffé). Full-rank exponential family $\to$ natural statistic is complete sufficient $\to$ UMVUEs are functions of it.

What this deck covers

Estimation flashcards FAQ