GATE Statistics Multivariate Analysis Syllabus

Multivariate Analysis syllabus — full chapter and topic list

Expand any chapter to see its topics and sub-topics. This is the whole examinable outline for Multivariate Analysis in GATE Statistics, not a summary of it.

1. 1

   Multivariate normal distribution

   2 topics

   • Properties
   • Conditional and marginal distributions
2. 2

   Maximum likelihood estimation of mean vector and dispersion matrix

   1 topic

   • Properties
3. 3

   Hotelling’s T2 test

   1 topic

   • Properties
4. 4

   Wishart distribution and its basic properties

   1 topic

   • Properties
5. 5

   Multiple and partial correlation coefficients and their basic properties

   1 topic

   • Properties

Multivariate Analysis flashcards for GATE Statistics

25 of 49 cards from the Multivariate Analysis deck — real questions with worked answers.

1. What is the probability density function (pdf) of a $p$-dimensional multivariate normal random vector $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$?

   $$f(\vec{x}) = \frac{1}{(2\pi)^{p/2}\,|\Sigma|^{1/2}}\exp\left(-\frac{1}{2}(\vec{x}-\vec{\mu})^{\top}\Sigma^{-1}(\vec{x}-\vec{\mu})\right)$$ where $\vec{\mu}$ is the mean vector and $\Sigma$ is the positive-definite covariance matrix.

2. For $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$, what are $E[\vec{X}]$ and $\operatorname{Cov}(\vec{X})$?

   $E[\vec{X}] = \vec{\mu}$ and $\operatorname{Cov}(\vec{X}) = \Sigma$, where the $(i,j)$ entry is $\sigma_{ij} = \operatorname{Cov}(X_i, X_j)$.

3. State the moment generating function (MGF) of $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$.

   $$M_{\vec{X}}(\vec{t}) = \exp\left(\vec{t}^{\top}\vec{\mu} + \frac{1}{2}\vec{t}^{\top}\Sigma\,\vec{t}\right), \quad \vec{t} \in \mathbb{R}^{p}.$$

4. State the characteristic function of $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$.

   $$\phi_{\vec{X}}(\vec{t}) = \exp\left(i\,\vec{t}^{\top}\vec{\mu} - \frac{1}{2}\vec{t}^{\top}\Sigma\,\vec{t}\right).$$

5. Property: If $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$ and $A$ is a $q \times p$ constant matrix and $\vec{b}$ a constant $q$-vector, what is the distribution of $\vec{Y} = A\vec{X} + \vec{b}$?

   $\vec{Y} \sim N_{q}(A\vec{\mu} + \vec{b},\; A\Sigma A^{\top})$. Any linear transformation of a multivariate normal is again multivariate normal.

6. Property: How is the marginal distribution of any subset of components of $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$ distributed?

   Every marginal (any subset of components) is itself multivariate normal. If $\vec{X}_1$ is a $q$-subvector, then $\vec{X}_1 \sim N_{q}(\vec{\mu}_1, \Sigma_{11})$, where $\vec{\mu}_1$ and $\Sigma_{11}$ are the corresponding sub-vector and sub-matrix.

7. In a multivariate normal, what is the relationship between zero covariance and independence of two components $X_i$ and $X_j$?

   For jointly multivariate normal variables, $\operatorname{Cov}(X_i, X_j) = 0$ if and only if $X_i$ and $X_j$ are independent. (Uncorrelatedness implies independence only under joint normality.)

8. Partition $\vec{X} = (\vec{X}_1, \vec{X}_2)$ with $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$. What is the conditional distribution of $\vec{X}_1 \mid \vec{X}_2 = \vec{x}_2$?

   $$\vec{X}_1 \mid \vec{X}_2 = \vec{x}_2 \sim N\!\left(\vec{\mu}_1 + \Sigma_{12}\Sigma_{22}^{-1}(\vec{x}_2 - \vec{\mu}_2),\;\; \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}\right).$$

9. In the multivariate normal conditional $\vec{X}_1 \mid \vec{X}_2$, what is notable about the conditional covariance matrix $\Sigma_{11\cdot 2} = \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$?

   It does not depend on the conditioning value $\vec{x}_2$ (homoscedasticity). It is called the Schur complement of $\Sigma_{22}$ in $\Sigma$.

10. For a multivariate normal, the conditional mean $E[\vec{X}_1 \mid \vec{X}_2 = \vec{x}_2]$ is what kind of function of $\vec{x}_2$?

    It is a linear (affine) function of $\vec{x}_2$: $\;E[\vec{X}_1 \mid \vec{X}_2 = \vec{x}_2] = \vec{\mu}_1 + \Sigma_{12}\Sigma_{22}^{-1}(\vec{x}_2 - \vec{\mu}_2)$. This linear regression of $\vec{X}_1$ on $\vec{X}_2$ is exact under normality.

11. For the bivariate normal $(X, Y)$ with means $\mu_X, \mu_Y$, variances $\sigma_X^2, \sigma_Y^2$ and correlation $\rho$, what is $E[Y \mid X = x]$?

    $$E[Y \mid X = x] = \mu_Y + \rho\frac{\sigma_Y}{\sigma_X}(x - \mu_X).$$

12. For the bivariate normal $(X, Y)$ with correlation $\rho$, what is the conditional variance $\operatorname{Var}(Y \mid X = x)$?

    $$\operatorname{Var}(Y \mid X = x) = \sigma_Y^{2}(1 - \rho^{2}),$$ which is constant (independent of $x$).

13. Write the covariance matrix $\Sigma$ of a bivariate normal $(X, Y)$ in terms of $\sigma_X, \sigma_Y, \rho$.

    $$\Sigma = \begin{pmatrix} \sigma_X^{2} & \rho\sigma_X\sigma_Y \\ \rho\sigma_X\sigma_Y & \sigma_Y^{2} \end{pmatrix}, \quad |\Sigma| = \sigma_X^{2}\sigma_Y^{2}(1-\rho^{2}).$$

14. Property: What is the distribution of the linear combination $\vec{a}^{\top}\vec{X}$ for a fixed vector $\vec{a}$ when $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$?

    $\vec{a}^{\top}\vec{X} \sim N\!\left(\vec{a}^{\top}\vec{\mu},\; \vec{a}^{\top}\Sigma\,\vec{a}\right)$. In fact, $\vec{X}$ is multivariate normal iff every linear combination $\vec{a}^{\top}\vec{X}$ is univariate normal.

15. Define the squared Mahalanobis distance for $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$ and state its distribution.

    The squared Mahalanobis distance is $D^{2} = (\vec{X}-\vec{\mu})^{\top}\Sigma^{-1}(\vec{X}-\vec{\mu})$, and $D^{2} \sim \chi^{2}_{p}$ (chi-square with $p$ degrees of freedom).

16. What geometric shape do the contours of constant density of $N_{p}(\vec{\mu}, \Sigma)$ form?

    Ellipsoids centered at $\vec{\mu}$, defined by $(\vec{x}-\vec{\mu})^{\top}\Sigma^{-1}(\vec{x}-\vec{\mu}) = c^{2}$. Their axes point along the eigenvectors of $\Sigma$ with lengths proportional to $\sqrt{\lambda_i}$.

17. Property: If $\vec{X}_1, \dots, \vec{X}_n$ are independent with $\vec{X}_i \sim N_{p}(\vec{\mu}_i, \Sigma_i)$, what is the distribution of $\sum_{i=1}^{n} \vec{X}_i$?

    $$\sum_{i=1}^{n}\vec{X}_i \sim N_{p}\!\left(\sum_{i=1}^{n}\vec{\mu}_i,\; \sum_{i=1}^{n}\Sigma_i\right).$$ The multivariate normal is closed under (independent) addition.

18. What is the standardization (whitening) transform that converts $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$ into $\vec{Z} \sim N_{p}(\vec{0}, I)$?

    $\vec{Z} = \Sigma^{-1/2}(\vec{X} - \vec{\mu})$, where $\Sigma^{-1/2}$ is a symmetric square root of $\Sigma^{-1}$. Then $\vec{Z}$ has independent standard normal components.

19. For $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$, partition into $\vec{X}_1, \vec{X}_2$. When are $\vec{X}_1$ and $\vec{X}_2$ independent?

    $\vec{X}_1$ and $\vec{X}_2$ are independent if and only if the cross-covariance block $\Sigma_{12} = \mathbf{0}$ (equivalently $\Sigma_{21} = \mathbf{0}$).

20. What is the marginal distribution of a single component $X_i$ of $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$?

    $X_i \sim N(\mu_i, \sigma_{ii})$, where $\mu_i$ is the $i$-th element of $\vec{\mu}$ and $\sigma_{ii}$ is the $i$-th diagonal entry of $\Sigma$.

21. Define the population multiple correlation coefficient of $X_1$ with $(X_2, \dots, X_p)$ in a multivariate normal.

    $$R_{1\cdot 2\dots p} = \sqrt{\frac{\Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}}{\sigma_{11}}},$$ the maximum correlation between $X_1$ and any linear combination of $X_2,\dots,X_p$; it equals the correlation between $X_1$ and its best linear predictor.

22. Define the partial correlation coefficient $\rho_{12\cdot 3}$ between $X_1$ and $X_2$ given $X_3$.

    $$\rho_{12\cdot 3} = \frac{\rho_{12} - \rho_{13}\rho_{23}}{\sqrt{(1-\rho_{13}^{2})(1-\rho_{23}^{2})}}.$$ It measures the correlation between $X_1$ and $X_2$ after removing the linear effect of $X_3$.

23. What is a key consequence of the conditional-covariance formula regarding partial correlations in a multivariate normal?

    Partial correlations are obtained by standardizing the conditional covariance matrix $\Sigma_{11\cdot 2} = \Sigma_{11} - \Sigma_{12}\Sigma_{22}^{-1}\Sigma_{21}$. A zero partial correlation corresponds to conditional independence under normality.

24. Property: What is the distribution of $\vec{X} - \vec{\mu}$ when $\vec{X} \sim N_{p}(\vec{\mu}, \Sigma)$, and is the family closed under translation?

    $\vec{X} - \vec{\mu} \sim N_{p}(\vec{0}, \Sigma)$. The multivariate normal family is closed under translation (shifting only changes the mean vector, not $\Sigma$).

25. State the definition of the Wishart distribution and one role it plays.

    If $\vec{X}_1,\dots,\vec{X}_n$ are i.i.d. $N_{p}(\vec{0}, \Sigma)$, then $W = \sum_{i=1}^{n}\vec{X}_i\vec{X}_i^{\top} \sim W_{p}(n, \Sigma)$. It is the multivariate generalization of the chi-square and describes the distribution of the sample covariance (scatter) matrix.

See more Multivariate Analysis flashcards →

Planning Multivariate Analysis for GATE Statistics

Multivariate Analysis (GATE Statistics) FAQ