GATE Statistics Matrix Theory Syllabus

What does the Gram-Schmidt process accomplish?

It converts a linearly independent set $\{v_1,\dots,v_k\}$ into an orthogonal (or orthonormal) set $\{u_1,\dots,u_k\}$ spanning the same subspace.

Write the Gram-Schmidt formula for the orthogonal vector $u_k$.

$$u_k = v_k - \sum_{j=1}^{k-1} \frac{\langle v_k, u_j\rangle}{\langle u_j, u_j\rangle}\, u_j$$ subtracting the projections of $v_k$ onto each previous orthogonal vector.

In Gram-Schmidt, how do you obtain an orthonormal vector from an orthogonal vector $u_k$?

Normalize it: $$e_k = \frac{u_k}{\lVert u_k \rVert}.$$ The resulting $\{e_k\}$ are mutually orthogonal and each of unit length.

What is the projection of a vector $v$ onto a nonzero vector $u$ (used in Gram-Schmidt)?

$$\text{proj}_u(v) = \frac{\langle v, u\rangle}{\langle u, u\rangle}\, u.$$

What matrix factorization is naturally produced by applying Gram-Schmidt to the columns of a matrix $A$?

The QR decomposition $A = QR$, where $Q$ has orthonormal columns and $R$ is upper triangular.

Define the characteristic polynomial of an $n \times n$ matrix $A$.

$$p(\lambda) = \det(A - \lambda I).$$ It is a degree-$n$ polynomial in $\lambda$ whose roots are the eigenvalues of $A$.

For a $2 \times 2$ matrix $A$, express the characteristic polynomial using trace and determinant.

$$p(\lambda) = \lambda^{2} - \text{tr}(A)\,\lambda + \det(A).$$

How are the trace and determinant of $A$ related to its eigenvalues $\lambda_1,\dots,\lambda_n$?

$$\text{tr}(A) = \sum_{i=1}^{n}\lambda_i, \qquad \det(A) = \prod_{i=1}^{n}\lambda_i.$$

Distinguish the algebraic and geometric multiplicity of an eigenvalue.

Algebraic multiplicity is the multiplicity of $\lambda$ as a root of the characteristic polynomial. Geometric multiplicity is $\dim(\ker(A-\lambda I))$, the number of independent eigenvectors. Always $1 \leq \text{geometric} \leq \text{algebraic}$.

State the Cayley-Hamilton Theorem.

Every square matrix satisfies its own characteristic equation: if $p(\lambda) = \det(A - \lambda I)$, then $p(A) = 0$ (the zero matrix).

How can the Cayley-Hamilton Theorem be used to compute $A^{-1}$ for an invertible $n \times n$ matrix?

From $p(A)=0$, isolate the identity term and solve. E.g. for $A^{2} - \text{tr}(A)A + \det(A)I = 0$ ($2\times2$): $$A^{-1} = \frac{1}{\det(A)}\big(\text{tr}(A)I - A\big).$$

How does Cayley-Hamilton help express high powers of $A$, such as $A^{n}$, in terms of lower powers?

Since $p(A)=0$ gives a degree-$n$ relation, $A^{n}$ can be written as a linear combination of $I, A, A^{2}, \dots, A^{n-1}$, and the relation can be applied repeatedly to reduce any higher power.

What is a change of basis matrix from basis $B$ to basis $C$?

The matrix $P_{C \leftarrow B}$ whose columns are the coordinate vectors of the $B$-basis vectors expressed in the $C$-basis. It transforms coordinates via $[v]_C = P_{C \leftarrow B}\,[v]_B$.

How is the inverse change of basis matrix related to $P_{C \leftarrow B}$?

$$P_{B \leftarrow C} = \left(P_{C \leftarrow B}\right)^{-1}.$$ Change of basis matrices are always invertible.

If $B$ is a basis with columns forming matrix $P$ (relative to the standard basis), how do you convert standard coordinates $x$ to $B$-coordinates?

$$[x]_B = P^{-1} x,$$ where $P = [\,b_1 \; b_2 \; \cdots \; b_n\,]$ has the basis vectors as columns. Conversely $x = P[x]_B$.

How does the matrix of a linear operator transform under a change of basis with transition matrix $P$?

If $A$ represents the operator in the old basis, then in the new basis it becomes $A' = P^{-1} A P$ — a similarity transformation.

When are two $m \times n$ matrices $A$ and $B$ called equivalent?

When $B = P A Q$ for some invertible matrices $P$ ($m\times m$) and $Q$ ($n\times n$). Equivalently, one can be obtained from the other by elementary row and column operations.

What single invariant completely characterizes matrix equivalence?

Rank. Two $m\times n$ matrices are equivalent if and only if they have the same rank.

When are two square matrices $A$ and $B$ called similar?

When there exists an invertible matrix $P$ such that $B = P^{-1} A P$. Similar matrices represent the same linear operator in different bases.

List four invariants shared by similar matrices.

Determinant, trace, rank, characteristic polynomial (hence eigenvalues with their algebraic multiplicities). Geometric multiplicities and the minimal polynomial are also preserved.

What is the key logical relationship between similarity and equivalence of matrices?

Similarity implies equivalence (take $Q = P^{-1}$, so $B = P^{-1}AP$ is a special case of $PAQ$), but equivalence does not imply similarity. Equivalence only requires equal rank.

When is an $n \times n$ matrix $A$ diagonalizable?

When it is similar to a diagonal matrix: $A = PDP^{-1}$ with $D$ diagonal. Equivalently, $A$ has $n$ linearly independent eigenvectors (which form the columns of $P$).

State a sufficient condition for diagonalizability based on eigenvalues.

If an $n\times n$ matrix has $n$ distinct eigenvalues, it is diagonalizable. (This is sufficient but not necessary.)

Give the necessary and sufficient condition for diagonalizability in terms of multiplicities.

For every eigenvalue, the geometric multiplicity equals the algebraic multiplicity. Equivalently, the sum of geometric multiplicities equals $n$.

If $A = PDP^{-1}$ is diagonalizable, how is $A^{k}$ computed?

$$A^{k} = P D^{k} P^{-1},$$ where $D^{k}$ is obtained by raising each diagonal entry (eigenvalue) to the power $k$.

What does the Spectral Theorem say about real symmetric matrices?

Every real symmetric matrix is orthogonally diagonalizable: $A = Q D Q^{T}$ with $Q$ orthogonal ($Q^{T}=Q^{-1}$) and $D$ real diagonal. Its eigenvalues are real and eigenvectors for distinct eigenvalues are orthogonal.

What is a quadratic form in $n$ variables, written in matrix notation?

$$Q(x) = x^{T} A x = \sum_{i,j} a_{ij} x_i x_j,$$ where $A$ is taken to be a symmetric matrix. Each quadratic form corresponds to a unique symmetric matrix.

Classify quadratic forms as positive definite, negative definite, indefinite, and semidefinite in terms of eigenvalues of the symmetric matrix $A$.

Positive definite: all $\lambda_i > 0$. Negative definite: all $\lambda_i < 0$. Positive semidefinite: all $\lambda_i \geq 0$. Negative semidefinite: all $\lambda_i \leq 0$. Indefinite: both positive and negative eigenvalues present.

State Sylvester's criterion for a symmetric matrix $A$ to be positive definite.

All leading principal minors of $A$ are positive: $\det(A_1) > 0, \det(A_2) > 0, \dots, \det(A_n) > 0$, where $A_k$ is the top-left $k\times k$ submatrix.

What does Sylvester's Law of Inertia state about a quadratic form?

The number of positive, negative, and zero eigenvalues (the signature, $p$, $q$, $z$) is invariant under any congruence transformation $A \mapsto P^{T} A P$ with $P$ invertible — it does not depend on the diagonalizing change of variables.