Chapter 28 Quiz — Positive Definite Matrices and Quadratic Forms
Twelve conceptual checks. Try each before opening the answer. One-line explanations follow every answer. Throughout, $A$ is a real symmetric matrix.
Q1. What is the geometric shape of the surface $z = \mathbf{x}^{\mathsf{T}}A\mathbf{x}$ when $A$ is positive definite?
Answer
An upward-opening **bowl** (a paraboloid) with a single lowest point at the origin. *Height is positive in every direction away from the origin — the defining property of positive definiteness.*Q2. A matrix has every entry positive. Must it be positive definite?
Answer
**No.** $\begin{psmallmatrix}2 & 3\\ 3 & 2\end{psmallmatrix}$ has all-positive entries but is indefinite (its form is $-2$ along $(1,-1)$). *Definiteness depends on eigenvalues, not entries; positive entries are neither necessary nor sufficient.*Q3. State the three equivalent tests for positive definiteness of a symmetric matrix.
Answer
(1) All **eigenvalues** positive; (2) all **pivots** positive; (3) all **leading principal minors** positive (Sylvester's criterion). *All three agree because each is a sign-faithful reading of the same surface.*Q4. Why do the eigenvalue test and the pivot test give the same verdict even though the eigenvalues and pivots are usually different numbers?
Answer
Both $A = QDQ^{\mathsf{T}}$ (eigenvalues) and $A = LDL^{\mathsf{T}}$ (pivots) write the form as a sum of squares, and by **Sylvester's law of inertia** congruent matrices share the same sign pattern. *Skewing your coordinates changes the curvature values but cannot turn a bowl into a saddle.*Q5. The contour lines of a quadratic form are nested ellipses. What is the definiteness, and what do the ellipse axes correspond to?
Answer
**Positive (or negative) definite.** The axes point along the **eigenvectors** of $A$, with half-lengths $\propto 1/\sqrt{\lambda}$. *Ellipses ⇒ definite; hyperbolas ⇒ indefinite; parallel lines ⇒ semidefinite.*Q6. Along which eigenvector does the long axis of the contour ellipse point — the largest or smallest eigenvalue?
Answer
The **smallest** eigenvalue. *Half-length is $\sqrt{c/\lambda}$, so a small $\lambda$ (shallow curvature) gives a long axis; steep curvature (big $\lambda$) gives a short axis. Steep ⇒ short.*Q7. At a critical point of a smooth function, the Hessian is positive definite. Is the point a maximum, minimum, or saddle?
Answer
A local **minimum**. *A positive definite Hessian means the surface is locally a bowl; this is the multivariable second-derivative test, which is exactly the definiteness classification.*Q8. Why is every covariance matrix positive semidefinite?
Answer
A covariance matrix is $\Sigma = \tfrac1N B^{\mathsf{T}}B$ for centered data $B$, so $\mathbf{w}^{\mathsf{T}}\Sigma\mathbf{w} = \tfrac1N\lVert B\mathbf{w}\rVert^2 \ge 0$. *The quadratic form is the variance of the data projected onto $\mathbf{w}$, which can never be negative.*Q9. What does a zero eigenvalue of a covariance matrix tell you about the data?
Answer
There is a direction of **zero variance** — a perfect linear dependence among the features, so the data lies flat in a lower-dimensional subspace. *That redundant direction is exactly what PCA (Chapter 32) discards.*Q10. State the Cholesky factorization and the condition under which it exists.
Answer
$A = LL^{\mathsf{T}}$ with $L$ lower-triangular and positive diagonal; it exists **if and only if $A$ is positive definite**. *This is why a successful `np.linalg.cholesky(A)` is the fast practical test for positive definiteness.*Q11. You are told a symmetric matrix has leading principal minors $\Delta_1 = -3$, $\Delta_2 = 4$, $\Delta_3 = -5$. What is its definiteness?
Answer
**Negative definite.** *The leading minors of a negative definite matrix alternate in sign starting negative ($-,+,-,\dots$), because $-A$ is positive definite. Positive definite would need all $\Delta_k > 0$.*Q12. Someone applies "all eigenvalues positive" to a non-symmetric matrix to conclude it is positive definite. What is the flaw?