Chapter 27 Exercises — The Spectral Theorem

Work the tiers in order; each builds on the last. ⭐ checks understanding, ⭐⭐ asks for hand computation, ⭐⭐⭐ asks for a proof (A) or code (C), and ⭐⭐⭐⭐ is an open-ended application. Problems marked [code] want a short numpy snippet (use np.linalg.eigh for symmetric/Hermitian matrices); [proof] wants a rigorous argument in the §10 four-part style. Throughout, $A$ denotes a symmetric real matrix ($A = A^{\mathsf{T}}$) unless stated otherwise, and $Q$ an orthogonal matrix of its eigenvectors. State the symmetry/Hermitian condition wherever the Spectral Theorem is invoked.

Tier 1 — ⭐ Conceptual

27.1. State the Spectral Theorem for real symmetric matrices in one sentence, naming all three of its guarantees and its single hypothesis.

27.2. True or false, each with a one-line reason: (a) every matrix is diagonalizable; (b) every symmetric matrix is diagonalizable; (c) every diagonalizable matrix is symmetric; (d) every symmetric matrix is orthogonally diagonalizable.

27.3. Geometrically, what does a symmetric transformation do to space? Use the words "perpendicular," "stretch," and (in the negative) "rotation" or "shear."

27.4. A matrix has a complex eigenvalue $2 + 3i$. Can it be symmetric (real)? Why or why not?

27.5. Two eigenvectors of a symmetric matrix correspond to eigenvalues $7$ and $-1$. What is the angle between them, with no calculation? Which theorem tells you?

27.6. Why is the change-of-basis matrix in $A = Q\Lambda Q^{\mathsf{T}}$ called "free to invert," and how does that compare to the general $A = PDP^{-1}$ of Chapter 25?

27.7. What is the complex analogue of a symmetric matrix called, what equation defines it, and what does the Spectral Theorem guarantee about its eigenvalues? Write the factorization with the correct (conjugate-transpose) notation.

27.8. Why must a quantum observable be represented by a Hermitian matrix? Answer in terms of what the eigenvalues physically represent.

27.9. In the spectral decomposition $A = \sum_i \lambda_i \mathbf{q}_i\mathbf{q}_i^{\mathsf{T}}$, what kind of matrix is each term $\mathbf{q}_i\mathbf{q}_i^{\mathsf{T}}$ (give its rank and one geometric description), and what does the eigenvalue $\lambda_i$ do to it?

Tier 2 — ⭐⭐ Hand computation

27.10. Orthogonally diagonalize $A = \begin{psmallmatrix}3 & 1\\ 1 & 3\end{psmallmatrix}$ by hand: find the eigenvalues, the orthonormal eigenvectors, and write $Q$, $\Lambda$. Verify $\mathbf{q}_1\cdot\mathbf{q}_2 = 0$ and reconstruct $A = Q\Lambda Q^{\mathsf{T}}$.

27.11. Orthogonally diagonalize $A = \begin{psmallmatrix}0 & 2\\ 2 & 0\end{psmallmatrix}$. Interpret the eigenvalues and eigenvectors geometrically (what does this matrix do to the plane?).

27.12. Write the spectral decomposition $A = \lambda_1\mathbf{q}_1\mathbf{q}_1^{\mathsf{T}} + \lambda_2\mathbf{q}_2\mathbf{q}_2^{\mathsf{T}}$ for the matrix of Exercise 27.10 explicitly (compute the two $2\times2$ projector matrices) and confirm the weighted sum equals $A$.

27.13. For $A = \begin{psmallmatrix}2 & 1\\ 1 & 2\end{psmallmatrix}$ (eigenvalues $3$ and $1$), use $A^k = Q\Lambda^k Q^{\mathsf{T}}$ to compute $A^4$ by hand without four matrix multiplications. Check your answer against $\operatorname{tr}(A^4) = 3^4 + 1^4$.

27.14. Verify the identities $\operatorname{tr}(A) = \sum_i\lambda_i$ and $\det(A) = \prod_i\lambda_i$ for $A = \begin{psmallmatrix}5 & 2\\ 2 & 2\end{psmallmatrix}$ by computing the eigenvalues from the characteristic polynomial and comparing.

27.15. Is the matrix $B = \begin{psmallmatrix}1 & 4\\ 2 & 1\end{psmallmatrix}$ symmetric? Find its eigenvalues and eigenvectors, and check whether the eigenvectors are orthogonal. Explain how your finding is consistent with the Spectral Theorem's conditions.

27.16. Show that the matrix $Y = \begin{psmallmatrix}0 & -i\\ i & 0\end{psmallmatrix}$ is Hermitian (compute $Y^{*}$) but not symmetric in the plain-transpose sense ($Y^{\mathsf{T}} \ne Y$). What does this tell you about which condition the Spectral Theorem actually needs over $\mathbb{C}$?

27.17. A symmetric matrix has the spectral decomposition $A = 4\,\mathbf{q}_1\mathbf{q}_1^{\mathsf{T}} + 0\cdot\mathbf{q}_2\mathbf{q}_2^{\mathsf{T}}$ with $\mathbf{q}_1 = \tfrac{1}{\sqrt2}(1,1)$. Write $A$ as an explicit $2\times2$ matrix. What are its rank, its null space, and its column space? (Hint: a zero eigenvalue drops a projector.)

Tier 3 — ⭐⭐⭐ Proof (A) / Coding (C)

27.18. [proof] Prove that a real symmetric matrix has only real eigenvalues. Use the four-part shape, and state clearly where you use the hypothesis $A = A^{\mathsf{T}}$ (equivalently $A^{*} = A$).

27.19. [proof] Prove that eigenvectors of a real symmetric matrix corresponding to distinct eigenvalues are orthogonal. Identify the exact step where the self-adjoint identity $(A\mathbf{u})\cdot\mathbf{w} = \mathbf{u}\cdot(A\mathbf{w})$ is used.

27.20. [proof] Prove the easy converse: if $A = Q\Lambda Q^{\mathsf{T}}$ with $Q$ orthogonal and $\Lambda$ real diagonal, then $A$ is symmetric. (One line, using the transpose-of-a-product rule from Chapter 8.)

27.21. [proof] Let $A$ be symmetric and $P_i = \mathbf{q}_i\mathbf{q}_i^{\mathsf{T}}$ a rank-one projector onto a unit eigenvector. Prove that (a) $P_i^{\mathsf{T}} = P_i$, (b) $P_i^2 = P_i$ (idempotent), and (c) $P_iP_j = 0$ for $i \ne j$ (using orthonormality). Conclude that $\sum_i P_i = I$.

27.22. [proof] Prove that if $A$ is symmetric then $A^k = Q\Lambda^k Q^{\mathsf{T}}$ for every positive integer $k$, using the orthonormality $Q^{\mathsf{T}}Q = I$ to telescope the product. Then extend the argument to define $f(A) = Qf(\Lambda)Q^{\mathsf{T}}$ for a polynomial $f$.

27.23. [proof] Suppose $A$ is symmetric with all eigenvalues positive. Prove there is a symmetric matrix $S$ with $S^2 = A$ (a "symmetric square root"). Where do you use that the eigenvalues are positive, and what goes wrong if one is negative?

27.24. [code] Write orthogonally_diagonalize(A) that checks $A = A^{\mathsf{T}}$ (raising an error otherwise), then returns $Q, \Lambda$ via np.linalg.eigh. Test it on $\begin{psmallmatrix}2&1\\1&2\end{psmallmatrix}$ and confirm np.allclose(Q @ Λ @ Q.T, A) and np.allclose(Q.T @ Q, np.eye(2)). Then call it on a non-symmetric matrix and confirm it raises.

27.25. [code] Numerically demonstrate all three failure modes of §27.8: print the (complex) eigenvalues of a $90°$ rotation, show the shear $\begin{psmallmatrix}1&1\\0&1\end{psmallmatrix}$ has only one independent eigenvector, and show the eigenvectors of $\begin{psmallmatrix}2&1\\0&3\end{psmallmatrix}$ have nonzero dot product. Label each output with which guarantee of the theorem it violates.

27.26. [code] Implement the spectral decomposition as a sum of rank-one projectors for a symmetric $4\times4$ matrix of your choice. Print each $\lambda_i\mathbf{q}_i\mathbf{q}_i^{\mathsf{T}}$, confirm the projectors satisfy $\sum_i\mathbf{q}_i\mathbf{q}_i^{\mathsf{T}} = I$, and confirm the weighted sum reconstructs $A$.

27.27. [code] Verify the Hermitian Spectral Theorem for the Pauli-$Y$ matrix $\begin{psmallmatrix}0&-i\\i&0\end{psmallmatrix}$: confirm $Y^{*} = Y$, that np.linalg.eigh returns real eigenvalues $\pm1$ and an orthonormal (unitary) $U$ with $U^{*}U = I$, and that $U\Lambda U^{*}$ reconstructs $Y$.

Tier 4 — ⭐⭐⭐⭐ Application

27.28. [code] Covariance and PCA preview. Generate 500 points of correlated 2D data, center them, and form the (symmetric) covariance matrix $C = \tfrac{1}{n-1}X^{\mathsf{T}}X$. Use eigh to find its eigenvalues (variances) and eigenvectors (principal axes). Confirm the eigenvectors are orthogonal, that the eigenvalues are non-negative, and report the fraction of total variance captured by the larger eigenvalue. Explain how this is the Spectral Theorem applied to a covariance matrix (forward link to Chapter 32).

27.29. [code + proof] Quadratic form classification. (a) [code] For $A = \begin{psmallmatrix}3&1\\1&3\end{psmallmatrix}$, evaluate the quadratic form $q(\mathbf{x}) = \mathbf{x}^{\mathsf{T}}A\mathbf{x}$ at 1,000 random unit vectors and confirm every value lies between the smallest and largest eigenvalues. (b) [proof] Prove in general that for symmetric $A$ and unit $\mathbf{x}$, $\lambda_{\min} \le \mathbf{x}^{\mathsf{T}}A\mathbf{x} \le \lambda_{\max}$, using the spectral form $q(\mathbf{x}) = \sum_i\lambda_i y_i^2$ with $\sum_i y_i^2 = 1$. (This is the Rayleigh-quotient bound; it foreshadows Chapter 28.)

27.30. [code] Stress tensor / principal axes. A 2D stress state at a point in a loaded beam is the symmetric tensor $\sigma = \begin{psmallmatrix}50 & 30\\ 30 & -10\end{psmallmatrix}$ (MPa). Use eigh to find the principal stresses (eigenvalues) and the principal directions (eigenvectors). Confirm the principal directions are perpendicular, and state which named theorem guarantees they are. (The maximum shear stress equals half the difference of the principal stresses — compute it.)

27.31. [open-ended] Why symmetry, specifically? Write a short paragraph (no computation required) explaining to a classmate why the single condition $A = A^{\mathsf{T}}$ is exactly what forces real eigenvalues and orthogonal eigenvectors, using the self-adjoint identity $(A\mathbf{x})\cdot\mathbf{y} = \mathbf{x}\cdot(A\mathbf{y})$ as the central idea. Then explain why dropping symmetry can break each of the three guarantees, with a one-line example of each.

27.32. [code] Graph Laplacian and spectral clustering. Build the symmetric Laplacian $L = D - A$ of a small undirected graph (say two triangles joined by a single edge — a "barbell"). Use eigh to find its eigenvalues and eigenvectors. Confirm $L$ is symmetric with a zero eigenvalue, and show that the eigenvector for the second-smallest eigenvalue (the "Fiedler vector") separates the two triangles by the sign of its entries. Explain why this is a real-world, non-physics use of the Spectral Theorem.