Chapter 30 Quiz — The Singular Value Decomposition
Twelve conceptual checks. Try each before opening the answer. One-line explanations follow. Remember the headline: every matrix factors as $A = U\Sigma V^{\mathsf{T}}$ — rotate, stretch, rotate — with no exceptions.
Q1. State the rotate–stretch–rotate picture of $A = U\Sigma V^{\mathsf{T}}$ in one sentence.
Answer
Reading right to left, $V^{\mathsf{T}}$ **rotates** (or reflects) the input, $\Sigma$ **stretches** along perpendicular axes by the singular values, and $U$ **rotates** (or reflects) the result into place. *All of a matrix's distortion lives in the diagonal $\Sigma$; the two orthogonal factors preserve length.*Q2. Which matrices have an SVD?
Answer
**Every** real matrix — square or rectangular, symmetric or not, full-rank or rank-deficient (§30.5). *This universality is the SVD's defining advantage over diagonalization, which can fail.*Q3. Can a singular value be negative? Can it be zero? Can it be complex?
Answer
Singular values are always **real and non-negative** ($\sigma_i \ge 0$); they can be **zero** (signalling rank deficiency) but never negative and never complex. *They are $\sqrt{\lambda_i}$ where $\lambda_i \ge 0$ are the eigenvalues of the symmetric positive-semidefinite $A^{\mathsf{T}}A$.*Q4. How is the SVD of $A$ related to the eigen-decomposition of $A^{\mathsf{T}}A$?
Answer
The **right singular vectors** (columns of $V$) are the eigenvectors of $A^{\mathsf{T}}A$, and the **singular values** are $\sigma_i = \sqrt{\lambda_i}$ where $\lambda_i$ are its eigenvalues (§30.5–30.6). *Likewise the left singular vectors are eigenvectors of $AA^{\mathsf{T}}$. The SVD is the spectral theorem applied to $A^{\mathsf{T}}A$.*Q5. Why does the SVD exist for every matrix when diagonalization does not?
Answer
Because the SVD never asks $A$ to be nice — it asks $A^{\mathsf{T}}A$ to be nice, and $A^{\mathsf{T}}A$ is **always symmetric**, hence always orthogonally diagonalizable by the Spectral Theorem (Chapter 27). *The symmetrization launders away complex eigenvalues, defectiveness, and non-squareness.*Q6. A matrix has singular values $\{9, 5, 0\}$. What is its rank?
Answer
**2** — the rank equals the number of *nonzero* singular values (§30.7). *Multiplying by the orthogonal $U, V^{\mathsf{T}}$ preserves rank, so $\operatorname{rank}(A) = \operatorname{rank}(\Sigma) = $ count of nonzero diagonal entries.*Q7. Which columns of $U$ and $V$ give a basis for the null space of $A$, and which for the column space?
Answer
For rank $r$: the **last** $n - r$ columns of $V$ (the right singular vectors with $\sigma_i = 0$) span the **null space**; the **first** $r$ columns of $U$ span the **column space** (§30.8). *The SVD gives orthonormal bases for all four fundamental subspaces at once.*Q8. What does the largest singular value $\sigma_1$ equal, as a norm?
Answer
The **operator (2-)norm** $\lVert A\rVert_2 = \max_{\lVert\mathbf{x}\rVert=1}\lVert A\mathbf{x}\rVert$ — the largest factor by which $A$ stretches any unit vector (§30.9). *Geometrically, it is the longest semi-axis of the ellipse $A$ makes from the unit circle.*Q9. What is the condition number of $A$ in terms of singular values, and what does a large value mean?
Answer
$\kappa(A) = \sigma_1/\sigma_n$ (largest over smallest). A **large** condition number means the matrix stretches some direction far more than another — a long, thin ellipse — so the system $A\mathbf{x} = \mathbf{b}$ is **ill-conditioned** and small input errors are badly amplified (§30.9, tees up Chapter 38). *Near $1$ is well-conditioned; infinite means singular.*Q10. np.linalg.svd(A) returns three things. What are they, and what is the classic bug?
Answer
It returns `(U, S, Vt)`: the left singular vectors $U$, a 1-D array `S` of singular values (descending), and **$V^{\mathsf{T}}$** — not $V$. *The classic bug is using the third output as $V$; to get $V$, transpose it (`V = Vt.T`).*Q11. Your hand SVD gives $U = I$ but numpy returns $U = \begin{psmallmatrix}-1 & 0\\0 & 1\end{psmallmatrix}$, with a sign-flipped first row of $V^{\mathsf{T}}$. Did you make an error?
Answer
**No.** The SVD is not unique: flipping the signs of a paired $\mathbf{u}_i$ *and* $\mathbf{v}_i$ together leaves $A = U\Sigma V^{\mathsf{T}}$ unchanged (§30.4). *The singular values and the reconstruction are unique; the singular vectors carry a paired-sign freedom. Never flip a $\mathbf{u}_i$ without flipping its $\mathbf{v}_i$.*Q12. For which matrices do the singular values equal the eigenvalues?