Case Study 39.1 — Compressing a Photograph with Your Own SVD (Track A)
A fully worked capstone walkthrough in the image-compression track. We follow one project from blank file to a defensible results section, using the toolkit you built. Every number here matches the accompanying numpy.
The brief
Maya, a data-science student, wants her capstone to make the SVD visible. Her one-sentence pitch: "Show that a photograph is a matrix, and that keeping only its most important singular values reconstructs the image at a fraction of the storage." This is Track A, the book's signature application, and it has the advantage that the result is something you can literally see. She will use toolkit/capstone/image_compression.py, validate it against the Eckart–Young theorem (Chapter 31), and — to honor the spirit of the project — confirm that her from-scratch svd_from_scratch (Chapter 30) reproduces numpy's singular values.
Step 1 — Encode the problem as a matrix
A grayscale image is a matrix A: row i, column j holds the brightness of one pixel. Maya works with a 512×512 image, so A is 512×512 = 262,144 numbers. The modeling decision of §39.8 is already made for her by the domain — rows and columns are pixel coordinates, entries are intensities — which is part of why this track is a gentle first capstone. She loads the image as a float matrix; the demo's load_grayscale falls back to a deterministic, photograph-like synthetic image (a power-law singular-value spectrum, σᵢ = 300/i) when no file is supplied, so her numbers are reproducible by anyone who runs the code.
Step 2 — Choose the operation: the truncated SVD
The SVD factors A = UΣVᵀ. Truncating to the top k terms gives
$$A_k = \sum_{i=1}^{k} \sigma_i\, \mathbf{u}_i \mathbf{v}_i^{\mathsf{T}},$$
which, by the Eckart–Young theorem, is the best possible rank-k approximation of A in the Frobenius norm. Maya states the condition explicitly in her report — Eckart–Young is about the best approximation among all rank-k matrices — because the rubric (item 1) rewards naming a theorem's conditions, not just invoking it. Each rank-one layer σᵢ uᵢ vᵢᵀ is one "stratum" of the image; the layers are ordered by importance because the singular values are sorted descending.
The storage accounting is exact. Keeping k layers costs k(m + n + 1) numbers — k columns of U, k rows of Vᵀ, and k singular values — instead of mn. The fidelity lost is exactly the energy in the discarded singular values.
Step 3 — Compute it with the toolkit
The two functions doing the work are short. Reconstruction:
# A_k = U_k Σ_k V_k^T, reusing the kept k columns/rows.
def rank_k_approximation(U, s, Vt, k):
return (U[:, :k] * s[:k]) @ Vt[:k]
And the per-rank report, which computes storage, ratio, relative error, and captured energy, then asserts the two Eckart–Young invariants before returning anything:
U, s, Vt = np.linalg.svd(A, full_matrices=False) # or Maya's svd_from_scratch
A_k = rank_k_approximation(U, s, Vt, k)
rel_error = np.linalg.norm(A - A_k, "fro") / np.linalg.norm(A, "fro")
ey_tail = np.sqrt(np.sum(s[k:]**2)) / np.linalg.norm(A, "fro")
assert np.isclose(rel_error, ey_tail) # fidelity == discarded energy
storage = k * (m + n + 1)
Maya runs python -m toolkit.capstone.image_compression from the repository root.
Step 4 — Results
The demo prints, exactly:
image: 512 x 512 = 262144 pixels; full storage = 262144 numbers
rank k | storage | ratio | % orig | rel err | energy
5 | 5125 | 51.15x | 2.0% | 0.3304 | 0.8908
10 | 10250 | 25.58x | 3.9% | 0.2382 | 0.9433
50 | 51250 | 5.12x | 19.6% | 0.1042 | 0.9891
200 | 205000 | 1.28x | 78.2% | 0.0430 | 0.9982
Verified: error decreases with k, and equals the Eckart-Young tail at every rank.
She reads the story straight off the table. At rank 10, she stores 10,250 numbers — 3.9% of the original — and captures 94.3% of the image's energy, with a relative error of 0.2382. At rank 50 the error drops to 0.1042 (energy 98.91%) while storage climbs to 19.6% of the original. By rank 200 the reconstruction is at 0.0430 relative error — visually indistinguishable from the original — but storage is now 78.2% of the full image, so the compression has nearly evaporated. The headline she will lead with: "A photograph compressed to 4% of its size and you can still tell what it is."
Step 5 — Validation
Maya validates at all three levels of §39.8.2.
Agreement with a theoretical invariant. The demo asserts both Eckart–Young facts internally: (i) the relative error decreases monotonically as k grows — 0.3304 > 0.2382 > 0.1042 > 0.0430 — and (ii) at every k the measured relative error equals the discarded-energy tail sqrt(Σ_{i>k} σᵢ²)/‖A‖_F to floating-point precision. These are checks against the mathematics, independent of any reference implementation, and they are the strongest evidence she has that her reconstruction is correct.
Agreement up to a documented ambiguity. She then swaps numpy's np.linalg.svd for her own svd_from_scratch, which computes the singular values as the square roots of the eigenvalues of Aᵀ A. On a small test matrix A = [[3,2,2],[2,3,-2]] she confirms both routes give singular values [5, 3] — an exact match. On the full image the singular values agree to many digits, but her U differs from numpy's by some column signs. She does not treat this as a bug: she notes in her report that singular vectors are unique only up to sign, that the signs cancel in σᵢ uᵢ vᵢᵀ, and that her reconstruction A_k matches numpy's to tolerance regardless. This is exactly the Computational Note from the chapter, and demonstrating that she understands it is worth more than a clean equality test.
Agreement on a known answer. Finally, she sanity-checks the storage arithmetic by hand: at k = 10, k(m+n+1) = 10 × 1025 = 10,250, and mn/(k(m+n+1)) = 262144/10250 = 25.58×. The numbers match the table.
Step 6 — Presentation
Maya's figures make the transformation visible, satisfying the G-track rubric item:
- Side-by-side reconstructions at
k = 5, 10, 50, 200next to the original. The rank-5 image is blurry but recognizable; by rank 50 it is sharp; rank 200 is indistinguishable. This is the "wow." - The singular-value spectrum on a log scale, showing the steep power-law decay that explains why truncation works: a few singular values carry most of the energy.
- Relative error versus
k, the accuracy-vs-storage trade-off, with the "knee" of the curve annotated — the point past which extra layers buy little fidelity for a lot of storage.
She closes with the Check-Your-Understanding insight from the chapter: storage grows linearly in k, but captured energy grows fast then flattens, so the useful compression lives at small k, on the steep part of the energy curve. Choosing k near the knee is the entire art.
Step 7 — Limitations and next steps
Honest edges (rubric item 5): the demo uses a synthetic image with an idealized power-law spectrum, so a real photograph (especially one with sharp edges or text) may need a larger k for the same perceived quality. The truncated SVD is also not how JPEG actually compresses — JPEG uses a block discrete cosine transform — so Maya is careful to call this a demonstration of low-rank approximation, not a competitive codec. Her next step (the ⭐⭐⭐⭐ extension): run the SVD on each RGB channel to compress a color image, and compare the storage she needs against a JPEG at matched visual quality.
What this capstone demonstrates
Maya's project lands every theme the book has been building. A matrix is a transformation — the image is a matrix, and the SVD reveals its layered structure. Geometry and algebra are one object — the singular values are both an algebraic factorization and the literal "importance" of each visual stratum. Computation validated theory — the measured error equals the Eckart–Young prediction exactly, and her from-scratch SVD met numpy. And linear algebra is the most applied branch of pure mathematics — the same decomposition that compressed this image will, in the next case study's sibling tracks, factor a recommender and reduce a dataset. She has not just finished a chapter; she has built, validated, and can defend a complete piece of applied linear algebra.