Chapter 40 — Exercises

This is the final chapter, so the exercises are unlike any other set in the book. They are reflective and exploratory rather than drill: many have no single right answer, several ask you to connect ideas across chapters, and the hardest ones are open-ended invitations to keep building. Work the ones that match where you want to go next. Tiers: ⭐ conceptual/reflective · ⭐⭐ computation & cross-chapter synthesis · ⭐⭐⭐ proof (A) / coding (C) · ⭐⭐⭐⭐ open-ended exploration.

A note on the open-ended problems: there is no answer key for the ⭐⭐⭐⭐ tier on purpose. Treat them as the start of a project, a blog post, or a conversation. The point of a closing chapter is not to test you one more time but to send you off with somewhere to go.

⭐ Conceptual & reflective

40.1. In one sentence each, state the two honest answers this chapter gave to "what comes after linear algebra?" (Hint: one keeps the objects and generalizes; one keeps the methods and re-applies.)

40.2. A scalar has 0 indices, a vector has 1, a matrix has 2. What is the order of a tensor with components $t_{ijk\ell}$? How many numbers does it store if each index runs over $n$ values?

40.3. Explain, in your own words, the difference between the two meanings of the word "rank" when it appears near tensors. Give an example matrix where the two numbers differ.

40.4. What single property distinguishes a Hilbert space from an ordinary inner product space, and why did this book never need to mention that property?

40.5. The chapter claims "quantum mechanics is functional analysis" but also that the qubit could be studied "with ordinary linear algebra." Reconcile these two statements.

40.6. For each of the six recurring themes, write the one-sentence version from memory, then check it against §40.6. Which theme do you find yourself reaching for most naturally? Which least? (There is no wrong answer — this is a self-assessment.)

40.7. The chapter says a quantum gate is a unitary matrix and that this is "exactly the kind of map that preserves length." Why must a gate preserve length? (Connect to what the squared amplitudes represent.)

40.8. Name one DataField book this chapter connects to, state the specific linear-algebra object that forms the connection, and say which chapter of this book introduced that object.

⭐⭐ Computation & cross-chapter synthesis

40.9. (C) Reproduce the contraction snippet from §40.2: build a random $3\times 4\times 5$ tensor with np.random and contract its last axis with a length-5 vector using np.einsum('ijk,k->ij', ...). Confirm the result has shape $(3,4)$. Then contract the middle axis instead and report the new shape.

40.10. (C) Verify that matrix multiplication is a tensor contraction: for two random $3\times 3$ matrices $A,B$, confirm that A @ B equals np.einsum('ij,jk->ik', A, B). Which index is being summed over, and what is its linear-algebra meaning (Chapter 8)?

40.11. (C) Build the rank-one order-3 tensor $t_{ijk}=u_i v_j w_k$ for three small vectors using np.einsum('i,j,k->ijk', u, v, w). Verify by hand that one chosen entry equals the product of the corresponding components. (Remember math indices are 1-based, numpy 0-based.)

40.12. (C) Take a 5-node cycle graph (nodes $0$–$1$–$2$–$3$–$4$–$0$). Build its adjacency matrix $A$, form the Laplacian $L=D-A$, and compute its eigenvalues with np.linalg.eigh. How many are zero? What does that count tell you about the graph (§40.4)?

40.13. (C) For the path graph in §40.4, the Fiedler vector had signs $(+,+,-,-)$. Compute the Fiedler vector of a graph made of two triangles joined by a single edge, and check that its sign pattern separates the two triangles. (This is spectral clustering in miniature.)

40.14. (C) Confirm the Hadamard gate's key facts: build $H=\frac{1}{\sqrt 2}\begin{bsmallmatrix}1&1\\1&-1\end{bsmallmatrix}$, verify it is unitary (H.conj().T @ H is the identity), apply it to $|0\rangle$ and to $|1\rangle$, and check that the squared amplitudes of each result sum to 1.

40.15. (C) Build the phase gate $S=\begin{bsmallmatrix}1&0\\0&i\end{bsmallmatrix}$ (complex!). Show that S.T @ S is not the identity but S.conj().T @ S is. Explain in one sentence why the conjugate transpose is the right adjoint here (Chapter 27, §40.4 Computational Note).

40.16. Without running code, predict the shape of np.einsum('bij,bjk->bik', X, Y) where X is $(8,3,4)$ and Y is $(8,4,5)$. What familiar operation is this doing, and what is the role of the index b? (This is "batched matrix multiplication," the workhorse of §40.2.)

⭐⭐⭐ Proof & deeper coding

40.17. (A) Prove that the outer product $\mathbf{u}\mathbf{v}^{\mathsf{T}}$ of two nonzero vectors is a matrix of column-space rank exactly 1. Then explain why the SVD (Chapter 30) writing $A=\sum_k \sigma_k \mathbf{u}_k\mathbf{v}_k^{\mathsf{T}}$ is literally a sum of rank-one pieces, and connect this to the §40.2 claim that the order-3 outer product is the "rank-one building block one order up."

40.18. (A) Prove the finite-dimensional Parseval identity used in §40.3: if $\{\mathbf{q}_1,\dots,\mathbf{q}_n\}$ is an orthonormal basis of $\mathbb{R}^n$ and $\mathbf{f}=\sum_i c_i\mathbf{q}_i$, then $\lVert\mathbf{f}\rVert^2=\sum_i c_i^2$. (Hint: expand $\langle\mathbf{f},\mathbf{f}\rangle$ and use orthonormality — this is Chapter 18.) State clearly which step would require completeness if the sum were infinite.

40.19. (A) Prove that a real matrix $U$ with $U^{\mathsf{T}}U=I$ preserves the dot product: $\langle U\mathbf{x},U\mathbf{y}\rangle=\langle\mathbf{x},\mathbf{y}\rangle$ for all $\mathbf{x},\mathbf{y}$. Conclude that it preserves length and angle. State the complex (unitary) analog with the conjugate transpose, and explain why this is exactly the property §40.4 needs for quantum gates.

40.20. (A) The graph Laplacian $L=D-A$ is symmetric and positive semidefinite. Prove it is positive semidefinite by showing $\mathbf{x}^{\mathsf{T}}L\mathbf{x}=\sum_{(i,j)\in E}(x_i-x_j)^2$ (sum over edges). Use this to argue that the all-ones vector $\mathbf{1}$ is always an eigenvector with eigenvalue 0, and connect to the §40.4 claim that the number of zero eigenvalues counts connected components.

40.21. (C) Implement the randomized-SVD sketch of §40.4 as a function randomized_svd(A, k) (random Gaussian probe of width $k+5$ → QR → exact SVD of the small projection → return top-$k$ singular values). Test it on a $300\times 300$ matrix of true rank 12 and confirm the top 12 singular values match np.linalg.svd to several decimals. Then test it on a full-rank random matrix and observe that the approximation is no longer exact — explain why (Chapter 31's low-rank lesson).

40.22. (C) Implement a minimal sparse matrix–vector product: store a matrix as a dictionary {(i,j): value} of nonzeros and write spmv(sparse, x) returning the dense product. Verify against A @ x for a sparse random matrix. (This is the data structure real large-scale linear algebra runs on — and the first Build Your Toolkit option for this chapter.)

⭐⭐⭐⭐ Open-ended exploration (no answer key — go build something)

40.23. Pick your road. Re-read the §40.4 FAQ "which frontier should I learn next." Choose the road that matches your goals, find one introductory resource for it (use this chapter's further-reading list), and write a one-page plan: what you'll read, what you'll build, and which chapters of this book it relies on. The deliverable is your own roadmap.

40.24. The everywhere hunt. Over the next week, find linear algebra "in the wild" — in a paper, a library's documentation, a lecture, a news article about AI. For each sighting, identify which chapter's idea it is and which of the six themes it illustrates. Collect at least five. (This exercise is the entire point of the book in miniature: recognizing the familiar structure under unfamiliar surfaces.)

40.25. Extend the toolkit. Implement one of the three Build Your Toolkit options from §40.7 — a sparse matrix class, a tiny tensor class with contract, or a randomized SVD — and use it to redo one application from earlier in the book at larger scale (PageRank on a sparse graph, an image compression with randomized SVD, etc.). Verify against numpy throughout. Write a short note on where your from-scratch version diverges from the library and why (conditioning, Chapter 38).

40.26. Teach one theme. Choose the one recurring theme you understand best and write a 500-word explanation aimed at someone about to start Chapter 1, using an example from a field outside this book. Teaching a thing is the surest test that you own it.

40.27. The honest limits. This chapter was deliberately careful not to overclaim — it said tensors are "genuinely harder" than matrices and that practitioners' "tensors" are often just arrays. Pick one frontier and write a paragraph on what this book did not teach you about it, and what you'd need to learn next. Cultivating an accurate map of your own ignorance is a real mathematical skill.