Further Reading — Chapter 38: Numerical Linear Algebra

Numerical linear algebra is a deep field with three modern classics, listed first. Each entry notes where in the book to go and how it maps to our treatment.

The three standard references

  • Lloyd N. Trefethen & David Bau III, Numerical Linear Algebra (SIAM, 1997). The single best book to read after this chapter, and the closest in spirit to our geometry-first approach. Organized as 40 short "lectures." For this chapter: Lecture 12 (conditioning and condition numbers) is the rigorous version of Section 38.2; Lectures 14–15 (stability, backward stability) deepen Section 38.4; Lecture 16 (stability of back substitution) and Lecture 22 (stability of LU and the need for pivoting) cover Section 38.4's pivoting argument; Lectures 32–40 (iterative methods, Arnoldi, GMRES, conjugate gradient, Krylov subspaces) are the full version of Section 38.6. If you read one book on this subject, read this one — its treatment of $\kappa(A)$ and backward stability is unmatched for clarity.

  • Gene H. Golub & Charles F. Van Loan, Matrix Computations (4th ed., Johns Hopkins, 2013). The encyclopedic reference — the book practitioners keep on the shelf and consult for decades. Comprehensive and algorithm-focused. For this chapter: Chapter 2 (matrix analysis, norms, condition numbers), Chapter 3 (general linear systems, LU, pivoting, error analysis), Chapter 5 (orthogonalization and least squares — the QR-vs-normal-equations story of Section 38.5 in full rigor), and Chapters 10–11 (Krylov methods and conjugate gradient). More demanding than Trefethen–Bau; use it as a reference rather than a first read.

  • Nicholas J. Higham, Accuracy and Stability of Numerical Algorithms (2nd ed., SIAM, 2002). The definitive treatise on exactly the subject of this chapter: floating-point arithmetic and the rigorous error analysis of every standard algorithm. For this chapter: Chapter 1 (principles of finite-precision computation), Chapter 2 (floating-point arithmetic and the model $\text{fl}(x) = x(1+\delta)$), Chapters 7–9 (perturbation theory and the error analysis of linear systems, LU, and Cholesky), and Chapter 20 (least squares — the authoritative account of why normal equations square the conditioning). The most thorough source on backward-error analysis and the Wilkinson tradition cited in Section 38.4.

Mapping to the books we've used throughout

  • Gilbert Strang, Introduction to Linear Algebra (5th/6th ed.). Strang's Chapter on numerical linear algebra (the placement varies by edition; look for "Numerical Linear Algebra" or the sections on iterative methods and the SVD) gives an accessible first pass on conditioning, the singular-value view, and iterative methods, in his characteristic intuitive style. His treatment of $A = U\Sigma V^{\mathsf{T}}$ is the right warm-up for understanding $\kappa(A) = \sigma_{\max}/\sigma_{\min}$. The companion MIT OpenCourseWare 18.06 lectures (free video) include excellent sessions on the SVD and on solving large systems.

  • Sheldon Axler, Linear Algebra Done Right (4th ed.). Axler is deliberately not a numerical book — it avoids determinants and computation by design — so it has little to say about conditioning or floating point directly. Read it instead for the cleanest abstract foundations of the SVD and operator theory (his Chapter 7 on operators on inner-product spaces) that underlie the condition number conceptually. Pair it with Trefethen–Bau for the numerics it omits.

  • Stephen Boyd & Lieven Vandenberghe, Introduction to Applied Linear Algebra (VMLS) (Cambridge, 2018; free PDF). The most applied of our references, and excellent on least squares from the practitioner's angle. Its chapters on least squares and the QR factorization motivate exactly why one solves regression via QR rather than the normal equations — the applied complement to Section 38.5. Free online, with Julia/Python companion code.

Free online resources

  • The numpy and SciPy documentation for numpy.linalg.cond, numpy.linalg.lstsq, numpy.linalg.qr, numpy.linalg.svd, and scipy.sparse.linalg.cg / gmres. Reading the docstrings (and the "Notes" sections, which name the LAPACK routine each function calls) is the fastest way to connect this chapter's theory to the tools you actually use.

  • The LAPACK Users' Guide (free at netlib.org) documents the routines (*gesv, *gelsd, *geqrf, *gesdd, *syev) named in Section 38.7. The Quick Reference and the chapter on accuracy and stability show how the library reports condition-number estimates alongside solutions.

  • Cleve Moler, Numerical Computing with MATLAB (free at mathworks.com/moler). Moler — a co-founder of MATLAB and a co-author of the original LINPACK — writes beautifully on conditioning and the Hilbert matrix; his chapter on linear equations is a delightful, accessible companion to Section 38.3, and his "Cleve's Corner" essays on the condition number and floating point are gems.

  • The IEEE 754 standard (and David Goldberg's classic free article "What Every Computer Scientist Should Know About Floating-Point Arithmetic," ACM Computing Surveys, 1991) for the bit-level details behind Section 38.1.

Where to go next

This chapter closes the book's coverage of how to compute linear algebra reliably. To go further: a graduate course or self-study from Trefethen–Bau (theory) plus Golub–Van Loan (algorithms) is the standard path. For the iterative-methods world at scale, Yousef Saad's Iterative Methods for Sparse Linear Systems (2nd ed., free PDF) is the specialist's text on Krylov methods, GMRES, and preconditioning — the natural sequel to Section 38.6 for anyone who will solve genuinely enormous systems.