Further Reading — Chapter 20: Gram-Schmidt and QR Decomposition

Annotated pointers for going deeper, mapped to the standard texts. Read the chapter first; these expand and challenge it.

Core textbooks

  • Gilbert Strang, Introduction to Linear Algebra (6th ed.), §4.4 "Orthonormal Bases and Gram-Schmidt." The closest match to this chapter's spirit and the source of the geometry-first, "subtract the projection" framing. Strang derives $A = QR$ as the matrix form of Gram-Schmidt and connects it immediately to least squares. His MIT 18.06 video lecture on Gram-Schmidt is an excellent companion and is freely available on MIT OpenCourseWare. Start here.

  • Lloyd N. Trefethen and David Bau III, Numerical Linear Algebra, Lectures 7, 8, and 10. The reference for the numerical side of this chapter, and essential if you care about computation. Lecture 7 presents QR factorization geometrically; Lecture 8 is a careful treatment of classical vs. modified Gram-Schmidt and exactly why classical loses orthogonality — the rigorous version of our §20.11 Warning. Lecture 10 introduces Householder triangularization, the stable algorithm numpy actually uses instead of Gram-Schmidt. Written with unusual clarity; the natural next step after Strang.

  • Sheldon Axler, Linear Algebra Done Right (4th ed.), §6.B "Orthonormal Bases." The abstract, coordinate-free treatment. Axler proves the Gram-Schmidt procedure for any inner product space (not just $\mathbb{R}^n$) and derives the existence of orthonormal bases as a clean theorem. Best for math majors who want the version that transfers verbatim to function spaces (Chapter 22, Chapter 34). Light on computation by design.

  • Stephen Boyd and Lieven Vandenberghe, Introduction to Applied Linear Algebra (VMLS), Chapters 5 and 12. Application-first and free online. Chapter 5 covers Gram-Schmidt and orthonormal sets with a strong algorithmic flavor; Chapter 12 develops least squares and explicitly uses QR for the solution. Excellent for CS / data science readers who want to see the methods deployed on real problems with minimal abstraction.

On numerical stability (the §20.11 Warning, in depth)

  • Trefethen and Bau, Lecture 9 ("MATLAB"/experiments) and Lecture 16 (stability of least squares). Quantifies how QR-based least squares avoids the $\kappa^2$ conditioning penalty of the normal equations — the rigorous backing for Case Study 1. The experiments are worth reproducing in numpy.

  • Nicholas J. Higham, Accuracy and Stability of Numerical Algorithms (2nd ed.), Chapter 19. The definitive, encyclopedic reference on the numerical behavior of QR and Gram-Schmidt, including the precise bounds on loss of orthogonality. Advanced; consult it when you need the theorem, not the intuition.

Historical and primary sources

  • Erhard Schmidt (1907), "Zur Theorie der linearen und nichtlinearen Integralgleichungen." The paper where the orthogonalization process appears in its modern form, in the setting of integral equations — note that Schmidt's "vectors" are functions, foreshadowing Chapter 22. The attribution to Gram (1883) and Schmidt, with earlier appearances in Laplace and Cauchy, is discussed in most histories of numerical linear algebra [verify].

  • J. G. F. Francis (1961–62), "The QR Transformation, Parts I and II," The Computer Journal; and V. N. Kublanovskaya (1961). The independent origins of the QR algorithm for eigenvalues previewed in §20.10, routinely listed among the most important algorithms of the twentieth century [verify]. Historical interest more than a first read; revisit alongside Chapters 24 and 29.

Free online resources

  • MIT OpenCourseWare 18.06 (Strang). Full video lectures, problem sets, and exams; the Gram-Schmidt and QR lectures pair directly with this chapter.
  • 3Blue1Brown, Essence of Linear Algebra (YouTube). No episode is dedicated to Gram-Schmidt, but the projection and change-of-basis episodes build the geometric intuition this chapter extends.
  • numpy / scipy documentation: numpy.linalg.qr (note the mode argument: 'reduced' vs. 'complete') and scipy.linalg.qr (which exposes more options, including pivoting). Read these before relying on the defaults — especially the sign convention, which neither library pins down.

Where to go next in this book

If the function-space remark in the Math-Major Sidebar intrigued you, jump ahead to Chapter 22 (Fourier series as projection) and Chapter 34 (inner product spaces). If you want to see QR iterated into an eigenvalue algorithm, Chapters 24 and 29. For the full numerical-stability story, Chapter 38.