Chapter 14 — Key Takeaways

The big ideas

  • There are exactly four fundamental subspaces, and together they organize all of linear algebra. Two live in the input space $\mathbb{R}^n$ — the row space $C(A^{\mathsf{T}})$ (span of the rows) and the null space $N(A)$ (solutions of $A\mathbf{x}=\mathbf{0}$) — and two live in the output space $\mathbb{R}^m$ — the column space $C(A)$ (span of the columns) and the left null space $N(A^{\mathsf{T}})$ (solutions of $A^{\mathsf{T}}\mathbf{y}=\mathbf{0}$, the dependence relations among the rows). The row space of $A$ is the column space of $A^{\mathsf{T}}$, which is why it earns that symbol. (Theme: the four fundamental subspaces are the organizing framework for all of linear algebra.)
  • Rank is one number that equals both the dimension of the column space and the dimension of the row space. Row rank $=$ column rank because both count the pivots of $A$, and a matrix has exactly one set of pivot positions. So $\operatorname{rank}(A) = \dim C(A) = \dim C(A^{\mathsf{T}}) = r$ — a single, well-defined invariant. (Theme: geometry and algebra are two views of one object — rank is at once a pivot count, an image dimension, and a measure of information content.)
  • The four dimensions are fixed by $m$, $n$, and $r$ alone: $\dim C(A) = r$, $\dim N(A) = n - r$, $\dim C(A^{\mathsf{T}}) = r$, $\dim N(A^{\mathsf{T}}) = m - r$. The input pair sums to $n$; the output pair sums to $m$; the rank $r$ appears twice, once in each space.
  • The Rank-Nullity Theorem: $\operatorname{rank}(A) + \dim N(A) = n$. The right-hand side is $n$, the number of columns (the input dimension) — never $m$, never the rank. It is a conservation law for dimension: every input direction is either used faithfully (rank) or destroyed (nullity), with none left over. The output-space version is $\operatorname{rank}(A) + \dim N(A^{\mathsf{T}}) = m$. The proof constructs the kernel basis (one vector per free variable), so rank-nullity and "set each free variable to 1 in turn" are the same fact.
  • Finding bases — the asymmetry to engrave. Row space: the nonzero rows of the RREF (row operations preserve the row space). Column space: the pivot columns of the original $A$ (row operations change the column space but preserve the dependence relations among columns). Null space: one vector per free column. Left null space: the row-dependence vectors (or solve $A^{\mathsf{T}}\mathbf{y}=\mathbf{0}$).
  • Rank classifies linear systems. Full row rank ($r=m$) ⟹ $A\mathbf{x}=\mathbf{b}$ solvable for every $\mathbf{b}$. Full column rank ($r=n$) ⟹ at most one solution (nullity $0$). Both ($r=m=n$) ⟹ unique solution always (square, invertible). When solvable, the solution set has dimension $n-r$. (Theme: computation validates theory — every dimension was confirmed with np.linalg.matrix_rank and scipy.linalg.null_space.)
  • The four subspaces meet at right angles (the Part IV preview). Row space ⟂ null space inside $\mathbb{R}^n$; column space ⟂ left null space inside $\mathbb{R}^m$. In each space the paired subspaces are orthogonal complements — which is why their dimensions tile the space perfectly.

Skills you gained

  • Finding a basis and the dimension for all four fundamental subspaces directly from a matrix's RREF.
  • Justifying that row rank equals column rank via pivot counting (and via the rank factorization $A = CR$).
  • Stating the rank-nullity theorem with its conditions and reconstructing its motivated proof by counting pivot columns against free columns.
  • Giving all four subspace dimensions in terms of $m$, $n$, $r$, and checking that the two pairs sum to $n$ and $m$.
  • Applying rank-nullity to predict solvability, uniqueness, and the dimension of a solution set from shape and rank alone.
  • Recognizing the two orthogonality relationships and verifying them numerically.
  • Implementing rank(A) from scratch and checking rank-nullity against numpy.

Terms to know

row space $C(A^{\mathsf{T}})$ · left null space $N(A^{\mathsf{T}})$ · four fundamental subspaces · rank (number of pivots = $\dim C(A) = \dim C(A^{\mathsf{T}})$) · row rank · column rank · nullity ($\dim N(A)$) · rank-nullity theorem ($\operatorname{rank}+\text{nullity}=n$) · pivot column / free column · reduced row echelon form (RREF) · rank factorization ($A=CR$) · full row rank / full column rank · rank deficient · orthogonal complement · degrees of freedom (residual d.f. $= n-p$ in regression) · mechanism / self-stress state (truss) · state of self-stress.

Notation reinforced

  • Four subspaces (LOCKED glyphs): $C(A)$ column, $N(A)$ null, $C(A^{\mathsf{T}})$ row, $N(A^{\mathsf{T}})$ left null.
  • Rank as a roman operator: $\operatorname{rank}(A) = r$.
  • Transpose always $A^{\mathsf{T}}$ (never $A^T$); the left null space is defined by $A^{\mathsf{T}}\mathbf{y} = \mathbf{0}$, equivalently $\mathbf{y}^{\mathsf{T}}A = \mathbf{0}$.
  • Dimensions: $\dim C(A)=r$, $\dim N(A)=n-r$, $\dim C(A^{\mathsf{T}})=r$, $\dim N(A^{\mathsf{T}})=m-r$.
  • numpy: np.linalg.matrix_rank(A), scipy.linalg.null_space(A) and null_space(A.T), sympy.Matrix(A).rref(); remember math is 1-indexed, numpy/sympy 0-indexed (pivot columns print as (0, 2) for the 1st and 3rd columns).

How this connects forward

  • Chapter 15 (Dimension, Basis, and Coordinates) makes rigorous the one fact this chapter quietly assumed every time it counted dimensions: that every basis of a subspace has the same size, so "dimension" is well-defined. The basis-extension lemma there is what powers the abstract rank-nullity proof.
  • Chapter 16 (Change of Basis) shows the same transformation wearing different matrices in different coordinate systems — the four subspaces are coordinate-free, but their bases depend on the coordinates.
  • Chapter 17 (Linear Regression) is Case Study 1 in full: least squares finds the projection of $\mathbf{b}$ onto the column space when $\mathbf{b}$ is unreachable, and the residual lives in the left null space.
  • Part IV (Orthogonality, Chapters 18–22) is where the right angles of §14.10 become rigorous and powerful: the dot product and angles (18), orthogonal projection onto a subspace and least squares (19), Gram-Schmidt and QR (20), and orthogonal matrices and rotations (21). The orthogonal-complement decomposition $\mathbb{R}^n = C(A^{\mathsf{T}}) \oplus N(A)$ is the seed of all of it.
  • Chapters 30–32 (SVD, low-rank approximation, PCA) are rank taken to its limit: the rank factorization $A = CR$ becomes the SVD, low-rank approximation is image compression and denoising, and PCA keeps only the few independent directions that carry the variance — exactly the dimensionality reduction reading of rank.
  • Chapter 38 (Numerical Linear Algebra) formalizes near-rank-deficiency via the condition number — what to do when columns are almost dependent.

The recurring themes, revisited

This chapter advanced three of the book's six themes. The four fundamental subspaces are the organizing framework for all of linear algebra — you now hold the complete diagram, with every dimension and both orthogonality relationships, and you saw it decide a regression's degrees of freedom, a circuit's loops, and a truss's stability. Geometry and algebra are two views of one object — rank is simultaneously a pivot count (algebra), the dimension of an image (geometry), and a measure of how much information a dataset contains (application). And computation validates theory — you proved row rank equals column rank and proved rank-nullity, then confirmed every dimension with numpy and built rank from scratch. Keep the four-subspaces diagram in front of you; the right angles you glimpsed here are about to become the most powerful tool in the book, and you will be drawing this picture for the rest of it.