Chapter 13 Exercises — Column Space and Null Space

How to use these. Work the ⭐ problems first to lock in the concepts (no computation needed). The ⭐⭐ problems are by-hand calculations — finding bases from RREF, solvability checks, complete solutions. The ⭐⭐⭐ problems split into proofs (the A track) and coding with numpy/scipy (the C track); do the ones that match your path, but the strongest students do both. The ⭐⭐⭐⭐ problems are applied: find the column-space / null-space structure hiding in a real system. Tags: [hand] = pencil only, [code] = needs numpy, [proof] = rigorous argument, [essay] = written explanation. Notation is locked: $C(A)$ for the column space, $N(A)$ for the null space.

Tier ⭐ — Conceptual (what is / why)

13.1 [hand] In one sentence each, define the column space $C(A)$ and the null space $N(A)$ of an $m \times n$ matrix. For each, state which space it lives in ($\mathbb{R}^m$ or $\mathbb{R}^n$) and why.

13.2 [hand] Complete the sentence and explain why it is true: "The system $A\mathbf{x} = \mathbf{b}$ has a solution if and only if $\mathbf{b}$ is in ______." What goes in the blank, and what is the one-line reason?

13.3 [hand] A matrix $A$ is $4 \times 6$. (a) What is the largest possible dimension of $C(A)$? (b) What is the smallest possible dimension of $N(A)$? (c) Explain why a $4 \times 6$ matrix can never have a trivial null space.

13.4 [hand] When you find a basis for $C(A)$ from the RREF $R$, you take the pivot columns of the original $A$, not of $R$. State the reason in one sentence, and say which subspace the columns of $R$ are a basis for instead (you may peek ahead to Chapter 14).

13.5 [essay] Explain in your own words why the null space "controls the number of solutions" of $A\mathbf{x} = \mathbf{b}$. Address all three cases: no solution, exactly one, infinitely many.

13.6 [hand] True or false, with a one-line reason for each: (a) If $N(A) = \{\mathbf{0}\}$, the columns of $A$ are linearly independent. (b) If $A$ is $5 \times 3$, then $C(A) = \mathbb{R}^5$ is possible. (c) Every special solution lies in $N(A)$. (d) The zero vector is in both $C(A)$ and $N(A)$.

13.7 [hand] A square matrix $A$ is singular ($\det A = 0$). What does this tell you, separately, about $C(A)$ and about $N(A)$? Why do the two defects always arrive together for a square matrix?

13.8 [essay] Restate the criterion "$\mathbf{b} \in C(A) \iff A\mathbf{x} = \mathbf{b}$ is solvable" in terms of the column picture of Chapter 3. Why does the column picture make this criterion obvious?

Tier ⭐⭐ — Computation by hand

13.9 [hand] For $A = \begin{bmatrix} 1 & 2 \\ 2 & 4 \end{bmatrix}$: (a) row-reduce to RREF; (b) give a basis for $C(A)$ and state what geometric object it is; (c) give a basis for $N(A)$ by finding the special solution; (d) confirm $\dim C(A) + \dim N(A) = 2$.

13.10 [hand] For $A = \begin{bmatrix} 1 & 0 & 2 \\ 0 & 1 & 3 \end{bmatrix}$: identify pivot and free columns, give bases for $C(A)$ and $N(A)$, and state the dimension of each. (Answer check: $N(A)$ is 1-dimensional.)

13.11 [hand] For $A = \begin{bmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{bmatrix}$: (a) row-reduce and find the rank; (b) give a basis for $C(A)$; (c) find the special solution(s) spanning $N(A)$; (d) read off the dependence relation among the columns that the special solution encodes.

13.12 [hand] Is $\mathbf{b} = (1, 1, 1)$ in the column space of $A = \begin{bmatrix} 1 & 1 \\ 0 & 1 \\ 1 & 0 \end{bmatrix}$? Reduce $[A \mid \mathbf{b}]$ by hand and identify the contradiction row if there is one. Then repeat for $\mathbf{b}' = (2, 1, 1)$ and give the recipe if it is reachable.

13.13 [hand] For the matrix $A = \begin{bmatrix} 1 & 2 & 1 & 1 \\ 2 & 4 & 3 & 4 \end{bmatrix}$: find the RREF, give a basis for $C(A)$ (which columns?), and find both special solutions spanning $N(A)$. (Answer check: pivots in columns 1 and 3; two special solutions.)

13.14 [hand] Take the $A$ from Exercise 13.13 and $\mathbf{b} = (4, 9)$. Find one particular solution by setting the free variables to zero, then write the complete solution as $\mathbf{x}_p + c_1\mathbf{s}_1 + c_2\mathbf{s}_2$.

13.15 [hand] A matrix row-reduces to $R = \begin{bmatrix} 1 & 0 & -2 & 0 \\ 0 & 1 & 5 & 0 \\ 0 & 0 & 0 & 1 \end{bmatrix}$. Without knowing the original matrix, state: (a) the rank; (b) which variables are free; (c) the dimension of the null space; (d) the single special solution that spans $N(A)$.

13.16 [hand] For each, decide whether $A\mathbf{x} = \mathbf{b}$ has no solution, a unique solution, or infinitely many, using only the given facts. (a) $A$ is $3 \times 3$, $\det A = 5$. (b) $A$ is $3 \times 3$ with rank $2$, and $\mathbf{b} \in C(A)$. (c) $A$ is $2 \times 4$ with rank $2$, and $\mathbf{b} \in C(A)$. (d) $A$ is $4 \times 2$ with rank $2$, and $\mathbf{b} \notin C(A)$.

13.17 [hand] Construct a $3 \times 3$ matrix whose column space is exactly the plane $\{(x, y, z) : z = 0\}$ and whose null space is exactly the $z$-axis. (Hint: a projection.) Verify both claims by hand.

Tier ⭐⭐⭐ — Proof (A) and Coding (C)

13.18 [proof] Prove directly from the definition that $N(A)$ is a subspace of $\mathbb{R}^n$: show it contains $\mathbf{0}$ and is closed under addition and scalar multiplication. State at which step you use the linearity of $A$.

13.19 [proof] Prove the complete-solution theorem: if $\mathbf{x}_p$ solves $A\mathbf{x} = \mathbf{b}$, then the full solution set is $\{\mathbf{x}_p + \mathbf{x}_n : \mathbf{x}_n \in N(A)\}$. (Show both inclusions.)

13.20 [proof] Prove that $N(A) = \{\mathbf{0}\}$ if and only if the columns of $A$ are linearly independent. (Use the fact that $A\mathbf{x}$ is a linear combination of the columns with weights $\mathbf{x}$.)

13.21 [proof] Let $A$ be $m \times n$ with $n > m$ (a wide matrix). Prove that $N(A)$ must be nontrivial, i.e. contains a nonzero vector. (Hint: count pivots versus columns; you may cite that the rank is at most $m$.)

13.22 [code] Implement column_space_basis(A) and null_space_basis(A) from scratch, reusing Chapter 4's row_reduce(A) (no numpy inside the implementations). Then verify on three matrices of your choice that len(column_space_basis(A)) == np.linalg.matrix_rank(A), that len(null_space_basis(A)) == scipy.linalg.null_space(A).shape[1], and that every special solution s satisfies np.allclose(A @ s, 0).

13.23 [code] Write a function is_solvable(A, b) that returns True iff $\mathbf{b} \in C(A)$, by comparing np.linalg.matrix_rank(A) with np.linalg.matrix_rank(np.column_stack([A, b])). Test it on the chapter's matrix with $\mathbf{b} = (1,5,5)$ (should be True) and $\mathbf{b} = (4,11,1)$ (should be False).

13.24 [code] For a random $3 \times 5$ matrix A = np.random.randn(3, 5), print matrix_rank(A) and null_space(A).shape[1], and confirm they sum to $5$. Repeat for ten random matrices. Explain in a comment why the sum is always $5$ (and why the rank is almost always $3$ for a random matrix).

Tier ⭐⭐⭐⭐ — Applications

13.25 [code][essay] Chemical balancing as a null space. Build the $3 \times 4$ atom-conservation matrix for the combustion $a\,\text{CH}_4 + b\,\text{O}_2 \to c\,\text{CO}_2 + d\,\text{H}_2\text{O}$ (rows = C, H, O; products carry negative signs). Compute its null space with scipy.linalg.null_space, scale the result to the smallest positive integers, and confirm you recover $1, 2, 1, 2$. Explain why the balanced equation is exactly a null-space vector. (See Case Study 1.)

13.26 [code][essay] Reachable states in control. A discrete system $\mathbf{x}_{k+1} = A\mathbf{x}_k + B\mathbf{u}_k$ has $A = \begin{bmatrix} 1 & 1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{bmatrix}$, $B = \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}$. Form the controllability matrix $\mathcal{C} = [\,B \ \ AB \ \ A^2B\,]$, compute its rank, and describe $C(\mathcal{C})$ geometrically. Which state can the input never affect, and how does that show up as a vector outside the column space? (See Case Study 2.)

13.27 [essay] Redundant features. A design matrix has columns age, age_in_months (= 12·age), and income. Without computing, state the rank, the dimension of the null space, and write down a vector in $N(X)$ that exhibits the redundancy. Why does dropping age_in_months leave the column space — and therefore everything a linear model can express — unchanged? Connect to the chapter's link on feature spaces in machine learning.

13.28 [code] Projection's null space. Build the matrix that projects $\mathbb{R}^3$ onto the $xy$-plane, $P = \begin{bmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{bmatrix}$. Find bases for $C(P)$ and $N(P)$ from its RREF, verify with numpy, and confirm geometrically that $C(P)$ is the $xy$-plane and $N(P)$ is the $z$-axis. Then check that $P^2 = P$ (projections are idempotent) and explain what that means for any vector already in $C(P)$.