Chapter 22 — Key Takeaways

The one idea

A Fourier coefficient is an orthogonal projection. Everything in this chapter is the projection of Chapter 19 and the orthonormal-basis coordinates of Chapter 20, carried out in a space whose vectors are functions and whose inner product is an integral. Decomposing a signal into frequencies is reading off coordinates in a perpendicular basis; reconstructing it is summing projections back up. If you understand projection onto an orthonormal basis, you understand Fourier analysis — the rest is recognizing the costume.

The big ideas, in order

  • Functions are vectors. The functions on $[-\pi,\pi]$ form an inner product space under $\langle f,g\rangle = \int_{-\pi}^{\pi} fg\,dx$. This inner product is symmetric, linear, and positive — the three properties from Chapter 18 — so it supplies length $\lVert f\rVert = \sqrt{\langle f,f\rangle}$, angle, and orthogonality exactly as in $\mathbb{R}^n$. The space is infinite-dimensional, the one genuinely new feature.
  • Sines and cosines are an orthogonal basis. The load-bearing facts: $\langle \sin kx,\sin mx\rangle$ and $\langle \cos kx,\cos mx\rangle$ equal $\pi$ when $k=m$ and $0$ otherwise; every sine is orthogonal to every cosine; the constant $1$ is orthogonal to all of them with $\lVert 1\rVert^2 = 2\pi$. These arrive pre-orthogonal — no Gram–Schmidt required.
  • Coefficients are projections. $a_k = \tfrac1\pi\langle f,\cos kx\rangle$, $b_k = \tfrac1\pi\langle f,\sin kx\rangle$, $a_0 = \tfrac1{2\pi}\langle f,1\rangle$. The $1/\pi$ and $1/(2\pi)$ are the $1/\lVert\cdot\rVert^2$ factors of the Chapter 19 projection formula.
  • Orthogonality makes the coefficients independent. Each coefficient is one integral, blind to every other frequency. Add a harmonic and the coefficients you already have do not change. This is recurring theme #4 — the same projection idea, a new space — and it is the entire reason sines and cosines are the right basis for periodic signals.
  • Truncation is best approximation. The partial sum $S_N$ is the orthogonal projection of $f$ onto the first $N$ harmonics, hence the closest point to $f$ in the norm (Chapter 19). Truncating a Fourier series is least-squares approximation — the function-space, low-rank-in-spirit cousin of what the SVD will do in Chapter 30.
  • Convergence is in energy; Gibbs is the gap. The series converges to $f$ in mean square ($\lVert f - S_N\rVert \to 0$) and pointwise wherever $f$ is continuous; at a jump it converges to the jump's midpoint. The Gibbs overshoot ($\approx 9\%$ of the jump) never shrinks in height, only in width — the gap between mean-square and pointwise convergence.
  • Parseval = Pythagoras in function space. $\lVert f\rVert^2 = 2\pi a_0^2 + \pi\sum(a_k^2+b_k^2)$: total energy is the sum of energies across frequencies. This is why "keep the big coefficients" is sound compression — in an orthogonal basis, energy is additive and discarding a small coefficient costs only its small energy.

The worked anchor

The square wave decomposes into odd-harmonic sines with $b_k = 4/(\pi k)$ (all cosines and even sines vanish by symmetry). A single sine captures $81\%$ of its energy; five terms reach $96\%$; fifty terms exceed $99.5\%$. The partial sums visibly converge while the Gibbs spike at each jump holds steady at $\approx 1.179$. Verified by numerical projection and by np.fft, both matching the hand computation.

Skills you gained

  • Treat a function as a vector and compute its norm and inner products as integrals.
  • State and verify the orthogonality relations among sines, cosines, and the constant.
  • Compute a Fourier coefficient as a projection — by hand, by numerical integration (np.trapezoid), and via the FFT.
  • Reconstruct a signal from truncated coefficients and reason about convergence, energy capture, and Gibbs ringing.
  • Recognize the same projection-and-truncate structure in MP3 audio and JPEG image compression.

Terms to know

Fourier series · Fourier coefficient · orthogonal basis (of functions) · function space / inner product space · square wave · partial sum · truncation · Gibbs phenomenon · fundamental frequency · harmonic · Parseval's identity · spectrum · complex exponential basis · discrete Fourier transform (DFT) · mean-square convergence.

Where this is going

This chapter closes Part IV by showing orthogonal projection at its most powerful, and it points straight into the heart of the book.

  • Eigenvalues and eigenvectors (Part V, starting Chapter 23). Sines and cosines are the eigenfunctions of the second-derivative operator: $\tfrac{d^2}{dx^2}\sin kx = -k^2\sin kx$. Expanding a signal in the Fourier basis is expanding it in eigenvectors — your first eigen-decomposition. Part V makes the eigenvector viewpoint the center of everything, and Chapter 27's Spectral Theorem will show that a symmetric matrix has an orthogonal eigenbasis, the same orthogonality-decouples-everything story you saw here.
  • The SVD and low-rank approximation (Chapters 30–31). Truncating a Fourier series — keeping the components that carry the energy, discarding the rest — is the spirit of the singular value decomposition. The SVD expands any matrix in an orthogonal basis ordered by importance, and keeping the top few yields the best low-rank approximation (a blurry-then-sharpening image in Chapter 31). Fourier compression is the first taste; the SVD makes it universal.
  • Inner product spaces, abstractly (Chapter 34). The space of functions we worked in is the Hilbert space $L^2$; Chapter 34 develops abstract inner product spaces and revisits completeness — the deep theorem that the Fourier basis leaves no direction unaccounted for.

The right angle you learned as a child turns out to organize the spectrum of a sound, the structure of an image, and — next — the essential action of every matrix.