Case Study 2 — Sixty Frames a Second: The Linear Algebra Inside a Video Game
Right now, somewhere, a teenager is piloting a spaceship through an asteroid field. The ship banks, the camera swings, an asteroid tumbles past, and the whole scene redraws sixty times every second without a hitch. It feels like motion. Underneath, there is no motion at all — only arithmetic. Each of those sixty frames is built by multiplying hundreds of thousands of vectors by a handful of matrices. This case study opens up a single frame and shows that the smooth, organic-feeling world of a 3D game is, at its computational core, the transformation viewpoint of Chapter 1 running at enormous speed. We will stay in 2D for clarity, but every idea scales straight to the 3D you actually play in.
A shape is a list of vectors
Begin with the spaceship. To a graphics engine, the ship is not a picture; it is a mesh — a list of points in space, called vertices, connected into triangles. A simple ship might have a few hundred vertices; a detailed character in a modern game has tens of thousands. Each vertex is a vector: a tip might sit at $(1, 0)$, a wing corner at $(-0.5, 0.4)$, and so on. The entire ship is just a bundle of these arrows, all anchored to a common origin at the ship's center.
This is the same opening move as the recommender in the first case study, and the same opening move as all of applied linear algebra: the object of interest becomes a collection of vectors. But where the recommender treated vectors as abstract coordinates of taste, here the vectors are literally positions in space — which means the transformations that act on them are the geometric ones we visualized in Chapter 1: rotations, scalings, shears, and (with a trick) translations. Graphics is the most physically literal home of the transformation viewpoint, which is exactly why it is such a good place to see it.
Moving the ship is multiplying by a matrix
The player pushes the stick and the ship rotates. To the engine, "rotate the ship 90° counterclockwise" means: take every vertex vector and multiply it by the rotation matrix from Figure 1.3,
$$R = \begin{bmatrix} 0 & -1 \\ 1 & 0 \end{bmatrix}.$$
The vertex at the ship's tip, $(1, 0)$, moves to $R(1,0) = (0, 1)$ — the tip now points up instead of right. Every other vertex turns by the same quarter-turn, and because the same matrix hits every vertex, the ship rotates rigidly, keeping its shape. That rigidity is guaranteed by a fact we noted in Chapter 1: a rotation has determinant 1, so it preserves every length and area. The ship turns without stretching or distorting — which is precisely what you want a spaceship to do.
Now suppose the ship also flies toward you, growing on screen. That's a scaling. Apply a uniform scale-by-two,
$$S = \begin{bmatrix} 2 & 0 \\ 0 & 2 \end{bmatrix},$$
after the rotation. The tip, already moved to $(0, 1)$ by the rotation, is sent by $S$ to $(0, 2)$.
# One vertex: rotate 90 deg, then scale by 2.
import numpy as np
v = np.array([1.0, 0.0]) # the ship's nose
R = np.array([[0.0, -1.0], [1.0, 0.0]]) # rotate 90 deg CCW
S = np.array([[2.0, 0.0], [0.0, 2.0]]) # uniform scale x2
print(R @ v) # [0. 1.] after rotation
print(S @ (R @ v)) # [0. 2.] after rotation, then scaling
The nose ends at $(0, 2)$, just as predicted. Apply that same rotate-then-scale to all the vertices and the ship spins and looms toward the camera. Notice we performed two transformations in a row — first $R$, then $S$. The engine could instead combine them once into a single matrix $SR$ and apply that one matrix to every vertex, doing the work in a single multiplication. Combining transformations into one matrix is composition, and that it corresponds to multiplying the two matrices together is the central revelation of Chapter 8. For now, hold the picture: stacking transformations means stacking matrices.
Why the order matters (and a frame's worth of matrices)
Try the operations in the other order — scale first, then rotate — and, for a uniform scale and a rotation, you happen to land in the same place. But that is a coincidence of this particular pair. In general, transformations do not commute: rotating then shearing is not the same as shearing then rotating, just as putting on socks then shoes is not the same as shoes then socks. This non-commutativity is one of the first surprises of matrix algebra (Chapter 8), and graphics programmers learn to respect it early, because a bug in transformation order produces a character whose limbs fly off in the wrong direction.
A real frame applies a whole pipeline of matrices to every vertex, in a fixed order:
- a model matrix that places and orients the ship in the world (rotation + scale + position);
- a view matrix that re-expresses everything from the camera's point of view (the world's "north" becomes the camera's "forward" — a change of basis, Chapter 16);
- a projection matrix that flattens the 3D scene onto your 2D screen (a projection, the cousin of Figure 1.5 that crushes a dimension).
Every visible vertex is multiplied through all three, every frame. There is one wrinkle the chapter flagged: moving the ship to a new position is a translation, and translations are not linear — they shift the origin, failing the test from Section 1.3. Graphics solves this with a beautiful hack called homogeneous coordinates: add a third coordinate (a "1") to every 2D vector, work with $3\times 3$ matrices, and a translation becomes expressible as a matrix multiplication after all. That trick is the whole subject of Chapter 12, and it is the reason the entire pipeline can be a uniform chain of matrix multiplications even though one of the operations is secretly affine.
The hardware is built for this
Multiply the work out. A scene with 200,000 visible vertices, three matrices in the pipeline, sixty frames per second: that is on the order of tens of millions of matrix–vector multiplications every second, just to move the geometry — before a single pixel is shaded. No general-purpose processor doing one operation at a time could keep up. The reason games run at all is the graphics processing unit (GPU), a chip with thousands of small cores designed to perform the same matrix multiplication on thousands of different vertices simultaneously. The GPU is, quite literally, a machine forged to execute the transformation viewpoint of Chapter 1 in parallel.
And here is the connection that closes the loop with the first case study. That same massively parallel matrix-multiplication hardware turned out to be exactly what training neural networks needs — because, as Section 1.6 explained, a neural network is also a chain of matrix multiplications. The GPUs built to spin spaceships are the GPUs now training the world's AI models. Two industries, the same linear algebra, the same silicon.
What you should take away
A video game frame looks like art and feels like motion, but mechanically it is the purest expression of this book's central claim: a matrix is a function that transforms space. The ship's spin is a rotation matrix; its approach is a scaling matrix; the camera is a change of basis; the screen is a projection. Stack them, multiply every vertex through, and do it sixty times a second. When you reach Chapter 12 you will build this pipeline yourself, and when you build the capstone in Chapter 39 you may choose to render a small 3D scene from scratch — at which point the asteroid field will no longer be magic. It will be matrices, multiplying vectors, exactly as you will have learned to see them here.
Forward references: matrix multiplication as composition and non-commutativity (Chapter 8); the determinant as area/volume scaling (Chapter 11); the full graphics pipeline and homogeneous coordinates (Chapter 12); change of basis (Chapter 16); rotations and orthogonal matrices (Chapter 21); a renderer as a capstone option (Chapter 39).