Case Study 2 — A Defective Pipeline: Why Cohorts Pile Up Before They Drain
Field: data science / operations analytics (customer-lifecycle and queueing models). This case study cashes in §36.7's power formula in a discrete-time setting: a two-stage pipeline whose stages decay at the same rate is governed by a defective transition matrix, and the Jordan block forces a $m\,\lambda^m$ — polynomial-times-geometric — term into its powers. That term is why a cohort of users "piles up" in the second stage before draining, a pattern a diagonalizable model can never reproduce.
The problem: tracking a cohort through a two-stage funnel
A SaaS company onboards new users through a simple pipeline. A user starts in Stage A ("activated" — signed up, exploring) and may progress to Stage B ("engaged" — using the core feature regularly), after which they eventually churn out of the tracked cohort entirely. The growth team wants to model, for a cohort of new sign-ups, how the populations of Stage A and Stage B evolve week by week — when Stage B peaks, how long the cohort stays engaged, and how the timing depends on the transition rates. This is a discrete-time linear dynamical system, the cousin of the PageRank Markov chains from Chapter 29 and the population models of earlier chapters, and it is governed by a transition matrix whose powers tell the whole story.
Let $\mathbf{x}_m = (a_m, b_m)$ be the cohort's populations in Stages A and B at week $m$. Each week, a fraction of Stage A users retain in A, a fraction progress from A into B, and a fraction of Stage B users retain in B (the rest churn). Write $r_A$ for A's retention, $p$ for the A-to-B progression rate, and $r_B$ for B's retention. The update rule $\mathbf{x}_{m+1} = T\mathbf{x}_m$ uses the transition matrix
$$ T = \begin{bmatrix} r_A & 0 \\ p & r_B \end{bmatrix}, $$
read column-by-column as "where this week's Stage-A user goes" (stays in A with weight $r_A$, moves to B with weight $p$) and "where this week's Stage-B user goes" (stays in B with weight $r_B$; the missing weight $1 - r_B$ is churn out of the cohort). After $m$ weeks the cohort is $\mathbf{x}_m = T^m \mathbf{x}_0$ — so understanding the cohort means understanding powers of $T$, which is exactly where the Jordan form lives.
When the two stages share a rate, the matrix is defective
For most rate choices, $T$ is harmless. Its eigenvalues are the diagonal entries $r_A$ and $r_B$ (it is triangular, so the diagonal is the spectrum, per Chapter 24). When $r_A \neq r_B$ the eigenvalues are distinct, $T$ is diagonalizable, and each stage's population is a clean blend of the pure geometric decays $r_A^m$ and $r_B^m$ — ordinary, well-behaved, no surprises.
The interesting case — and the one this chapter is about — is when the two stages decay at the same rate, $r_A = r_B = r$. This is not a contrived edge case: it is exactly what happens when both stages are governed by the same underlying churn mechanism (say, a subscription that lapses at a fixed rate regardless of engagement stage), so that the only difference between the stages is the one-way flow from A to B. Set $r_A = r_B = r$ and $p$ for the progression; with $r = 0.6$ and $p = 0.6$ for concreteness,
$$ T = \begin{bmatrix} 0.6 & 0 \\ 0.6 & 0.6 \end{bmatrix}. $$
Now $\lambda = 0.6$ is a repeated eigenvalue (algebraic multiplicity $2$). Count eigenvectors:
$$ T - 0.6\,I = \begin{bmatrix} 0 & 0 \\ 0.6 & 0 \end{bmatrix}, $$
which has rank $1$, so the eigenspace has dimension $2 - 1 = 1$: geometric multiplicity $1$, short of the algebraic $2$. The equal-rate pipeline matrix is defective. Its single eigenvector solves $0.6\,v_1 = 0$, i.e. $v_1 = 0$, giving $\mathbf{v} = (0, 1)$ — the "pure Stage B" direction. The progression coupling has consumed the second eigenvector, exactly as the shear did in §36.1.
# Equal-rate two-stage pipeline: the transition matrix is defective.
import numpy as np
r, p = 0.6, 0.6
T = np.array([[r, 0.],
[p, r ]])
print("eigenvalues:", np.linalg.eigvals(T)) # [0.6 0.6] (repeated)
print("geometric multiplicity:", 2 - np.linalg.matrix_rank(T - r*np.eye(2))) # 1 -> defective
To find the Jordan form, build a chain: solve $(T - 0.6I)\mathbf{w} = \mathbf{v}$ for the generalized eigenvector. The only nontrivial equation is $0.6\,w_1 = 0\cdot$ ... — concretely $0.6\,w_1 = 1$ (the second component of $\mathbf{v}$), giving $w_1 = 1/0.6$, so $\mathbf{w} = (1/0.6,\, 0)$. With $P = [\mathbf{v}\mid\mathbf{w}]$ we get the promised $J = \begin{bmatrix}0.6 & 1\\0 & 0.6\end{bmatrix}$ — verify with numpy:
# Build the Jordan chain and confirm J = P^{-1} T P.
import numpy as np
r = 0.6
T = np.array([[r, 0.], [r, r]])
v = np.array([0., 1.]) # eigenvector ("pure Stage B")
w = np.array([1/r, 0.]) # generalized: (T - rI) w = v
print("(T - rI) w == v :", np.allclose((T - r*np.eye(2)) @ w, v)) # True
P = np.column_stack([v, w])
print("J =\n", np.round(np.linalg.inv(P) @ T @ P, 8)) # [[0.6, 1.], [0., 0.6]]
The pile-up: a polynomial-times-geometric term
Now the payoff. The cohort starts as $100$ fresh sign-ups, all in Stage A: $\mathbf{x}_0 = (100, 0)$. The Stage-B population at week $m$ is the bottom entry of $T^m \mathbf{x}_0$, which is $100$ times the $(2,1)$ entry of $T^m$. Using the Jordan-block power formula $J^m = \begin{bmatrix}\lambda^m & m\lambda^{m-1}\\0&\lambda^m\end{bmatrix}$ and pulling back through $P$, that entry works out to a closed form:
$$ b_m = 100\,(T^m)_{21} = 100\,m\,r^{m}. $$
There it is — the defective signature in a business metric. The Stage-B population is not a pure geometric decay $r^m$; it is $r^m$ multiplied by a polynomial in $m$ (here, by $m$ itself). The factor of $m$ is the generalized eigenvector making itself felt, the discrete-time analogue of the $t\,e^{\lambda t}$ creep from Case Study 1. Let us watch the cohort and confirm the closed form:
# Cohort of 100 fresh Stage-A users: track Stage B, vs the closed form 100 m r^m.
import numpy as np
r = 0.6
T = np.array([[r, 0.], [r, r]])
x0 = np.array([100., 0.])
for m in range(1, 7):
b_num = (np.linalg.matrix_power(T, m) @ x0)[1] # Stage B from the model
b_cf = 100 * m * r**m # closed form 100 m r^m
print(f"week {m}: Stage B = {b_num:.4f} (closed form {b_cf:.4f})")
# week 1: Stage B = 60.0000 (closed form 60.0000)
# week 2: Stage B = 72.0000 (closed form 72.0000)
# week 3: Stage B = 64.8000 (closed form 64.8000)
# week 4: Stage B = 51.8400 (closed form 51.8400)
# week 5: Stage B = 38.8800 (closed form 38.8800)
# week 6: Stage B = 27.9936 (closed form 27.9936)
The match is exact. And read the numbers: Stage B starts at $0$, climbs to $60$ in week 1, peaks at $72$ in week 2, then declines — $64.8$, $51.8$, $38.9$, and onward to zero. The population rises before it falls. That hump is the cohort "piling up" in Stage B: new users flow in from Stage A faster than the existing Stage-B users churn out, until the influx from the depleting Stage A can no longer keep up and the geometric decay takes over.
Why the hump is impossible without defectiveness. The function $b_m = 100\,m\,r^m$ rises then falls precisely because of the polynomial factor $m$. A diagonalizable pipeline — distinct rates $r_A \neq r_B$ — gives Stage B a population of the form $\alpha r_A^m + \beta r_B^m$, a sum of two pure decays, which (for these sign patterns) decreases monotonically from its start with no interior peak. The interior maximum — the pile-up — is a direct fingerprint of the repeated eigenvalue and its Jordan block. When you see a cohort metric rise to a hump and fall, and the two pipeline stages share a churn rate, you are looking at a defective transition matrix in the wild.
The peak's timing is itself a Jordan-form fact. Maximizing $m\,r^m$ over $m$ (treating it as continuous, differentiating, and setting to zero) gives the peak at $m^\* = -1/\ln r = -1/\ln 0.6 \approx 1.96$ — so the integer peak is week $2$, exactly as the simulation shows. An analyst armed with the Jordan structure can therefore predict when engagement peaks directly from the retention rate, without running the simulation.
What the model tells the business, and the honest caveat
The defective structure carries an actionable message. Because Stage B follows $100\,m\,r^m$, the area under the curve — total user-weeks of engagement — and the timing of the peak depend on $r$ in a way that pure-decay intuition gets wrong. Raising retention $r$ pushes the peak later and fattens the whole curve (more cumulative engagement), while the polynomial factor means early-week engagement is higher than a naive single-rate decay model would predict. A growth team using a (wrong) diagonalizable approximation would under-forecast the week-2 pile-up and mis-time interventions like a "week 3 re-engagement email." The Jordan term is not a mathematical nicety; it is a measurable feature of the cohort curve.
The honest caveat (forward to Chapter 38). As §36.8 stressed, exact defectiveness is fragile. In reality $r_A$ and $r_B$ are estimated from noisy data and will essentially never be exactly equal, so the fitted transition matrix is, strictly, diagonalizable with two very close eigenvalues. But when $r_A \approx r_B$, the matrix is near-defective, and the cohort curve still shows the pile-up hump and still behaves like $m\,r^m$ over the weeks that matter — the diagonalizable formula with nearly-equal rates is numerically a near-copy of the defective one. This is the practical lesson of the whole chapter restated for data: you rarely compute a Jordan form from real data, but the Jordan structure explains the shape you observe, and reasoning with the clean defective model gives the right intuition for the messy near-defective reality. The exact $m\,r^m$ is the idealization the noisy fit hovers around.
The takeaway
A two-stage pipeline whose stages share a decay rate is a defective transition matrix, and its powers carry a polynomial-times-geometric $m\,r^m$ term that no diagonalizable model can produce. That term is the mathematics of a cohort piling up in the second stage before it drains — the interior peak in an engagement curve, timed at $m^\* \approx -1/\ln r$. The generalized eigenvector that completes the basis is, in business terms, the coupling between stages that a single eigenvector cannot capture. Whether the system evolves in discrete steps (this cohort) or continuous time (Case Study 1's door-closer), the Jordan block is the same idea: when modes merge, the eigenvalue alone tells you the rate, but only the Jordan structure tells you about the polynomial creep layered on top.