Case Study 2: When Normality Fails — Income, Wealth, and Power-Law Distributions

The Scenario

Professor Washington is guest-lecturing in an economics seminar when a student asks a question that stops him mid-sentence.

"You keep saying that many things in nature follow a bell curve," the student says. "So does income follow a bell curve too? And if it does, doesn't that mean most people are 'supposed' to be middle class, with only a few very rich or very poor?"

Washington pauses. "That's an outstanding question. And the answer is going to challenge everything you just learned in your statistics textbook."

He pulls up a dataset of U.S. household incomes and creates a histogram. The room goes quiet.

It's not a bell curve. Not even close.

The Shape of Income

U.S. household income is dramatically right-skewed. Most households cluster at the lower end, while a long tail stretches far to the right, representing a small number of extremely high earners.

Here are the approximate numbers from recent Census Bureau data:

Notice the enormous gap between the mean and median. In Chapter 6, you learned that when the mean is much larger than the median, the distribution is right-skewed. Income is the textbook example — literally. The mean is pulled upward by a small number of extremely high earners.

Why Income Can't Be Normal

There are structural reasons why income cannot follow a normal distribution:

1. Income has a floor but no ceiling. Nobody earns less than $0 (ignoring debt), but there's no upper limit. A normal distribution extends from $-\infty$ to $+\infty$, which would predict negative incomes. Income is bounded below at zero.

2. Multiplicative effects. The normal distribution arises from many small additive effects. But income growth is often multiplicative — a 10% raise on $50,000 adds $5,000, while a 10% raise on $500,000 adds $50,000. When effects multiply rather than add, you get skewed distributions.

3. Winner-take-all dynamics. In many fields, small differences in skill or luck lead to enormous differences in income. The top 0.1% of earners don't earn 10% more than average — they earn 100x or 1000x more. This creates the long right tail that defines income distributions.

4. Structural barriers. Access to education, networks, capital, and opportunity is not evenly distributed. These structural factors create clustering at certain income levels and gaps between groups.

The QQ-Plot Reveals Everything

import numpy as np
from scipy import stats
import matplotlib.pyplot as plt

# Simulate income data that mimics the U.S. distribution
# Using a log-normal distribution (common model for income)
np.random.seed(42)
income = np.random.lognormal(mean=10.9, sigma=0.8, size=5000)

fig, axes = plt.subplots(1, 3, figsize=(15, 5))

# Panel 1: Histogram
axes[0].hist(income, bins=50, density=True, alpha=0.7,
             color='steelblue', edgecolor='white')
axes[0].axvline(np.mean(income), color='red', linestyle='--',
                label=f'Mean: ${np.mean(income):,.0f}')
axes[0].axvline(np.median(income), color='green', linestyle='--',
                label=f'Median: ${np.median(income):,.0f}')
axes[0].set_xlabel('Household Income ($)')
axes[0].set_ylabel('Density')
axes[0].set_title('Income Distribution')
axes[0].legend()

# Panel 2: QQ-plot against normal
stats.probplot(income, dist="norm", plot=axes[1])
axes[1].set_title('QQ-Plot (vs. Normal)')

# Panel 3: QQ-plot against log-normal
stats.probplot(np.log(income), dist="norm", plot=axes[2])
axes[2].set_title('QQ-Plot of log(Income)\n(vs. Normal)')

plt.tight_layout()
plt.show()

# Shapiro-Wilk test
sample = np.random.choice(income, size=500, replace=False)
stat, p_val = stats.shapiro(sample)
print(f"Shapiro-Wilk test on income: W = {stat:.4f}, p = {p_val:.6f}")

stat_log, p_val_log = stats.shapiro(np.log(sample))
print(f"Shapiro-Wilk test on log(income): W = {stat_log:.4f}, p = {p_val_log:.4f}")

Visual description (three panels):

Panel 1 (Histogram): A dramatically right-skewed histogram. Most bars cluster between $0 and $100,000, with a rapid drop-off after $150,000 and a thin tail extending past $500,000. The mean (red dashed line) sits well to the right of the median (green dashed line), confirming the skewness. This looks nothing like a bell curve.

Panel 2 (QQ-plot vs. normal): A severely curved pattern. The points follow the line for the lower portion of the data, then curve sharply upward on the right side. The largest incomes are far above where a normal distribution would predict them. This QQ-plot screams "not normal."

Panel 3 (QQ-plot of log-income vs. normal): A remarkably straight line. When we take the logarithm of income, the QQ-plot becomes nearly linear, suggesting that log-transformed income is approximately normal. This means income itself follows a log-normal distribution.

The Shapiro-Wilk test confirms what the plots show: income data strongly rejects normality (p essentially zero), while log-transformed income passes the normality test.

The Log-Normal Alternative

When a variable is right-skewed with a floor at zero, statisticians often reach for the log-normal distribution. A variable follows a log-normal distribution when its logarithm follows a normal distribution.

$$\text{If } \ln(X) \sim N(\mu, \sigma^2) \text{, then } X \text{ is log-normally distributed}$$

This makes intuitive sense for income: the proportional changes in income (percentage raises, percentage investment returns) are approximately normal, even though the absolute dollar amounts are not.

The log-normal distribution has important properties: - It's always positive (no negative incomes) - It's right-skewed (long right tail) - The mean is always greater than the median - Small changes in the parameters can produce enormous changes in the tail

Wealth Is Even More Extreme

If income is skewed, wealth (total net worth) is astronomically more skewed. The wealth distribution doesn't even follow a log-normal — it's closer to a power-law or Pareto distribution, where the probability of having wealth $w$ or more decreases as a power of $w$:

$$P(W > w) \propto w^{-\alpha}$$

Recent Federal Reserve data shows:

No normal distribution could produce these numbers. In a normal distribution, the top 1% and bottom 1% are symmetric — equidistant from the mean. In the wealth distribution, the richest 1% hold more wealth than the bottom 90% combined.

Why This Matters: When the Normal Model Lies

Problem 1: The Mean Misrepresents

If you naively report that mean household income is $105,000 and assume normality, you'd conclude that "most families earn around $105,000." But the median is $75,000 — the typical family earns thirty thousand dollars less than the mean suggests. The skewness makes the mean a misleading measure of center, just as you learned in Chapter 6.

Worse, if you apply the Empirical Rule: "68% of households earn between $20,000 and $190,000" ($\mu \pm 1\sigma = 105{,}000 \pm 85{,}000$). But the lower bound hits $20,000, while the upper bound of $190,000 is far from the actual 84th percentile of income. The Empirical Rule gives wildly inaccurate results because the data isn't normal.

Problem 2: AI Systems Make the Same Mistake

Alex Rivera's team at StreamVibe once built a model predicting how much users would spend on in-app purchases. They assumed spending followed a normal distribution. The model dramatically underestimated the spending of "whale" users (the top 1-2% who account for a disproportionate share of revenue) and overestimated spending by typical users.

When they switched to a log-normal model, their predictions improved substantially. The lesson: when an AI system assumes normality and the data isn't normal, the predictions can be systematically wrong.

This happens constantly in: - Credit scoring: Default risk is not normally distributed. Heavy tails mean the normal model underestimates the probability of extreme defaults. - Insurance: Claim sizes follow heavy-tailed distributions. The 2008 financial crisis was partly caused by models that assumed normally distributed risks and were blindsided by tail events. - Network effects: Social media engagement, web traffic, and viral content follow power laws, not normal distributions. A few posts get millions of views while most get dozens.

Problem 3: Policy Implications

Maya Chen encounters this in public health. Healthcare costs per person are extremely right-skewed — a small number of patients account for a disproportionate share of spending. If a county health board assumes healthcare costs are normally distributed and budgets accordingly, they'll systematically underfund care for the sickest patients.

Studies consistently show that: - The top 5% of patients account for about 50% of healthcare spending - The top 1% account for about 20% - The bottom 50% account for only about 3%

No normal distribution produces these ratios. Using the wrong model doesn't just produce wrong numbers — it produces wrong policies.

How to Spot Non-Normal Data

This case study reinforces the normality assessment tools from Section 10.9, but with real stakes:

Professor Washington's Final Point

"Here's what I want you to take away," Washington says to the economics seminar. "The normal distribution is the most useful tool in statistics. I teach it. I use it. I believe in it. But I believe in it the way I believe in a hammer — it's great for nails, terrible for screws."

"Income, wealth, healthcare costs, social media engagement, city sizes, earthquake magnitudes, species extinction rates — these are all 'screws.' They don't follow the bell curve. And when we pretend they do, we make bad predictions, bad policies, and bad decisions."

"The bell curve tells you that most outcomes cluster near the average and extreme outcomes are vanishingly rare. That's true for heights and blood pressure. It's dangerously false for income and wealth. In those domains, extreme outcomes aren't rare — they define the entire system."

"George Box said all models are wrong. The normal model is wrong about income. But what makes a statistician is knowing when each model is useful and when it's not. That judgment is what this class is teaching you."

Discussion Questions

1. Mean vs. median in policy. When politicians cite "average income" to argue the economy is doing well, are they using the mean or the median? Which measure better represents the typical household's experience? How might the choice of statistic be used to mislead?

2. The log-normal connection. If log-transformed income is approximately normal, can you use z-scores and the z-table to find probabilities about income? Describe how. (Hint: transform to log scale, calculate the z-score, then look up the probability.)

3. AI and distributional assumptions. Choose a domain where AI makes predictions (credit scoring, medical diagnosis, content recommendation, insurance pricing). Research what distributional assumptions the models typically make. Are those assumptions justified? What happens when they fail?

4. The 2008 financial crisis. The phrase "fat tails" became famous during the financial crisis. Using what you've learned about the normal distribution, explain why models that assumed normally distributed financial returns dramatically underestimated the probability of extreme losses. What distribution might have been more appropriate?

5. Your own data. Identify a variable in your Data Detective Portfolio dataset that you think might NOT be normally distributed. Create a QQ-plot and run the Shapiro-Wilk test. If it's not normal, speculate about what distribution might be more appropriate.

Key Takeaways

• Income, wealth, healthcare costs, and many other real-world variables are not normally distributed
• The normal distribution assumes symmetric data with thin tails; many important phenomena have heavy right tails and are bounded below at zero
• The log-normal distribution often fits right-skewed positive data better than the normal
• Power-law distributions describe phenomena where extreme values are far more common than the normal model predicts
• Assuming normality when the data isn't normal leads to systematic errors in prediction, policy, and AI systems
• The mean is a misleading measure of center for skewed data — use the median instead
• "All models are wrong" includes the normal distribution — the skill is knowing when it's useful and when it's dangerously misleading