Case Study 2: What Happens to the Race Effect When You Control for Criminal History?

The Setup

Professor James Washington has been building this argument for months.

In Chapter 16, he compared false positive rates across racial groups and found that the algorithm flagged Black defendants as "high risk" at higher rates than White defendants, even among those who did not reoffend. In Chapter 19, he used a chi-square test to show that bail decisions were not independent of race. In Chapter 20, he found significant differences in algorithm risk scores across demographic groups.

But every time he presents these findings, someone raises the same objection: "You're not being fair to the algorithm. Of course risk scores differ by race — they're based on criminal history, and criminal history differs by race because of systemic factors in the criminal justice system. If you control for criminal history, the race effect will disappear."

Now, with multiple regression, James can finally test this claim directly. Does the race effect disappear when you control for prior convictions, offense severity, and age?

The answer will change the conversation.

The Data

James has obtained data on 500 defendants from the county court system: their race, prior convictions, offense severity (rated 1-5 by the court), age at arrest, and the algorithm's risk score (1-10 scale).

import numpy as np
import pandas as pd
import statsmodels.formula.api as smf
from statsmodels.stats.outliers_influence import variance_inflation_factor
import matplotlib.pyplot as plt
import seaborn as sns
from scipy import stats

np.random.seed(2026)
n = 500

# ============================================================
# JAMES'S CRIMINAL JUSTICE ANALYSIS
# Does the race effect on risk scores survive controlling
# for criminal history and other legal factors?
# ============================================================

# Generate defendant data with realistic structure
race = np.random.choice(['White', 'Black'], n, p=[0.55, 0.45])

# Prior convictions: slightly higher mean for Black defendants
# (reflecting differential policing and sentencing)
prior_convictions = np.where(
    race == 'Black',
    np.random.poisson(2.2, n),  # Black mean: 2.2
    np.random.poisson(1.6, n)   # White mean: 1.6
)

# Offense severity: similar distribution across races
offense_severity = np.random.choice([1, 2, 3, 4, 5], n,
                                      p=[0.15, 0.25, 0.30, 0.20, 0.10])

# Age: slightly younger average for Black defendants
age = np.where(
    race == 'Black',
    np.random.normal(30, 7, n).clip(18, 65).astype(int),
    np.random.normal(33, 8, n).clip(18, 65).astype(int)
)

# Risk score: based on legitimate factors + small race bias
risk_score = (
    1.5 +
    1.4 * prior_convictions +       # Prior record (legitimate)
    0.6 * offense_severity +        # Offense severity (legitimate)
    -0.04 * age +                   # Younger = higher risk (legitimate)
    0.7 * (race == 'Black') +       # Race bias (NOT legitimate)
    np.random.normal(0, 1.0, n)     # Random noise
).clip(1, 10)

defendants = pd.DataFrame({
    'race': race,
    'prior_convictions': prior_convictions,
    'offense_severity': offense_severity,
    'age': age,
    'risk_score': risk_score
})

print("=" * 70)
print("JAMES'S DEFENDANT DATA: SUMMARY")
print("=" * 70)
print(f"Total defendants: {n}")
print(f"\nRacial composition:")
print(defendants['race'].value_counts())
print(f"\nDescriptive statistics by race:")
print(defendants.groupby('race')[['prior_convictions', 'offense_severity',
                                    'age', 'risk_score']].mean().round(2))

Phase 1: The Simple Model (Race Only)

James starts with the claim at face value. What does the raw race effect look like?

# ---- PHASE 1: SIMPLE MODEL ----
print("\n" + "=" * 70)
print("MODEL 1: Risk Score ~ Race (no controls)")
print("=" * 70)

model1 = smf.ols(
    'risk_score ~ C(race, Treatment(reference="White"))',
    data=defendants
).fit()

print(f"\nRace coefficient (Black vs. White): "
      f"{model1.params.iloc[1]:.3f}")
print(f"Standard error: {model1.bse.iloc[1]:.3f}")
print(f"t-statistic: {model1.tvalues.iloc[1]:.3f}")
print(f"p-value: {model1.pvalues.iloc[1]:.6f}")
print(f"95% CI: ({model1.conf_int().iloc[1, 0]:.3f}, "
      f"{model1.conf_int().iloc[1, 1]:.3f})")
print(f"\nR² = {model1.rsquared:.4f}")
print(f"Adj. R² = {model1.rsquared_adj:.4f}")

# Visualize
fig, ax = plt.subplots(figsize=(8, 6))
sns.boxplot(data=defendants, x='race', y='risk_score',
            palette={'White': 'lightblue', 'Black': 'salmon'},
            order=['White', 'Black'], ax=ax)
ax.set_xlabel('Race', fontsize=12)
ax.set_ylabel('Algorithm Risk Score', fontsize=12)
ax.set_title('Risk Score Distribution by Race (No Controls)',
             fontsize=14, fontweight='bold')
plt.tight_layout()
plt.show()

What Model 1 Shows

The simple regression reveals a raw gap of approximately 1.4 points on the 10-point risk scale. Black defendants receive risk scores that are, on average, 1.4 points higher than White defendants.

$R^2 \approx 0.05$ — race alone explains only about 5% of the variability in risk scores. Most of the variability comes from other factors.

"This is the number the defense attorneys cite," James tells his class. "And the algorithm developers respond: 'That's because Black defendants in this dataset have more prior convictions on average, and prior convictions are what the algorithm uses. It's not race — it's criminal history.'"

"Let's test that claim."

Phase 2: Adding Criminal History

# ---- PHASE 2: CONTROLLING FOR CRIMINAL HISTORY ----
print("\n" + "=" * 70)
print("MODEL 2: Risk Score ~ Race + Prior Convictions")
print("=" * 70)

model2 = smf.ols(
    'risk_score ~ C(race, Treatment(reference="White")) + '
    'prior_convictions',
    data=defendants
).fit()

print(model2.summary().tables[1])
print(f"\nR² = {model2.rsquared:.4f}")
print(f"Adj. R² = {model2.rsquared_adj:.4f}")
print(f"\nRace coefficient BEFORE controlling: "
      f"{model1.params.iloc[1]:.3f}")
print(f"Race coefficient AFTER controlling for priors: "
      f"{model2.params.iloc[1]:.3f}")
change = ((model2.params.iloc[1] - model1.params.iloc[1]) /
          model1.params.iloc[1] * 100)
print(f"Change: {change:.1f}%")

What Happens

The race effect dropped from 1.40 to 0.92 — a 34% reduction. Adding prior convictions soaked up a substantial portion of the race-risk score association. The algorithm developers' argument has some validity: part of the raw race gap is explained by differences in criminal history.

But the race coefficient is still 0.92, and it's still highly significant. Criminal history doesn't explain all of it.

Phase 3: The Full Model

# ---- PHASE 3: FULL MODEL ----
print("\n" + "=" * 70)
print("MODEL 3: Risk Score ~ Race + Priors + Severity + Age (full model)")
print("=" * 70)

model3 = smf.ols(
    'risk_score ~ C(race, Treatment(reference="White")) + '
    'prior_convictions + offense_severity + age',
    data=defendants
).fit()

print(model3.summary())

Full Model Results

                                        coef   std err       t    P>|t|  [0.025  0.975]
---------------------------------------------------------------------------------------
Intercept                              2.354     0.468    5.030   0.000   1.434   3.274
C(race, Treatment(ref="White"))[T.Bl]  0.712     0.098    7.265   0.000   0.519   0.905
prior_convictions                      1.382     0.032   43.188   0.000   1.319   1.445
offense_severity                       0.587     0.044   13.341   0.000   0.500   0.674
age                                   -0.038     0.006   -6.333   0.000  -0.050  -0.026

R² = 0.838    Adj. R² = 0.837    F = 641.2    p(F) < 0.001

The Coefficient Progression

# Visualize the coefficient progression
fig, ax = plt.subplots(figsize=(10, 6))

models = ['Model 1\n(Race only)',
          'Model 2\n(+ Priors)',
          'Model 3\n(+ Severity, Age)']
coefficients = [model1.params.iloc[1], model2.params.iloc[1],
                model3.params.iloc[1]]
ci_lower = [model1.conf_int().iloc[1, 0], model2.conf_int().iloc[1, 0],
            model3.conf_int().iloc[1, 0]]
ci_upper = [model1.conf_int().iloc[1, 1], model2.conf_int().iloc[1, 1],
            model3.conf_int().iloc[1, 1]]

errors = [[c - lo for c, lo in zip(coefficients, ci_lower)],
          [hi - c for c, hi in zip(coefficients, ci_upper)]]

ax.barh(models, coefficients, color=['#e74c3c', '#e67e22', '#2ecc71'],
        xerr=errors, capsize=5, edgecolor='navy', linewidth=1.2)
ax.axvline(x=0, color='black', linewidth=1, linestyle='-')
ax.set_xlabel('Race Coefficient (Black vs. White)', fontsize=12)
ax.set_title('How the Race Effect Changes as Controls Are Added',
             fontsize=14, fontweight='bold')

for i, v in enumerate(coefficients):
    ax.text(v + 0.05, i, f'{v:.2f}', va='center', fontweight='bold')

plt.tight_layout()
plt.show()

Phase 4: VIF and Diagnostics

# ---- PHASE 4: DIAGNOSTICS ----
print("\n" + "=" * 70)
print("PHASE 4: DIAGNOSTICS")
print("=" * 70)

# Create numeric race variable for VIF
defendants['race_black'] = (defendants['race'] == 'Black').astype(int)
X = defendants[['race_black', 'prior_convictions',
                 'offense_severity', 'age']]
vif_data = pd.DataFrame()
vif_data['Variable'] = X.columns
vif_data['VIF'] = [variance_inflation_factor(X.values, i)
                   for i in range(X.shape[1])]
print("\nVariance Inflation Factors:")
print(vif_data.to_string(index=False))

# Residual diagnostics
fig, axes = plt.subplots(1, 3, figsize=(15, 5))

# Residuals vs. predicted
axes[0].scatter(model3.fittedvalues, model3.resid,
                color='steelblue', edgecolors='navy', s=30, alpha=0.5)
axes[0].axhline(y=0, color='red', linestyle='--', linewidth=1.5)
axes[0].set_xlabel('Predicted Risk Score')
axes[0].set_ylabel('Residual')
axes[0].set_title('Residuals vs. Predicted')

# QQ-plot
stats.probplot(model3.resid, plot=axes[1])
axes[1].set_title('Normal QQ-Plot')

# Residuals by race (critical check)
resid_white = model3.resid[defendants['race'] == 'White']
resid_black = model3.resid[defendants['race'] == 'Black']
axes[2].boxplot([resid_white, resid_black],
                labels=['White', 'Black'])
axes[2].axhline(y=0, color='red', linestyle='--', linewidth=1)
axes[2].set_ylabel('Residual')
axes[2].set_title('Residuals by Race')

plt.suptitle('Diagnostics for Full Model', fontsize=14,
             fontweight='bold')
plt.tight_layout()
plt.show()

Diagnostic Results

The low VIF values are notable: unlike Maya's health data, these predictors are not strongly correlated with each other. This means the coefficient estimates are relatively stable and interpretable.

Phase 5: The Interpretation Debate

James presents his findings to a seminar audience. The debate that follows illustrates why multiple regression in sensitive domains requires both statistical skill and ethical reasoning.

Position 1: "The Algorithm Is Mostly Fair"

"The race coefficient dropped by 49% — from 1.40 to 0.71 — when you controlled for legitimate factors. Most of the racial disparity is explained by criminal history and offense severity, which are exactly the factors a risk algorithm should use. The remaining 0.71-point difference could reflect unmeasured legitimate factors you haven't included, like failure to appear in court or substance abuse history. You can't conclude bias until you've controlled for everything."

Position 2: "The Algorithm Is Biased"

"Even after controlling for every factor in the model, Black defendants receive risk scores 0.71 points higher than otherwise identical White defendants. On a 10-point scale, that's meaningful — it could push someone from a 6 to a 7, crossing the 'high-risk' threshold that determines pretrial detention. And the factors we controlled for aren't neutral: prior convictions reflect policing patterns (more police in Black neighborhoods means more arrests for the same behavior), and offense severity reflects prosecutorial discretion. Controlling for these factors is controlling for the very systemic bias we're trying to detect."

Position 3: "The Question Is Deeper Than Regression"

"Multiple regression can tell us whether race is associated with risk scores after controlling for other variables. It cannot tell us why. Is the 0.71-point gap due to explicit bias in the algorithm's code? Implicit bias in the training data? Unmeasured confounders? A fundamental impossibility of 'fairness' in a biased system? These are questions that regression alone cannot answer. We need algorithmic audits, qualitative research, and community input alongside the statistics."

# ---- THE PRACTICAL IMPACT ----
print("\n" + "=" * 70)
print("PRACTICAL IMPACT: WHAT DOES 0.71 POINTS MEAN?")
print("=" * 70)

# What fraction of Black defendants are pushed above the
# "high risk" threshold (score >= 7) by the race effect?
defendants['predicted_no_race_bias'] = model3.fittedvalues - \
    0.712 * defendants['race_black']

threshold = 7
black_defs = defendants[defendants['race'] == 'Black']

above_with_bias = (black_defs['risk_score'] >= threshold).sum()
above_without = (black_defs['predicted_no_race_bias'] >= threshold).sum()

print(f"\nBlack defendants above high-risk threshold (score >= 7):")
print(f"  With current algorithm: {above_with_bias} "
      f"({above_with_bias/len(black_defs)*100:.1f}%)")
print(f"  Without race effect:    {above_without} "
      f"({above_without/len(black_defs)*100:.1f}%)")
print(f"  Difference: {above_with_bias - above_without} defendants "
      f"affected by the race effect")

The Human Cost

James calculates that approximately 15-20% of Black defendants who are flagged as "high risk" would not be flagged if the algorithm's race effect were removed. These are real people who may be held in pretrial detention, denied bail, or given harsher sentences because of a 0.71-point bump that cannot be explained by their criminal history, offense severity, or age.

"0.71 points sounds small," James tells his class. "But for the defendants sitting on the boundary between freedom and detention, it's everything."

James's Report: A Model of Statistical Reasoning

James writes up his findings for the county's Criminal Justice Review Board. His report models the kind of careful, nuanced communication that multiple regression demands:

Key Finding: After controlling for prior criminal record, current offense severity, and age, Black defendants receive algorithm risk scores approximately 0.7 points higher (on a 1-10 scale) than White defendants with identical profiles. This effect is statistically significant ($p < 0.001$) and has practical consequences for detention decisions.

What this means: The racial disparity in risk scores is not entirely explained by differences in criminal history. While criminal record accounts for roughly half the raw race gap, the remaining disparity persists after statistical controls.

What this does NOT mean: We cannot conclude from this analysis alone that the algorithm is "biased" in a legal or moral sense. There may be legitimate risk factors not captured in our model. However, the burden of proof should shift: the algorithm developer should demonstrate what unmeasured factors justify the residual race effect, rather than assuming it will disappear with more controls.

Recommendation: The county should commission an independent algorithmic audit, with access to the algorithm's full variable set, to determine whether the race effect can be fully explained by legitimate, race-neutral factors.

Discussion Questions

1. The race coefficient dropped from 1.40 to 0.71 when controls were added. A critic says: "See? More than half the disparity is explained by legitimate factors. With enough controls, the rest would disappear too." Evaluate this argument. Is it logically sound?

2. Position 2 argues that controlling for prior convictions is "controlling for the very bias we're trying to detect." Explain this argument. Do you agree? What would be the implications of not controlling for prior convictions?

3. James's model explains 84% of the variability in risk scores ($R^2 = 0.838$). Does this high $R^2$ make the finding of racial bias more or less convincing? Why?

4. If the VIF for the race variable were 8.0 instead of 1.2, how would that change your confidence in the race coefficient? Why does the low VIF in this case strengthen the finding?

5. James uses the phrase "otherwise identical" when interpreting the race coefficient. What does this phrase mean in the context of multiple regression? Why is it both powerful and misleading?

6. How does this case study illustrate the distinction between "statistical control" and "experimental control"? What could an experiment tell us that James's regression cannot?

7. Ethical reflection: If you were on the Criminal Justice Review Board, what would you do with James's findings? What additional information would you want before making a decision?