Key Takeaways: Chi-Square Tests: Categorical Data Analysis

One-Sentence Summary

Chi-square tests compare observed categorical frequencies to expected frequencies — either testing whether a single variable follows a specified distribution (goodness-of-fit) or whether two categorical variables are related (test of independence) — using the chi-square statistic $\chi^2 = \sum (O - E)^2 / E$ with effect sizes measured by Cramer's V and specific departures identified by standardized residuals.

Core Concepts at a Glance

The Chi-Square Goodness-of-Fit Test

Step by Step

1. State hypotheses: - $H_0$: The variable follows the specified distribution - $H_a$: The variable does not follow the specified distribution
2. Calculate expected frequencies: $E_i = n \times p_i$ where $p_i$ is the hypothesized proportion for category $i$
3. Check conditions: All expected counts $\geq 5$
4. Compute the test statistic: $\chi^2 = \sum (O_i - E_i)^2 / E_i$
5. Find the p-value: Use the chi-square distribution with $df = k - 1$
6. Decide and interpret: Reject or fail to reject $H_0$; state conclusion in context

Key Python Code

from scipy import stats

observed = [47, 52, 68, 33]
expected_proportions = [0.22, 0.30, 0.28, 0.20]
n = sum(observed)

chi2, p = stats.chisquare(observed,
                          f_exp=[p * n for p in expected_proportions])
print(f"Chi-square: {chi2:.3f}, p-value: {p:.4f}")

Excel

=CHISQ.TEST(observed_range, expected_range) returns the p-value directly.

The Chi-Square Test of Independence

Step by Step

1. State hypotheses: - $H_0$: The two variables are independent - $H_a$: The two variables are not independent
2. Calculate expected frequencies: $E_{ij} = (\text{Row total} \times \text{Column total}) / \text{Grand total}$
3. Check conditions: All expected counts $\geq 5$
4. Compute the test statistic: $\chi^2 = \sum (O_{ij} - E_{ij})^2 / E_{ij}$ (sum over all cells)
5. Find the p-value: Use the chi-square distribution with $df = (r-1)(c-1)$
6. Compute Cramer's V: $V = \sqrt{\chi^2 / (n \cdot (k-1))}$ where $k = \min(r, c)$
7. Compute standardized residuals: $(O - E) / \sqrt{E}$ to identify WHERE the association is
8. Decide and interpret: Report significance, effect size, and specific patterns

Key Python Code

from scipy import stats
import numpy as np

observed = np.array([
    [142, 58],
    [108, 92],
    [87, 63],
    [43, 7]
])

chi2, p, dof, expected = stats.chi2_contingency(observed)

# Cramer's V
n = observed.sum()
k = min(observed.shape) - 1
V = np.sqrt(chi2 / (n * k))

# Standardized residuals
residuals = (observed - expected) / np.sqrt(expected)

Expected Frequency Formulas

Degrees of Freedom

Effect Size: Cramer's V

$$V = \sqrt{\frac{\chi^2}{n \cdot (k-1)}} \quad \text{where } k = \min(r, c)$$

Why not just use $\chi^2$? The chi-square statistic is proportional to sample size. Double $n$ (keeping proportions fixed) and $\chi^2$ doubles, but $V$ stays the same. Cramer's V separates "how strong is the association?" from "how much data do we have?"

Standardized Residuals

$$\text{Standardized residual} = \frac{O_{ij} - E_{ij}}{\sqrt{E_{ij}}}$$

• Values beyond $\pm 2$ indicate notable departures from independence
• Positive residuals: more observations than expected in that cell
• Negative residuals: fewer observations than expected in that cell
• These identify WHERE the association lives — the chi-square statistic only tells you THAT it exists

Conditions for Chi-Square Tests

Common Misconceptions

Connection to Other Tests

How This Chapter Connects

The Key Themes

Theme 2: Categorical data often describes people. Every contingency table in this chapter classified people — by region, race, subscription tier, genre preference. The categories we choose shape what we can discover, and the categories institutions choose can reinforce or challenge existing power structures. The chi-square test reveals disparities; human judgment determines what to do about them.

Theme 1: Statistics as a superpower (via Theme 2). The chi-square test gave Maya the evidence she needed to demonstrate that disease burden falls unevenly across regions ($p < 0.001$). It gave James the data to show that bail decisions are associated with race ($p < 0.001$, $V = 0.206$). Without formal testing, these patterns might be dismissed as anecdotal. With chi-square tests, they become quantified, documented, and actionable.

The One Thing to Remember

If you forget everything else from this chapter, remember this:

The chi-square test compares observed categorical frequencies to expected frequencies. The goodness-of-fit test asks whether one categorical variable matches a specified distribution. The test of independence asks whether two categorical variables are related. Both use the formula $\chi^2 = \sum (O - E)^2 / E$ — a sum of standardized squared deviations that measures the total discrepancy between reality and expectation. Large values of $\chi^2$ indicate that the data don't fit the null hypothesis. But the chi-square statistic alone doesn't tell you WHERE the association is (use standardized residuals) or HOW STRONG it is (use Cramer's V). Always check that all expected frequencies are at least 5, and remember that chi-square tests establish association, not causation.

Key Terms