Chi-Square Tests for Categorical Data

Categorical data is the lifeblood of public health research, capturing everything from disease status (yes/no) and vaccination uptake to socioeconomic factors and health behaviors. To move from simple counts to meaningful insights, you need statistical tools that can detect patterns and associations within this type of data. The family of chi-square tests provides the fundamental framework for this analysis, allowing you to rigorously test hypotheses about how categorical variables relate to one another in your population of interest.

The Core Logic: Observed vs. Expected Frequencies

At its heart, every chi-square test operates on a simple yet powerful comparison: what you actually observed in your data versus what you would expect to observe if a specific null hypothesis were true. The null hypothesis ($H_0$) typically states that there is no association between variables or no deviation from a hypothesized distribution. The alternative hypothesis ($H_a$ or $H_1$) asserts that an association or deviation does exist.

The test quantifies the discrepancy between observed ($O$) and expected ($E$) frequencies. If the discrepancies are large and systematic, it becomes evidence against the null hypothesis. The calculation of expected frequencies depends entirely on the specific type of chi-square test you are conducting, which is determined by your research question. This elegant logic forms the basis for three primary applications: tests of independence, goodness-of-fit, and trend.

The Chi-Square Test Statistic and Its Assumptions

The aggregate measure of the difference between observed and expected counts is the chi-square test statistic, denoted as $X^2$. It is calculated using the formula:

$$X^2=\sum \frac{(O_i-E_i{)}^2}{E_i}$$

Here, you sum across all cells ($i$) in your frequency table. Each cell's contribution is the squared difference between its observed and expected count, divided by the expected count. This standardization is crucial—a difference of 5 is more meaningful if you expected 10 events than if you expected 1000. This calculated $X^2$ value is then compared to a critical value from the chi-square distribution, a theoretical distribution defined by its degrees of freedom (df). The degrees of freedom control the shape of the distribution and are determined by the dimensions of your data table.

For the results of this test to be valid, several key assumptions must be met:

1. The data are frequencies or counts, not percentages or proportions.
2. Categories are mutually exclusive. Each observation can belong to only one cell in the table.
3. Observations are independent. The value for one participant does not influence the value for another.
4. The sample is adequately large. A common rule is that all expected cell frequencies should be 5 or greater. Violations of this last assumption lead to the need for alternative methods like Fisher's exact test.

Primary Application 1: The Chi-Square Test of Independence

This is the most common application in public health. It assesses whether two categorical variables are associated in a population. You use it when you have two variables, like "Smoking Status" (Current, Former, Never) and "Lung Cancer Diagnosis" (Yes, No), and you want to know if the distribution of one variable differs across the levels of the other.

The data are organized into a contingency table (e.g., a 3x2 table for our example). The expected frequency for each cell is calculated under the assumption of independence: $E=(RowTotal\ast ColumnTotal)/GrandTotal$. For a contingency table with $r$ rows and $c$ columns, the degrees of freedom are $df=(r-1)(c-1)$.

Public Health Example: A researcher investigates the association between access to a community health worker (Yes/No) and adherence to a medication regimen (Adherent/Non-adherent) in a rural cohort. The chi-square test of independence would determine if the proportion of adherent patients is statistically different between those with and without access.

Primary Application 2: The Chi-Square Goodness-of-Fit Test

This test evaluates whether the distribution of a single categorical variable matches a hypothesized or theoretical distribution. It answers questions like, "Does the racial/ethnic breakdown of our clinic patients match the breakdown of the county population?" or "Do these genetic offspring follow a 9:3:3:1 Mendelian ratio?"

Here, you compare the observed counts for each category of your single variable to the counts you would expect based on the hypothesized proportions. The expected count for a category is $E_i=(HypothesizedProportio{n}_i)\ast (TotalSampleSize)$. The degrees of freedom are $df=(numberofcategories-1)$.

Public Health Example: A health department expects influenza vaccine uptake to be 45% for adults under 65 and 70% for adults 65+. A survey of 500 randomly selected adults yields observed counts in each age group. The goodness-of-fit test can determine if the observed survey data significantly deviates from the health department's expected distribution.

Primary Application 3: The Chi-Square Test for Trend

Also known as the Cochran-Armitage test for trend, this specialized test is used when you have a 2xk contingency table—that is, a binary outcome (like Disease/No Disease) and an exposure variable with ordinal levels (like Dose: Low, Medium, High). The test assesses whether there is a linear trend in the proportions of the outcome across the ordered levels of the exposure. It is more powerful for detecting an ordered dose-response relationship than the general test of independence.

The test assigns scores (often 1, 2, 3,...) to the ordered exposure categories and evaluates whether the proportion of "success" (e.g., disease) increases or decreases linearly with these scores. Its null hypothesis is that there is no linear trend.

Public Health Example: In a study on a new smoking cessation program, participants are assigned to weekly counseling sessions (1 session, 2 sessions, 3 sessions). The outcome is smoking abstinence at 6 months (Yes/No). The test for trend can specifically test if the probability of abstinence increases linearly with the number of counseling sessions attended.

Common Pitfalls and Corrections

Pitfall 1: Using chi-square tests with very small expected counts. When expected counts fall below 5, the chi-square distribution may not approximate the true sampling distribution well, leading to inflated Type I error rates (falsely rejecting the null). Correction: For 2x2 tables, use Fisher's exact test. This test calculates the exact probability of observing the table configuration, or one more extreme, given the fixed row and column totals, and is valid for any sample size. For larger tables, you may need to combine categories (if logically defensible) or use exact methods.

Pitfall 2: Interpreting a significant result as proof of causation. A significant chi-square test of independence indicates a statistically significant association, not that one variable caused the other. The observed relationship could be confounded by other unmeasured variables. Correction: Always frame conclusions in terms of association. Consider study design (e.g., a randomized trial supports causation more strongly than a cross-sectional survey) and use multivariate methods like logistic regression to control for potential confounders.

Pitfall 3: Applying the test to percentages or non-independent data. Feeding percentages or rates directly into the chi-square formula will produce invalid results. Similarly, if observations are matched (e.g., pre-test/post-test on the same individuals), the standard chi-square test violates the independence assumption. Correction: Always use raw counts. For paired or matched categorical data (e.g., case-control studies), use McNemar's test, which is the appropriate analogue for dependent samples.

Pitfall 4: Ignoring the strength and direction of the association. A significant p-value tells you an association exists but not how strong or meaningful it is. For a 2x2 table, a significant result could stem from a weak association in a very large sample or a strong association in a small one. Correction: Always report a measure of association strength alongside the test. For 2x2 tables, calculate the odds ratio (OR) or risk ratio (RR) with a confidence interval. For larger tables, consider standardized residuals to see which specific cells are driving the significant result.

Summary

• Chi-square tests are the primary tools for hypothesis testing with categorical (count) data, built on comparing observed frequencies to those expected under a null hypothesis.
• The three main types address distinct questions: the Test of Independence for associations between two variables, the Goodness-of-Fit Test for comparing a distribution to a theoretical one, and the Test for Trend for detecting linear patterns in ordinal data.
• Validity depends on key assumptions, most critically that expected cell counts are sufficiently large (typically ≥5). When this assumption is violated for 2x2 tables, Fisher's exact test provides a valid alternative.
• A significant result only demonstrates a statistical association, not causation. Proper interpretation must be paired with measures of association strength (like odds ratios) and an awareness of study design limitations.