Mixed ANOVA Design

When research questions involve tracking changes over time while also comparing distinct groups, a simple t-test or one-way ANOVA falls short. You need a method that can simultaneously evaluate the effects of a between-subjects factor (like treatment group) and a within-subjects factor (like time) and, crucially, how they might influence each other. This is the purpose of a Mixed ANOVA (Mixed Analysis of Variance). It is a foundational tool in fields like clinical trials, psychology, and education, allowing you to dissect complex data structures where participants are measured repeatedly but also belong to different, independent categories. Mastering its logic, assumptions, and interpretation is key to moving beyond basic comparisons to sophisticated, multidimensional analysis.

Understanding the Two Types of Factors

The power of a Mixed ANOVA stems from its ability to model two different sources of variance in one unified analysis. A between-subjects factor (also called an independent-measures factor) involves comparisons across different, independent groups of participants. Each participant belongs to only one level of this factor. Common examples include experimental condition (e.g., Treatment vs. Control), gender, or species. The variability introduced by this factor reflects differences between these separate groups.

Conversely, a within-subjects factor (also called a repeated-measures factor) involves comparisons across conditions for the same participants. Each participant is measured under every level of this factor. The most common example is time (e.g., pre-test, post-test, follow-up), but it could also be different task conditions or stimuli presented to the same individual. The variability here reflects differences within the same participants across conditions. A Mixed ANOVA design combines these, such as in a classic pre-post design with a treatment and control group.

Hypotheses: Main Effects and Interactions

In a Mixed ANOVA, you formally test three primary sets of hypotheses. First, you test the main effect of the between-subjects factor. This asks: "Ignoring time, is there an overall difference between the groups?" For a Treatment vs. Control study, this tests whether the average score across all time points differs between these two independent populations.

Second, you test the main effect of the within-subjects factor. This asks: "Ignoring group membership, do scores change overall across time?" This tests whether there is a significant change from pre-test to post-test when you collapse the data across all participants.

The most critical test is usually for the interaction effect between the factors. This asks: "Does the change over time depend on which group the participant is in?" Or, conversely, "Does the difference between groups depend on the time of measurement?" A significant interaction indicates that the effect of one factor is not constant across the levels of the other. In an intervention study, this is the key evidence for a treatment effect: the treatment group shows a different pattern of change over time than the control group.

The Sphericity Assumption and Its Corrections

Like all repeated-measures analyses, the within-subjects part of a Mixed ANOVA relies on a critical statistical assumption called sphericity (or circularity). Conceptually, sphericity assumes that the variances of the differences between all possible pairs of within-subjects levels are equal. For example, the variance of the (Post-Pre) difference scores should be similar to the variance of the (Follow-up - Post) difference scores.

Violations of sphericity inflate the risk of a Type I error (falsely rejecting the null hypothesis). Fortunately, you can check for this using Mauchly's Test of Sphericity. If this test is significant ($p<.05$), the sphericity assumption is violated. To correct for this, you adjust the degrees of freedom used to calculate the p-value for the within-subjects effects. The two most common sphericity corrections are the Greenhouse-Geisser correction and the Huynh-Feldt correction. The Greenhouse-Geisser correction ($ϵ$) is more conservative and is generally preferred when the estimate of sphericity is low (e.g., $ϵ<0.75$). You would report the corrected p-value instead of the original. Modern practice often recommends routinely using the Greenhouse-Geisser correction when reporting results.

Interpreting a Significant Interaction: Simple Effects Analysis

Finding a significant interaction is only the first step. The next, essential step is to probe or "unpack" the interaction to understand its nature. This is done through simple effects analysis (sometimes called simple main effects). This analysis examines the effect of one factor at specific levels of the other factor. In our treatment/time example, there are two logical approaches.

First, you could analyze the simple effect of time within each group. This asks: "Did scores change significantly from pre to post for the Treatment group?" and separately, "Did scores change for the Control group?" You would conduct separate repeated-measures ANOVAs or paired t-tests for each group, often applying a correction (like Bonferroni) for multiple comparisons.

Second, you could analyze the simple effect of group at each time point. This asks: "Were the Treatment and Control groups significantly different at the pre-test?" and "Were they different at the post-test?" This involves conducting independent t-tests or one-way ANOVAs at each time slice. The results of these simple effects analyses tell the story: ideally, you'd find no group difference at pre-test but a significant difference at post-test, and a significant change over time only for the treatment group.

Common Pitfalls

Ignoring sphericity and its corrections. Running a Mixed ANOVA without checking Mauchly's test and applying a correction if needed is a serious error. It can lead to reporting "significant" within-subjects effects that are statistical artifacts. Always inspect the sphericity test output and report the corrected Greenhouse-Geisser or Huynh-Feldt p-values when appropriate.

Failing to follow up a significant interaction. Reporting only that "there was a significant interaction" is insufficient and uninformative. You must conduct and report the results of simple effects analyses to explain how the factors interact. A graph of the estimated marginal means is an essential visual companion to this analysis, but statistical tests are required to confirm what the graph appears to show.

Misinterpreting main effects in the presence of an interaction. When a significant interaction exists, the interpretation of the main effects for the involved factors can be misleading or meaningless. A main effect might be averaged across patterns that are going in opposite directions. Your primary interpretive focus should always shift to the interaction and its simple effects.

Overlooking assumptions for the between-subjects factor. While much attention is given to sphericity, the between-subjects part of the analysis has its own assumptions: normality of the dependent variable within each group and homogeneity of variances across groups (tested with Levene's test). Violations of these may require data transformation or the use of non-parametric alternatives.

Summary

• A Mixed ANOVA analyzes data with at least one between-subjects factor (independent groups) and one within-subjects factor (repeated measures) in a single model, testing for two main effects and their interaction.
• The key result is often the interaction effect, which indicates that the effect of one factor differs across the levels of the other (e.g., treatment groups change differently over time).
• The sphericity assumption must be checked for the within-subjects effects using Mauchly's test; violations require reporting p-values corrected with Greenhouse-Geisser or Huynh-Feldt estimators.
• A significant interaction must be probed using simple effects analysis to determine the precise nature of the effect—for example, testing for change over time separately within each group.
• Correct analysis requires vigilance for all assumptions, including sphericity for within-subjects effects and normality/homogeneity of variance for the between-subjects comparisons.