You want to measure performance of the same individual measured over a period of time (repeated observations) on an interval scale dependant variable, but, which procedure to use? So we are looking for an equivalent of the paired samples t-test, but we want to allow for two or more levels of the categorical variable i.e. pre, during, post.
The Repeated Measures ANOVA [SPSS: ANALYZE / GENERAL LINEAR MODEL / REPEATED MEASURES] is simpler to use but sadly its often not as accurate and flexible as using Linear Mixed Models (SPSS: ANALYZE / MIXED MODELS / LINEAR).
Reminder that the Linear Mixed Model is just an extension of the general linear model in which the linear predictor contains random effects in addition to the usual fixed effects.
Karen Grace-Martin provides some excellent guidelines:
- Both Repeated Measures ANOVA and *Linear* Mixed Models assume that the dependent variable is continuous, unbounded, and measured on an interval scale and that residuals will be normally distributed. There are, however, generalized linear mixed models that work for other types of dependent variables: categorical, ordinal, discrete counts, etc. So if you have one of these outcomes, ANOVA is not an option.
- If the design is very simple and there are no missing data, you will be very likely to get identical results from both approaches. By simple, I mean something like a pre-post design (with only two repeats) or an experiment with one between-subjects factor and another within-subjects factor. If that’s the case, Repeated Measures ANOVA is generally fine.
- In many designs, there is a repeated measure over time (or space), but subjects are also clustered in some other grouping. Students within classroom, patients within hospital, clients within social worker, are common examples. A repeated measures ANOVA can’t incorporate this extra clustering of subjects in some other grouping, but mixed models can. (In fact, this kind of grouping can get quite complicated.)
- As implied above, mixed models do a much better job of handling missing data. Repeated measures ANOVA can only use listwise deletion, which can cause bias and reduce power substantially. So use repeated measures only when missing data is minimal.
- Repeated measures ANOVA can only treat a repeat as a categorical factor. In other words, if measurements are made repeatedly over time and you want to treat time as continuous, you can’t do that in RM ANOVA. So for example, let’s say you’re measuring anxiety level during weeks 1, 2, 4, 8, and 12 of an anxiety-reduction intervention.
- While mixed models can treat those as true numbers and incorporate the different spacing of the weeks, RM ANOVA can’t.
- Repeated measures ANOVA falls apart when repeats are unbalanced. For example, a common design is to observe behaviors of different types, then compare them. One of the data sets we use in our Repeated Measures workshop compares the time it takes an infant to breath out while uttering different types of sounds. Each infant utters each sound type a different number of times.
Repeated measures can’t incorporate the fact that each infant has a different number of each sound type. It can only use one measurement for each sound type, so the only option is to average multiple breath durations for each infant, which under-represents the true variability in the data (this is bad). Mixed models can handle this just fine.
So what it really comes down to is Repeated Measures ANOVA is a good tool for some very specific situations. Once you deviate from those, trying to use RM Anova is like sticking that square peg through the round hole. You might get it through, but you’ll mangle your peg in the process.
The really big disadvantage to mixed models is their complexity, which is the other side of their flexibility.
So there you go!