HQ BaselinePsychometrics
The Practice Effect Problem: How Repeated Testing Inflates Your Baseline Scores
Annual baselines can create an unrealistically high bar — and subtly bias post-injury interpretation.
Athletes who take the same cognitive test annually develop familiarity with the test format, stimuli, and strategies — a phenomenon known as the practice effect. This familiarity can artificially inflate baseline scores over time, creating an unrealistically high bar that the athlete must clear when tested after a concussion.
How ImPACT addresses practice effects
ImPACT addresses practice effects through the use of alternate test forms — different word lists, design patterns, and symbol configurations across administrations. Research by Iverson et al. (2003), published in the Archives of Clinical Neuropsychology, established reliable change indices (RCIs) based on one-week test-retest data that account for expected practice-related improvement. However, as Schatz (2010) demonstrated with two-year reliability data, the magnitude of practice effects and measurement error both increase with longer intervals between administrations.
When short-interval RCIs fall short
Clinically, this means that reliable change indices calculated from short-interval test-retest studies may not be appropriate for an athlete whose baseline was performed months or years before their concussion, as noted by researchers in PMC (Bailey et al., 2020). If an athlete’s baseline was inflated by practice effects from three years of annual testing, and their post-injury test is compared using RCIs derived from a one-week study, the system may underestimate the true magnitude of cognitive decline.
The practical takeaway
Clinicians interpreting post-injury ImPACT data should consider the athlete’s testing history (how many times they’ve taken the test) and the interval since their most recent baseline. Athletes with extensive testing histories may benefit from being baselined on a different platform periodically to reset the practice effect, or their longitudinal baseline trajectory should be analyzed for practice-related score inflation.
How we handle it
At Headquarters, we track each athlete’s full testing history and account for practice effects in our clinical interpretation. We also offer multi-platform baselines for athletes with extensive ImPACT testing histories.