We need more replication research – A case for test-retest reliability

Jimmie Leppink*, Patricia Pérez-fuster

*Corresponding author for this work

Research output: Contribution to journal(Systematic) Review article peer-review


Following debates in psychology on the importance of replication research, we have also started to see pleas for a more prominent role for replication research in medical education. To enable replication research, it is of paramount importance to carefully study the reliability of the instruments we use. Cronbach's alpha has been the most widely used estimator of reliability in the field of medical education, notably as some kind of quality label of test or questionnaire scores based on multiple items or of the reliability of assessment across exam stations. However, as this narrative review outlines, Cronbach's alpha or alternative reliability statistics may complement but not replace psychometric methods such as factor analysis. Moreover, multiple-item measurements should be preferred above single-item measurements, and when using single-item measurements, coefficients as Cronbach's alpha should not be interpreted as indicators of the reliability of a single item when that item is administered after fundamentally different activities, such as learning tasks that differ in content. Finally, if we want to follow up on recent pleas for more replication research, we have to start studying the test-retest reliability of the instruments we use.

Original languageEnglish
Pages (from-to)158-164
Number of pages7
JournalPerspectives on Medical Education
Issue number3
Publication statusPublished - 1 Jun 2017


  • Cronbach's alpha
  • Test-retest reliability
  • Factor analysis
  • Multilevel analysis
  • Medical education


Dive into the research topics of 'We need more replication research – A case for test-retest reliability'. Together they form a unique fingerprint.

Cite this