We need more replication research – A case for test-retest reliability

Jimmie Leppink, Patricia Pérez-fuster

Research output: Contribution to journalReview articleAcademicpeer-review

Abstract

Following debates in psychology on the importance of replication research, we have also started to see pleas for a more prominent role for replication research in medical education. To enable replication research, it is of paramount importance to carefully study the reliability of the instruments we use. Cronbach's alpha has been the most widely used estimator of reliability in the field of medical education, notably as some kind of quality label of test or questionnaire scores based on multiple items or of the reliability of assessment across exam stations. However, as this narrative review outlines, Cronbach's alpha or alternative reliability statistics may complement but not replace psychometric methods such as factor analysis. Moreover, multiple-item measurements should be preferred above single-item measurements, and when using single-item measurements, coefficients as Cronbach's alpha should not be interpreted as indicators of the reliability of a single item when that item is administered after fundamentally different activities, such as learning tasks that differ in content. Finally, if we want to follow up on recent pleas for more replication research, we have to start studying the test-retest reliability of the instruments we use.

Original languageEnglish
Pages (from-to)158-164
Number of pages7
JournalPerspectives on Medical Education
Volume6
Issue number3
DOIs
Publication statusPublished - 1 Jun 2017

Keywords

  • Cronbach's alpha
  • Test-retest reliability
  • Factor analysis
  • Multilevel analysis
  • Medical education
  • COGNITIVE LOAD MEASURES
  • CRONBACHS ALPHA
  • COEFFICIENT

Cite this