Questioning the rater idiosyncrasy explanation for error variance by searching for multiple signals within the noise

A.M. Gingerich

Research output: ThesisDoctoral ThesisExternal prepared

369 Downloads (Pure)

Abstract

Medical trainees are assessed performing clinical tasks but the examiners’ ratings can be highly variable. It is assumed that examiners assessing the same performance should form similar judgments and provide similar ratings. As such, the psychometric models currently used to analyze the ratings assume there is a single point of consensus. This research, however, found multiple clusters of consensus within the variable assessments provided by examiners for a single performance. This finding was consistent across two samples of participants and two different methodologies. Finding more than one point of consensus challenges the use of psychometric models to analyze examiners’ ratings.
Original languageEnglish
QualificationDoctor of Philosophy
Awarding Institution
  • Maastricht University
Supervisors/Advisors
  • van der Vleuten, Cornelis, Supervisor
  • Eva, K.W., Supervisor, External person
  • Regehr, G., Supervisor, External person
Award date3 Sep 2015
Place of PublicationMaastricht
Publisher
Print ISBNs9789461594556
Publication statusPublished - 2015

Keywords

  • rater cognition
  • medical education
  • rater-based assessment

Cite this