TY - JOUR
T1 - Competency-Based Medical Education in a Norm-Referenced World
T2 - A Root Cause Analysis of Challenges to the Competency-Based Paradigm in Medical School
AU - Ryan, Michael S
AU - Lomis, Kimberly D
AU - Deiorio, Nicole M
AU - Cutrer, William B
AU - Pusic, Martin V
AU - Caretta-Weyer, Holly A
PY - 2023/11
Y1 - 2023/11
N2 - Competency-based medical education (CBME) requires a criterion-referenced approach to assessment. However, despite best efforts to advance CBME, there remains an implicit, and at times, explicit, demand for norm-referencing, particularly at the junction of undergraduate medical education (UME) and graduate medical education (GME). In this manuscript, the authors perform a root-cause analysis to determine the underlying reasons for continued norm-referencing in the context of the movement toward CBME. The root-cause analysis consisted of 2 processes: (1) identification of potential causes and effects organized into a fishbone diagram, and (2) identification of the 5 whys. The fishbone diagram identified 2 primary drivers: the false notion that measures such as grades are truly objective and the importance of different incentives for different key constituents. From these drivers, the importance of norm-referencing for residency selection was identified as a critical component. Exploration of the 5 whys further detailed the reasons for continuation of norm-referenced grading to facilitate selection, including the need for efficient screening in residency selection, dependence upon rank-order lists, perception that there is a best outcome to the match, lack of trust between residency programs and medical schools, and inadequate resources to support progression of trainees. Based on these findings, the authors argue that the implied purpose of assessment in UME is primarily stratification for residency selection. Because stratification requires comparison, a norm-referenced approach is needed. To advance CBME, the authors recommend reconsideration of the approach to assessment in UME to maintain the purpose of selection while also advancing the purpose of rendering a competency decision. Changing the approach will require a collaboration between national organizations, accrediting bodies, GME programs, UME programs, students, and patients/societies. Details are provided regarding the specific approaches required of each key constituent group.
AB - Competency-based medical education (CBME) requires a criterion-referenced approach to assessment. However, despite best efforts to advance CBME, there remains an implicit, and at times, explicit, demand for norm-referencing, particularly at the junction of undergraduate medical education (UME) and graduate medical education (GME). In this manuscript, the authors perform a root-cause analysis to determine the underlying reasons for continued norm-referencing in the context of the movement toward CBME. The root-cause analysis consisted of 2 processes: (1) identification of potential causes and effects organized into a fishbone diagram, and (2) identification of the 5 whys. The fishbone diagram identified 2 primary drivers: the false notion that measures such as grades are truly objective and the importance of different incentives for different key constituents. From these drivers, the importance of norm-referencing for residency selection was identified as a critical component. Exploration of the 5 whys further detailed the reasons for continuation of norm-referenced grading to facilitate selection, including the need for efficient screening in residency selection, dependence upon rank-order lists, perception that there is a best outcome to the match, lack of trust between residency programs and medical schools, and inadequate resources to support progression of trainees. Based on these findings, the authors argue that the implied purpose of assessment in UME is primarily stratification for residency selection. Because stratification requires comparison, a norm-referenced approach is needed. To advance CBME, the authors recommend reconsideration of the approach to assessment in UME to maintain the purpose of selection while also advancing the purpose of rendering a competency decision. Changing the approach will require a collaboration between national organizations, accrediting bodies, GME programs, UME programs, students, and patients/societies. Details are provided regarding the specific approaches required of each key constituent group.
U2 - 10.1097/ACM.0000000000005220
DO - 10.1097/ACM.0000000000005220
M3 - Article
SN - 1040-2446
VL - 98
SP - 1251
EP - 1260
JO - Academic Medicine
JF - Academic Medicine
IS - 11
ER -