TY - JOUR
T1 - When Feedback Backfires: Influences of Negative Discrepancies Between Physicians' Self and Assessors' Scores on Their Subsequent Multisource Feedback Ratings
AU - van der Meulen, M.W.
AU - Arah, O.A.
AU - Heeneman, S.
AU - Egbrink, M.G.A.O.
AU - van der Vleuten, C.P.M.
AU - Lombarts, K.M.J.M.H.
N1 - Funding Information:
The authors thank Empirion/Medox.nl for their efforts in the design and continuous technical development of the multisource feedback instrument “INCEPT” web-based application.
Publisher Copyright:
Copyright © 2021 The Alliance for Continuing Education in the Health Professions.
PY - 2021
Y1 - 2021
N2 - Introduction: With multisource feedback (MSF) physicians might overrate their own performance compared with scores received from assessors. However, there is limited insight into how perceived divergent feedback affects physicians' subsequent performance scores. Methods: During 2012 to 2018, 103 physicians were evaluated twice by 684 peers, 242 residents, 999 coworkers, and themselves in three MSF performance domains. Mixed-effect models quantified associations between the outcome variable "score changes" between first and second MSF evaluations, and the explanatory variable "negative discrepancy score" (number of items that physicians rated themselves higher compared with their assessors' scores) at the first MSF evaluation. Whether associations differed across assessor groups and across a physician's years of experience as a doctor was analyzed too. Results: Forty-nine percent of physicians improved their total MSF score at the second evaluation, as assessed by others. Number of negative discrepancies was negatively associated with score changes in domains "organization and (self)management" (b = -0.02; 95% confidence interval [CI], -0.03 to -0.02; SE = 0.004) and "patient-centeredness" (b = -0.03; 95% CI, -0.03 to -0.02; SE = 0.004). For "professional attitude," only negative associations between score changes and negative discrepancies existed for physicians with more than 6-year experience (b(6-10yearsofexperience) = -0.03; 95% CI, -0.05 to -0.003; SE = 0.01; b(16-20yearsofexperience) = -0.03; 95% CI, -0.06 to -0.004; SE = 0.01). Discussion: The extent of performance improvement was less for physicians confronted with negative discrepancies. Performance scores actually declined when physicians overrated themselves on more than half of the feedback items. PA score changes of more experienced physicians confronted with negative discrepancies and were affected more adversely. These physicians might have discounted feedback due to having more confidence in own performance. Future work should investigate how MSF could improve physicians' performance taking into account physicians' confidence.
AB - Introduction: With multisource feedback (MSF) physicians might overrate their own performance compared with scores received from assessors. However, there is limited insight into how perceived divergent feedback affects physicians' subsequent performance scores. Methods: During 2012 to 2018, 103 physicians were evaluated twice by 684 peers, 242 residents, 999 coworkers, and themselves in three MSF performance domains. Mixed-effect models quantified associations between the outcome variable "score changes" between first and second MSF evaluations, and the explanatory variable "negative discrepancy score" (number of items that physicians rated themselves higher compared with their assessors' scores) at the first MSF evaluation. Whether associations differed across assessor groups and across a physician's years of experience as a doctor was analyzed too. Results: Forty-nine percent of physicians improved their total MSF score at the second evaluation, as assessed by others. Number of negative discrepancies was negatively associated with score changes in domains "organization and (self)management" (b = -0.02; 95% confidence interval [CI], -0.03 to -0.02; SE = 0.004) and "patient-centeredness" (b = -0.03; 95% CI, -0.03 to -0.02; SE = 0.004). For "professional attitude," only negative associations between score changes and negative discrepancies existed for physicians with more than 6-year experience (b(6-10yearsofexperience) = -0.03; 95% CI, -0.05 to -0.003; SE = 0.01; b(16-20yearsofexperience) = -0.03; 95% CI, -0.06 to -0.004; SE = 0.01). Discussion: The extent of performance improvement was less for physicians confronted with negative discrepancies. Performance scores actually declined when physicians overrated themselves on more than half of the feedback items. PA score changes of more experienced physicians confronted with negative discrepancies and were affected more adversely. These physicians might have discounted feedback due to having more confidence in own performance. Future work should investigate how MSF could improve physicians' performance taking into account physicians' confidence.
KW - continuing professional development
KW - multisource feedback
KW - performance improvement
KW - physicians' professional performance
KW - score changes
KW - experience
KW - workplace-based assessment
KW - OTHER AGREEMENT
KW - PERFORMANCE
KW - DOCTORS
KW - PERCEPTIONS
KW - TOOL
KW - CERTIFICATION
KW - REFLECTION
KW - ACCURACY
KW - PATIENT
KW - BOARD
U2 - 10.1097/CEH.0000000000000347
DO - 10.1097/CEH.0000000000000347
M3 - Article
C2 - 34009839
SN - 0894-1912
VL - 41
SP - 94
EP - 103
JO - Journal of Continuing Education in the Health Professions
JF - Journal of Continuing Education in the Health Professions
IS - 2
ER -