TY - JOUR
T1 - Using video-cases to assess student reflection: Development and validation of an instrument
AU - Koole, Sebastiaan
AU - Dornan, Tim
AU - Aper, Leen
AU - De Wever, Bram
AU - Scherpbier, Albert
AU - Valcke, Martin
AU - Cohen-Schotanus, Janke
AU - Derese, Anselme
PY - 2012/4/20
Y1 - 2012/4/20
N2 - Background: Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners' verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection. Methods: Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students' reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS (R)). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets. Results: Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff's alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and >= 6 cases from 5th year students. Conclusion: Use of StARS (R) to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.
AB - Background: Reflection is a meta-cognitive process, characterized by: 1. Awareness of self and the situation; 2. Critical analysis and understanding of both self and the situation; 3. Development of new perspectives to inform future actions. Assessors can only access reflections indirectly through learners' verbal and/or written expressions. Being privy to the situation that triggered reflection could place reflective materials into context. Video-cases make that possible and, coupled with a scoring rubric, offer a reliable way of assessing reflection. Methods: Fourth and fifth year undergraduate medical students were shown two interactive video-cases and asked to reflect on this experience, guided by six standard questions. The quality of students' reflections were scored using a specially developed Student Assessment of Reflection Scoring rubric (StARS (R)). Reflection scores were analyzed concerning interrater reliability and ability to discriminate between students. Further, the intra-rater reliability and case specificity were estimated by means of a generalizability study with rating and case scenario as facets. Results: Reflection scores of 270 students ranged widely and interrater reliability was acceptable (Krippendorff's alpha = 0.88). The generalizability study suggested 3 or 4 cases were needed to obtain reliable ratings from 4th year students and >= 6 cases from 5th year students. Conclusion: Use of StARS (R) to assess student reflections triggered by standardized video-cases had acceptable discriminative ability and reliability. We offer this practical method for assessing reflection summatively, and providing formative feedback in training situations.
U2 - 10.1186/1472-6920-12-22
DO - 10.1186/1472-6920-12-22
M3 - Article
C2 - 22520632
SN - 1472-6920
VL - 12
SP - 22
JO - BMC Medical Education
JF - BMC Medical Education
ER -