Optimizing the utility of communication OSCEs: Omit station-specific checklists and provide students with narrative feedback

Objective: To evaluate how the utility (reliability, validity, acceptability, feasibility, cost and educational impact) of a communication-OSCE was influenced by whether or not station-specific (StSp) checklists were used together with a generic instrument and whether or not narrative feedback was provided to students. Methods: At ten stations, faculty members rated standardized patient-student interactions using the Common Ground (CG) instrument (at all stations) and StSp-checklists. Both raters and patients provided written feedback. The impact of changing the design on the various utility parameters was assessed: reliability by means of a generalizability study, cost using the Reznick model and the other utility parameters by means of a survey. Results: Use of the generic instrument (CG) proved more reliable (G coefficient = 0.67) than using the StSp-checklists (G = 0.47) or both (G = 0.65) while there was a high correlation between both scale scores (Pearsons' r = 0.86). The cost was 6.5% higher when StSp-checklists were used and 5% higher when narrative feedback was provided. Conclusion: The utility of a communication OSCE can be enhanced by omitting StSp-checklists and by providing narrative feedback to students. Practice implications: The same generic assessment scale can be used in all stations of a communication OSCE. Providing feedback to students is promising but it increases the costs.