Structured Examinations Remove Racial Grading Bias In Oral Examinations In The General Surgery Clerkship
*Katharine E Caldwell, *Jorge G Zarate Rodriguez, *Annie Hess, *Britta J Han, Michael M Awad, *Bethany C Sacks
Washington University in St. Louis, SAINT LOUIS, MO
OBJECTIVE(S): Previous research has identified disparities in surgical clerkship grading with under-represented in medicine (URiM) students receiving lower scores than white students. At the resident level, board examinations have been shown to be subject to racial bias. As oral examinations are uncommon in medical student clerkships, the existence of this bias has not been rigorously studied. Here, we describe the implementation of a structured oral examination to assess third year medical student (MS3s) knowledge of general surgery topics and mitigate unintentional racial bias.
METHODS: We evaluated the results of the previous year of unstructured oral examinations, conducted during the MS3 surgical clerkship at our institution (white n=36, URiM n=47) and evaluated scores based on student-reported demographics. We designed a structured oral examination utilizing an electronic checklist to assess MS3 knowledge in multiple domains: history and physical exam taking, diagnosis, laboratory and radiographic interpretation, and treatment planning. Scores of MS3s who completed our structured oral examination (white n=34, URiM n=29) were stratified by student-reported demographic factors. Students were surveyed regarding their examination experience.
RESULTS: Previous oral examinations demonstrated evidence of bias against URiM students. URiM students received significantly lower scores than white students (65% v 73%, p 0.004). After implementation of our structured oral examination, there were no significant differences in the scores of URiM and white students (92% v 95%, p 0.09). While our new examination trended with other evaluations of MS3 surgical knowledge, including NBME score (r 0.17, p 0.18), and receipt of an honors grade (mean oral exam score honors 94.4% v non-honors 91.9%, p 0.14), these results did not reach statistical significance and further data is needed to determine this relationship. On post-examination surveys, 100% of MS3s reported they believed the examination was fair.
CONCLUSIONS: We implemented a structured oral examination for MS3s on the surgery clerkship with high levels of student satisfaction and perceived fairness. Our exam eliminated the previously demonstrated racial bias in oral exam scoring. From these findings, we concluded that structured examinations create concrete grading criteria and remove ambiguities in grading structures which may contribute to unintentional racial bias.
Back to 2021 Abstracts