The reliability of operative rating tool evaluations: How late is too late to provide operative performance feedback? - 06/12/18
, Meredith L. Meyer , Damon S. Cooney , Gedge D. Rosson , Scott D. Lifchez , Carisa M. Cooney ⁎ Abstract |
Background |
Operative rating tools can enhance performance assessment in surgical training. However, assessments completed late may have questionable reliability. We evaluated the reliability of assessments according to evaluation time-to-completion.
Methods |
We stratified assessments from MileMarker‘s™ Operative Entrustability Assessment by evaluation time-to-completion, using concordance correlation coefficient (CCC) between self-assessment and evaluator scores as a measure of reliability.
Results |
Overall, self-assessment and evaluator scores were strongly correlated (CCC = 0.72; p < 0.001) though self-assessments were slightly higher (p = 0.048). Reliability remained stable for evaluations completed within 0 days (CCC = 0.77; p < 0.001), 1–3 days (CCC = 0.73; p < 0.001), and 4–13 days after surgery (CCC = 0.69; p < 0.001), but dropped for evaluations completed within 14–38 days (CCC = 0.60; p < 0.001) and over 38 days (CCC = 0.54; p < 0.001) after surgery. There was strong evidence for an interaction between time-to-completion and reliability (p < 0.001).
Conclusions |
Our data support the reliability of assessments completed until 2 weeks after surgery. This finding may help refine the interpretation of evaluation scores as surgical specialties move toward competency-based accreditation.
Le texte complet de cet article est disponible en PDF.Highlights |
• | Global rating scales may be used for operative performance assessment. |
• | Feedback given within 13 days of a case demonstrates validity. |
• | Self-assessment and evaluator scores completed soon after cases correlate closely. |
• | Consider with caution performance evaluations submitted >13 days after a case. |
Keywords : Operative skills, Resident evaluation, Resident performance, Graduate medical education, Surgical education
Plan
Vol 216 - N° 6
P. 1052-1055 - décembre 2018 Retour au numéro
Article précédent
- “Are we there yet?”- factors affecting postoperative follow-up after general surgery procedures
- Kristen Harkey, Nicole Kaiser, Michael Inman, Caroline E. Reinke
- FRIEND or FOE: A prospective evaluation of risk factors for reintubation in surgical and trauma patients
- Christopher P. Michetti, Margaret M. Griffen, Erik J. Teicher, Jennifer L. Rodriguez, Hani Seoudi, Chang Liu, Elena Lita, Anna Newcomb
Bienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?