Crowd-sourced assessment of technical skills: an opportunity for improvement in the assessment of laparoscopic surgical skills - 21/01/16
, Thomas S. Lendvay, M.D., F.A.C.S. b, Mohamad I. Haque, M.D., L.T.C. c, Timothy Brand, M.D., L.T.C. c, Bryan Comstock, M.S. b, Justin Warren, M.B.A. b, Adnan Alseidi, M.D., M.Ed. aAbstract |
Background |
Objective, unbiased assessment of surgical skills remains a challenge in surgical education. We sought to evaluate the feasibility and reliability of Crowd-Sourced Assessment of Technical Skills.
Methods |
Seven volunteer general surgery interns were given time for training and then testing, on laparoscopic peg transfer, precision cutting, and intracorporeal knot-tying. Six faculty experts (FEs) and 203 Amazon.com Mechanical Turk crowd workers (CWs) evaluated 21 deidentified video clips using the Global Objective Assessment of Laparoscopic Skills validated rating instrument.
Results |
Within 19 hours and 15 minutes we received 662 eligible ratings from 203 CWs and 126 ratings from 6 FEs over 10 days. FE video ratings were of borderline internal consistency (Krippendorff’s alpha = .55). FE ratings were highly correlated with CW ratings (Pearson’s correlation coefficient = .78, P < .001).
Conclusion |
We propose the use of Crowd-Sourced Assessment of Technical Skills as a reliable, basic tool to standardize the evaluation of technical skills in general surgery.
Le texte complet de cet article est disponible en PDF.Keywords : Surgical skills education, Psychomotor skills, Surgical skills assessment, Crowd sourced data
Plan
| C-SATS served as a study sponsor and donated the use of their product for the completion of this study. In addition, three of the contributing authors: Thomas Lendvay MD FACS, Bryan Comstock MS and Justin Warren MBA are C-SATS employees who conducted the C-SATS preparation of videos and analysis of data. Interpretation of data and writing of the manuscript was a collaborative endeavor. |
Vol 211 - N° 2
P. 398-404 - février 2016 Retour au numéro
Article précédent
- Aligning institutional priorities: engaging house staff in a quality improvement and safety initiative to fulfill Clinical Learning Environment Review objectives and electronic medical record Meaningful Use requirements
- Meghan R. Flanagan, Carolyn C. Foster, Anneliese Schleyer, Gene N. Peterson, Samuel P. Mandell, Kristina E. Rudd, Byron D. Joyner, Thomas H. Payne
- Where do we start? The first survey of surgical residency education in Japan
- Yo Kurashima, Yusuke Watanabe, Yuma Ebihara, Soichi Murakami, Toshiaki Shichinohe, Satoshi Hirano
Bienvenue sur EM-consulte, la référence des professionnels de santé.
L’accès au texte intégral de cet article nécessite un abonnement.
Déjà abonné à cette revue ?