Evaluating Script Concordance Tests (SCTs) through the Lens of Bayesian Reasoning: Enhancing Assessment in Medical Education - 25/12/24

Doi : 10.1016/j.jeph.2024.202804

Luc Dauchet ^1,², Raphaël Bentegeac ^1,², Haress Ghauss ^1,², Marc Hazzan ⁴, Patrick Truffert ³, Philippe Amouyel ^1,², Victoria Gauthier ^1,², Aghiles Hamroun ^1,^2,^⁎
¹ Service de Santé Publique, Epidémiologie, Economie de la Santé et Prévention, CHU Lille, 59000 Lille, France
² UMR1167 RIDAGE, Institut Pasteur de Lille, Inserm, Univ. Lille, CHU Lille, 59000 Lille, France
³ Service de Néonatalogie, Hôpital Jeanne de Flandres, Univ. Lille, CHU Lille, 59000 Lille, France
⁴ Service de Néphrologie, Dialyse, Transplantation rénale et Aphérèse, Hôpital Claude Huriez, Univ. Lille, CHU Lille, 59000 Lille, France

^⁎Corresponding author: Dr Aghiles Hamroun, MD, PhD, Permanent address: Service de Santé Publique, Epidémiologie, Economie de la Santé et Prévention – Maison régionale de la recherche clinique, CHU Lille, 6 rue du Professeur Laguesse, 59000 Lille, France.Service de Santé Publique, Epidémiologie, Economie de la Santé et Prévention – Maison régionale de la recherche cliniqueCHU Lille, 6 rue du Professeur LaguesseLille59000France

Sous presse. Manuscrit accepté. Disponible en ligne depuis le Wednesday 25 December 2024

Highlights

•	Script Concordance Tests (SCTs) which have been included in the 2024 French National Ranking Exams (EDN), assessing students’ clinical reasoning under uncertainty.
•	SCTs concept aligns with Bayesian probability, updating hypothesis likelihood based on new information, highlighting their educational value.
•	Key ambiguities include the specific clinical reasoning stage evaluated, appropriate Likert scale labels, and the suitability of expert panels as benchmarks.
•	Addressing these ambiguities could enhance SCTs' effectiveness, improve medical education, and offer a rich research avenue for refining clinical practice.

Le texte complet de cet article est disponible en PDF.

Abstract

Background

Script Concordance Tests (SCTs) represent an innovative assessment method which have been introduced in the 2024 French National Ranking Exams (EDN). These tests compare a student's clinical reasoning with that of an expert panel under conditions of uncertainty. Typically, the question involves the impact of new information on an initially proposed hypothesis, with responses given on a Likert scale.

Main findings

This article aims to illustrate didactically how SCTs align with probabilistic reasoning modeled by Bayes' theorem. Additionally, by comparing SCT writing recommendations with Bayesian reasoning concepts, several ambiguities were identified: (1) What stage of clinical reasoning do SCTs evaluate? (2) What are the appropriate labels for Likert scale responses? (3) Does the expert panel provide a relevant reference for SCTs?

Conclusions

Currently, many of these questions remain unanswered in the literature, with recent data suggesting that experienced physicians' responses to SCTs are often biased. Beyond their use as an assessment tool in the EDN, SCTs present a valuable opportunity to develop and deepen the teaching of probabilistic reasoning in medical education and serve as a potential research field to improve clinical practice.

Le texte complet de cet article est disponible en PDF.

Keywords : script concordance test, SCT, clinical reasoning, probabilistic reasoning, Bayesian reasoning