Assessing learner engagement with virtual educational events: Development of the Virtual In-Class Engagement Measure (VIEM) - 09/12/21

Doi : 10.1016/j.amjsurg.2021.09.021

K.J. Dickinson ^a,^b,^c,^⁎ , K.E. Caldwell ^d, E.A. Graviss ^b,^e, D.T. Nguyen ^e, M.M. Awad ^d, S. Tan ^f, J.H. Winer ^g, K.Y. Pei ^h
on behalf of the ASE Educational Technology Committee
^a Department of Surgery, University of Arkansas for Medical Sciences, Little Rock, AR, USA
^b Department of Surgery, Houston Methodist Hospital, Houston, TX, USA
^c Department of Interprofessional Education, University of Arkansas for Medical Sciences, Little Rock, AR, USA
^d Department of Surgery, Washington University in St Louis, St Louis, MO, USA
^e Department of Pathology and Genomic Medicine, Houston Methodist Research Institute, Houston, TX, USA
^f Department of Surgery, University of Florida Health, Gainesville, FL, USA
^g Department of Surgery, Emory University, Atlanta, GA, USA
^h Department of Graduate Medical Education, Parkview Health, Fort Wayne, IN, USA

^∗Corresponding author. University of Arkansas for Medical Sciences, 4301 W Markham Street, Little Rock, AR, 72205, USA.University of Arkansas for Medical Sciences4301 W Markham StreetLittle RockAR72205USA

Abstract

Background

The COVID-19 pandemic has necessitated virtual education, but effects on learner engagement are unknown. We developed a virtual in-class engagement measure (VIEM) to assess learner engagement in online surgical education events.

Methods

Using the STROBE, an observer collected tool to document student engagement, as a template an ASE committee workgroup developed the VIEM. The VIEM had two parts: observer assessment and learner self-assessment of engagement. Trained observers collected engagement data from two institutions using the VIEM. Surgical attendings, fellows and residents were observed during virtual learning events. Educator attitudes towards online teaching were also assessed via survey.

Results

22 events with 839 learners were observed. VIEM distinguished between sessions with low and high engagement. 20% of learners pretended to participate. Half of instructors were comfortable with virtual teaching, but only 1/3 believed was as effective as in-person. 2/3 of teachers believed video learners were more engaged than audio learners.

Conclusions

Virtual platforms do not automatically translate into increased engagement. Standard tools such as VIEM may help with assessment of engagement during virtual education.

Le texte complet de cet article est disponible en PDF.

Highlights

•	Increased virtual surgical education has been necessary during the COVID-19 pandemic.
•	The effect of a virtual platform on learner engagement is unknown.
•	Current observer led learner engagement tools assess in-person engagement only.
•	Virtual platforms do not automatically translate into increased learner engagement.
•	The Virtual In Class Engagement Measure may help educators assess virtual learner engagement and strategize to increase it.