PS2-04: OBJECTIVE ASSESSMENTS OF MEDICAL STUDENTSâ COMMUNICATION AND RELATIONSHIPS WITH NON-TECHNICAL SKILLS
Yuhao Peng1, Jackie Cha1, Nicholas Anton, MS2, Tomoko Mizota, MD2, Julie Hennings, MSN, RN, CCRN2, Ryan Stambro3, Katie Stanton, MD2, Dimitrios Stefanidis, MD, PhD2, Denny Yu, PhD1; 1Purdue University, 2Indiana University School of Medicine, 3Indiana University Health
Introduction: Communication and non-technical skills (NTS) have been shown to impact clinical performance and outcomes. However, current methodologies for assessment depend on expert observers or self-report measures; both are limited subjectivity and are subjected to bias. Objective sensors can provide an automated approach to assessing NTS. This study hypothesizes that speech features extracted from audio sensors provide objective metrics that predict NTS scores.
Methods: Fifty-seven third-year medical students at a large academic hospital provided written consent and participated in acute care team simulations (ACTS). Each student assumed the physician in charge role in one of six randomly assigned ACTS scenarios. During these ~10-minute scenarios, each student performed a focused patient assessment and developed a differential diagnosis and treatment plan. The patient was simulated by a manikin, and a nurse confederate played the nurse role during scenarios. Students’ NTS was assessed by two raters using a NT assessment tool on NTS constructs (leadership, communication, etc.). Overall NTS score was calculated as the average of all NTS constructs. A lapel microphone recorded audio for each student, nurse, and patient. Recordings were processed into audio metrics (i.e., speech duration, ratio, intensity, rate, and number of communications with nurse and patient) using PRAAT software. Pearson’s correlation and multiple regression analyses were conducted to identify the relationship between sensing metrics and NTS ratings.
Results: Overall, students’ speech duration and intensity correlated significantly with the NTS constructs of decision making (r=0.29, p=0.027) and cooperation (r=-0.30, p=0.021). The number of communications from student to nurse/patient (Figure 1) were also correlated significantly with overall score (r=0.40, p=0.002). The relation of communication metrics with the overall NTS score using a multiple regression model yielded an r2 of 0.21.
Figure 1. Relationship of number of communications and average NT skills score.
Conclusion: These findings suggest communication metrics are correlated with gold-standard NTS approach with raters. Audio features showed that high NTS ratings resulted when team members communicated more frequently. Further study is needed to integrate objective communication measurements into training feedback and NTS in future physician training, to determine clinical performance, and to explore generalizability to other clinical settings.