Measuring the Intangible: Approaches to Assessing Non-Technical Skills in Surgery
Session TypePanel
Yes
- Assessment, Curriculum, and Evaluation
Non-technical skills—including communication, leadership, professionalism, and systems-based practice—are critical for safe and effective surgical care. However, they remain challenging to teach, observe, and assess. While surgical education has traditionally emphasized technical skill acquisition, non-technical skills are increasingly recognized as essential for trainee development and patient outcomes. Competency-based frameworks, including the ACGME milestones and Entrustable Professional Activities (EPAs), offer structured ways to assess learner performance. Nonetheless, mapping abstract constructs like communication or teamwork to these frameworks remains a persistent challenge.
Format:
This panel, led by members of the ASE-ACE Committee, will consist of expert presentations followed by moderated discussion. Panelists will share their experiences, research findings, and best practices related to evaluating and mapping non-technical skills in surgical education. The session aims to foster an interactive dialogue among attendees, encouraging the exchange of ideas and strategies to enhance the assessment of non-technical skills necessary to graduate a competent resident or trainee.
Key Themes and Presentations:
(1) Defining Non-Technical Skills:
- An exploration of the core non-technical skills critical for surgical education, including communication, leadership, professionalism, and systems-based practice.
- Discussion of existing frameworks such as the SCORE curriculum, which teach four key non-technical skills: Situational Awareness, Decision Making, Communication, and Teamwork.
(2) Current Evaluation Practices:
- Examination of existing methods for assessing non-technical skills, including direct observation, simulation, formative tools, and faculty assessments.
- Case studies illustrating the application of these methods in various surgical training programs.
(3) Gaps in Existing Frameworks:
- Identification of non-technical skills that are not represented or under-represented in current EPAs and milestone mapping.
- Discussion of barriers to accurate measurement and strategies to address these gaps.
(4) Innovative Approaches to Assessment:
- Presentation of novel assessment tools and methodologies for capturing abstract skills more effectively.
- Strategies for integrating non-technical skills into competency-based evaluations, with a focus on mapping to ACGME competencies and EPAs.
Learning Objectives:
By the end of this session, participants will be able to:
- Define the core non-technical skills essential for surgical education.
- Identify current practices and tools for assessing non-technical skills in surgical trainees.
- Recognize gaps in existing frameworks and propose strategies to address these gaps.
- Implement innovative approaches to assess and map non-technical skills to ACGME competencies and EPAs.
Conclusion:
Through this discussion, attendees will gain insight into both the conceptual and practical challenges of integrating non-technical skills into surgical education. The panel will conclude with a synthesis of actionable strategies for aligning non-technical skill assessment with ACGME competencies and EPAs, emphasizing how programs can measure meaningful outcomes beyond the operating room.
This session will be of interest to a broad range of surgical educators, particularly those involved in resident evaluation and assessment. It will be especially relevant for faculty who serve on Clinical Competency Committees (CCC), Program Directors (PD), Associate Program Directors (APD), and others working within Graduate Medical Education (GME). Senior residents themselves will benefit from this session, as clearer definitions and standardized metrics for non-technical skills can help them understand expectations, enhance their learning, and inform teaching when they supervise more junior trainees.
Identify core, non-technical skills critical to surgical education, including communication, leadership, professionalism, and systems-based practice.
Assess current strategies and tools for evaluating non-technical skills in surgical trainees and their alignment with ACGME competencies and EPAs.
Differentiate and recognize gaps in existing assessment frameworks and recognize areas for improvement in mapping non-technical skills to milestones.
Develop strategies to implement innovative assessment approaches for non-technical skills within residency programs.
Integrate insights from expert panelists to enhance evaluation practices, benefiting faculty, CCC members, and residents alike.
Activity Order | Title of Presentation or Activity | Presenter/Faculty Name | Presenter/Faculty Email | Time allotted in minutes for activity |
---|---|---|---|---|
1 |
Welcome and Panel Introduction |
Tasha Posid |
Tasha.Posid@osumc.edu |
2 |
2 |
Introduction to Panel Topic (Non-Technical Skills) |
Baila Maqbool |
bmaqbool@salud.unm.edu |
3 |
3 |
Defining Non-Technical Skills: Core skills: • Communication, leadership, professionalism, systems-based practice • SCORE framework: situation awareness, decision making, communication, teamwork (+ 2 min Q+A by Moderators) |
Jennifer Cone |
jcone2@bsd.uchicago.edu |
10 |
3 |
Defining Non-Technical Skills: Core skills: • Communication, leadership, professionalism, systems-based practice • SCORE framework: situation awareness, decision making, communication, teamwork (+ 2 min Q+A by Moderators) |
Baila Maqbool |
bmaqbool@salud.unm.edu |
0 |
5 |
Gaps in Existing Frameworks • Underrepresented skills: EPAs, milestones • Measurement challenges: barriers and solutions (+ 2 min Q+A by Moderators) |
Minna Wieck |
mmwieck@health.ucdavis.edu |
10 |
5 |
Gaps in Existing Frameworks • Underrepresented skills: EPAs, milestones • Measurement challenges: barriers and solutions (+ 2 min Q+A by Moderators) |
Nina Zhao |
nxz212@case.edu |
0 |
6 |
Innovative Approaches to Assessment • Novel tools: capturing abstract skills • Integration strategies: competency mapping, ACGME alignment (+2 min Q+A by Moderators) |
Dandan Chen |
dchen43@mgh.harvard.edu |
10 |
6 |
Innovative Approaches to Assessment • Novel tools: capturing abstract skills • Integration strategies: competency mapping, ACGME alignment (+2 min Q+A by Moderators) |
Ellen Hagopian |
Ellen.Hagopian@UToledo.Edu |
0 |
4 |
Current Evaluation Practices • Assessment methods: observation, simulation, formative tools, faculty • Case examples: program applications (+ 2 min Q+A by Moderators) |
Nina Zhao |
nxz212@case.edu |
10 |
4 |
Current Evaluation Practices • Assessment methods: observation, simulation, formative tools, faculty • Case examples: program applications (+2 min Q+A by Moderators) |
Tasha Posid |
Tasha.Posid@osumc.edu |
0 |
7 |
General Discussion: • Cross-topic synthesis, takeaways, key points • Application ideas • Next steps • Resources |
Amanda Cooper |
acooper2@pennstatehealth.psu.edu |
13 |
8 |
General Discussion: • Cross-topic synthesis, takeaways, key points • Application ideas • Next steps • Resources |
Fariha Sheikh |
fsheikh003@gmail.com |
0 |
8 |
General Discussion: • Cross-topic synthesis, takeaways, key points • Application ideas • Next steps • Resources |
Jenny Guido |
Jenny.guido@sanfordhealth.org |
0 |
8 |
General Discussion: • Cross-topic synthesis, takeaways, key points • Application ideas • Next steps • Resources |
Jennifer Cone |
jcone2@bsd.uchicago.edu |
0 |
9 |
Closing Remarks |
Tasha Posid |
Tasha.Posid@osumc.edu |
2 |