Podium IIIC - Potpourri
(S060) CASE LOGS PROVIDE DETAILED INSIGHT INTO STUDENT ENGAGEMENT IN THE SUB-INTERNSHIP AND REVEAL OPERATIVE DIFFERENCES IN SETTINGS DEVOID OF RESIDENTS
Ali Schroeder1, Anjali Kumar, MD, MPH1, Tamir Bresler, MD2, Philip Bohlmann, MD3, Sean Morton, MD4; 1Washington State University College of Medicine, 2Department of Surgery, Los Robles Regional Medical Center, 3Department of Surgery, Oregon Health and Science University, 4Department of Surgery, Bassett Medical Center
Introduction:
This study evaluates a case-log assignment to assess the range of surgical experiences encountered by 4th-year medical students during their surgical sub-internships at a distributed-model, community-based medical school.
Methods:
We designed an assignment of a case-log for 4th-year medical students enrolled in surgical sub-internships. The form included drop-down entry options for operative area (by ACGME grouping category) surgical approach, and level of student involvement (ranked in 5 categories from observation to 75-100% hands-on-instruments time). Additionally, students had the option of free text narratives about the most memorable aspects of the case. Four weeks of data were collected by each student over three academic years (AY). AY21-22, 22-23 and 23-24 (data collection is ongoing). Analyses were performed by chi-square where significance was set at p<0.05.
Results:
Logs from three cohorts, comprising 50 students, resulted in 2,798 entries: 1,298 (46%) operations, 727 (26%) consultation rounds, and 222 (8%) clinic visits. Due to the rotation location and the paucity of residency training programs in our state, 62% of cases were logged without the presence of a resident (n=1741). Students demonstrated higher hands-on involvement (51-100% of time) more frequently in the absence of residents (p < 0.05). In contrast, when residents or other surgical assists were present (n=647), students were more likely to have lower levels of involvement (observation to 50%). Overall, the most common procedures were intra-abdominal (n=1,098) and integumentary (n=311). Weekend experiences represented 13.4% of all entries, consisting of 137 “Rounds or Consultation” logs and 100 operations. Students’ narrative reflections highlighted key learning experiences, including first encounters, notable observations, newly acquired techniques, and significant patient interactions.
Conclusion:
Case-log tracking serves as a valuable educational tool, revealing patterns in student exposure and engagement during surgical sub-internships. This study supports the use of real-time tracking platforms which are level-set for medical students (such as a modification of the SIMPL app) to enhance engagement and provide the course director detailed insight of the learning environment. 
(S061) EPAS HAVE ENTERED THE ROOM: AN ANALYSIS OF RESIDENTS' AND FACULTY MEMBERS' PERCEPTIONS
Julia Kasmirski, MD, Alfonsus A Harsono, MD, MSPH, Ivan I Herbey, MD, MPH, Andrea Gillis, MD, MSPH, Jessica Fazendin, MD, Britney Corey, MD, MACM, Brenessa Lindeman, MD, MEHP; University of Alabama at Birmingham
Background: Entrustable Professional Activities (EPAs) have emerged as a method of competency-based assessment in general surgery residency programs. While EPAs complement competencies and milestone assessments, identifying practical considerations for faculty and resident participation is needed to secure buy-in of the framework. This study aims to determine facilitators and barriers cited by trainees and faculty members to using the general surgery EPAs.
Methods: Sixteen semi-structured interviews were conducted with eight residents and eight faculty members at a single general surgery program to assess perceptions regarding aspects of the EPA framework, including barriers, concerns, overall experiences, approach to mentoring, interpersonal conflict, quality, and quantity of feedback. Transcripts were inductively coded by two independent researchers with agreement on themes and sub-themes using the NVivo software.
Results: Faculty had an overall positive perception about EPAs, sharing that they were well-informed of the method. They perceived that EPAs provided more granular data about trainees’ performance when compared to traditional feedback, moving surgery toward higher quality training. Faculty did not believe that EPAs impacted their interprofessional relationships or mentorship of trainees. The most cited faculty pitfalls were forgetfulness and lack of time to complete evaluations. Faculty, in general, were unaccustomed to accessing trainees’ previous EPA evaluations. Similarly, residents viewed EPAs positively and they were provided with adequate training in how to implement them. Improvement points noted by residents were gathering more faculty buy-in, more and higher-quality narrative feedback, and more frequent assessments. Residents expressed frustration at the lack of EPAs that cover all their activities and the potential negative repercussions that evaluations may have on delaying graduation and career progression.
Conclusion: Compared to traditional feedback, faculty had an overall positive impression regarding EPAs, especially to foster higher-quality surgical training, though they cited forgetfulness and lack of time as barriers. Resident physicians desired more frequent and detailed feedback while fearing the potential implications of this method on their ability to graduate and succeed as practicing general surgeons. Buy-in may be increased through providing dedicated time to complete EPAs and updated messaging on how EPAs will be used for certification.
(S062) THE OFTEN-FORGOTTEN NON-INTRAOPERATIVE EPA; HOW A SIMPLE INTERVENTION LED TO 15% INCREASE IN COMPLETION
Austin Dixon, DO1, Matthew Zeller, DO1, Stephanie Chairs, MBA1, Chris D'Adamo, PhD2, Marcie Feinman, MD, MEHP, FACS1; 1Sinai Hospital of Baltimore, 2University of Maryland School of Medicine
Introduction:
The general surgery community’s experience adopting Entrustable Professional Activities (EPAs), while rewarding, has proven challenging. As residents and faculty alike work to integrate EPAs into their daily workflows, the intraoperative phase is often the sole focus. This project aims to increase awareness and completion of the non-intraoperative phase EPAs (namely pre-operative and post-operative), a challenge not yet explored in the literature.
Methods:
Through conversations with attendings and residents on EPA completion, three areas of need were clarified: What platform does one use to complete an EPA? Which EPA relates to which attending/service? Where/when to elicit pre/post-operative feedback? A poster was developed that contained two main elements: QR code directly linked to the SIMPL platform and a list of all 18 general surgery EPAs crossmatched with their appropriate attending/service. The poster was duplicated and placed throughout all resident staffed clinics.
Results:
After just three months (July – Sept 2024) utilizing this poster intervention, there was a 15% increase in non-intraoperative EPAs (8% to 23%, pre to post-intervention of all completed EPAs, p < 0.05) when compared to the entire AY23-24. Out of our resident cohort of 19, 14 residents completed at least two non-intraoperative EPAs to date this academic year compared to only three residents during AY23-24. Lastly, we are on track to nearly quadruple our EPA total from the previous year, already achieving 93% of last year’s EPA total after three months (195 vs. 210).
Conclusion:
Our customizable poster template can be utilized by any general surgery program hoping to increase their EPA awareness and completion. While we identified resident staffed clinic spaces as best location for our posters, we envision similar utility with poster placement near resident work rooms, perioperative surgeon lounges, and attending office corridors. In the instance a platform outside of SIMPL is used, the QR code can easily be adjusted to link to the EPA collection platform of one’s choice.
(S063) IMPLEMENTING ENTRUSTABLE PROFESSIONAL ACTIVITIES IN PLASTIC SURGERY RESIDENCY TRAINING
I Falade, C Yim, E A Kim; University of California - San Francisco, Department of Surgery, Division of Plastic and Reconstructive
Introduction: Entrustable Professional Activities (EPAs) offer a competency-based framework in which trainees are progressively entrusted to perform tasks with varying levels of supervision as they demonstrate competence. By facilitating direct observation, EPAs enable residents to assess their proficiency in performing specific tasks and procedures across different training stages. EPAs have been adopted by several surgical specialties, but no formal plastic surgery (PS) EPA curriculum exists. This study describes the design and first implementation of an EPA assessment tool in a PS residency program.
Methods: An EPA framework was developed based on Olle ten Cate’s principles, aligned with Accreditation Council for Graduate Medical Education milestones, and drawing from the American Board of Surgery’s general surgery EPA guidelines. Entrustment levels were categorized from limited participation to practice-ready, with competencies organized into three phases: preoperative, intraoperative, and postoperative. Each EPA contained 4-5 questions and underwent review by residents, directors, and educators with EPA experience. The framework was intentionally generalized to cover a wide scope of PS cases. Badges with QR codes linking to assessments were distributed to faculty and residents, enabling immediate feedback. After faculty evaluations, residents received self-assessment prompts. These combined scores generated a resident specific overall entrustment level.
Results: Since its launch in July 2024, adoption has been steady, with 20-25 EPA assessments completed each month. Approximately 90% of these assessments were intraoperative phase with senior residents evaluated more frequently due to their higher surgical case volumes. Recognizing that residents often perform specific portions of a procedure based on their level, "Nested" EPAs were developed. Senior residents commonly supervised junior residents, leading to assessments where seniors evaluated juniors under their supervision.
Conclusions: While core EPAs by the American Board of Plastic Surgery (ABPS) is anticipated, our findings underscore the value of institution-specific EPAs, nested EPAs, and peer-assessments in enhancing residency education. Early feedback has led to more structured performance reviews, clearer expectations, and the identification of areas for improvement. These preliminary insights may inform the future development of core EPAs by ABPS and potentially serve as a foundation for multi-center studies.
(S064) COMPARISON OF THE QUALITY OF FEEDBACK FROM FACULTY AND SELF-REFLECTIONS FROM RESIDENTS ON GENERAL SURGERY ENTRUSTABLE PROFESSIONAL ACTIVITIES
Poojha Prabaharasundar1, Andrew Krumm2, Faith Ocoko1, Nancy Ly3, Sarah Jung1; 1University of Wisconsin-Madison, 2University of Michigan, 3Northwestern
Objective: Quality feedback is essential for the training and development of excellent surgeons. Coupled with this, medical education is experiencing a paradigm shift towards competency-based training. The successful implementation of the Entrustable Professional Activities (EPAs) assessment framework into many surgical training residency and fellowship programs allows for a broad focus on utilizing not only quality feedback but also trainees’ self-reflections as tools to ensure readiness for autonomous practice. This study comprehensively investigates the quality of EPA feedback and self-reflections for General Surgery (GS) EPAs implemented within a GS residency program.
Methods: A mobile application, SEPA, designed by a Midwestern academic department of surgery was used by the residents and faculty members from the department at a single tertiary care center. Entrustment levels and free text feedback were collected from July 2018 to November 2021 from four GS EPAs, right lower quadrant pain, gallbladder disease, inguinal hernia repair, and care for a traumatically injured patient. We applied a previously developed natural language processing (NLP) model that was validated using SIMPL OR narrative feedback to generate probabilities that an instance of feedback in SEPA could be classified as “high quality.” We then analyzed predicted probabilities by different features of an EPA: rater, procedure, perioperative stage, and autonomy level earned by trainee.
Results: The quality of feedback and self-reflection did not vary between faculty and resident ratings. However, feedback quality did differ based on perioperative stage for which feedback was given and the level of autonomy at which the resident was assessed. Intraoperative feedback was of higher quality than pre- and post-operative feedback. Feedback based on autonomy level showed a rainbow shaped curve as shown in Figure 1, with the lowest and highest autonomy levels receiving the lowest quality feedback.
Conclusions: Residents’ self-reflections and faculty’s feedback on behaviors related to EPAs were similar in quality. To improve the equal and consistent provision of quality feedback to resident trainees, this study's results suggest that focus must be placed on how to improve the quality of pre- and post-operative feedback and feedback given to trainees assessed at the lowest and highest levels of autonomy.
(S065) FEASIBILITY OF A MUTUAL INTRAOPERATIVE EVALUATION SYSTEM FOR RESIDENTS AND ATTENDINGS IN JAPAN: A PILOT SINGLE-CENTER STUDY
Keita Ishido1, Saseem Paudel2, Yuka Takakuwa1, Yuma Aoki1, Shota Kuwabara1, Kazuyuki Yamamoto1, Yasuhito Shoji1, Akira Fukuanaga1, Tatsunosuke Ichimura1, Hiroto Manase1, Satoshi Hirano2; 1Asahikawa Red Cross Hospital, 2Department of Gastroenterological Surgery 2, Hokkaido University
Background: While assessments of attending surgeons are integrated into surgical training systems in North America, such evaluations are rarely conducted in Japan. Furthermore, assessments of trainees typically emphasize technical skills, often neglecting non-technical skills (NTS). The objective of this pilot observational study is to implement a mutual intraoperative assessment system between residents and attendings, with a focus on NTS, to assess the educational environment in the operating room (OR).
Methods: An assessment tool incorporating both technical and non-technical skills was developed based on the Non-Technical Skills for Surgeons (NOTSS) framework. Assessment items included "Surgical Skills," "Knowledge," "Teamwork," "Communication," "Attitude," "Feedback," "Professionalism," and "Safety Management," each rated on a 4-point Likert scale. Residents and attendings evaluated each other for every surgery in which the resident participated, with data recorded on paper-based forms. The first half of the residents' rotation was compared with the second half to measure educational impact. A survey of participants was conducted to assess the system's impact.
Results: The study period spanned from June 2023 to March 2024, involving 278 surgeries and achieving a 99% response rate. Eleven residents (PGY1: 10, PGY2: 1) and ten attendings participated in the evaluations. Residents showed significant improvement in scores for "Surgical Skills" and "Knowledge" in the latter half of the study (p < 0.01, p = 0.01). However, no significant improvement was observed in NTS. In the residents' assessment of attendings, no significant differences were found. While 100% of residents reported satisfaction with the evaluation system, 18.2% felt burdened by the process.
Conclusion: This study successfully implemented a mutual assessment system between residents and attendings, demonstrating its feasibility in the OR setting. Although the system facilitated technical skill improvement for residents, it did not significantly enhance NTS. Future studies should explore methods to optimize NTS training while minimizing the burden on participants.