ASE 2024 Abstracts

Objective: While previous research has acknowledged a range of influential selection factors in selecting residency programs, it has not provided a clear ranking of their relative significance. This study aims to evaluate the relative influence of 21 selection factors on the creation of a surgical program rank list among self-identified cisgender male and female applicants’ evaluation of surgical programs. Our hypothesis was that men and women aligned in their top 3 impact factors but diverged thereafter.

Design: We surveyed a convenience sample of cross-institutional residents from January 2023 to present. Our survey employed Best-Worst Scaling (BWS) to prioritize 21 established factors influencing residency program selection. BWS required respondents to select the most and least important factors from different item combinations. We conducted a comparative analysis to assess relative prioritization and segmented the data to examine factors prioritized by self-identifying cisgender male and female trainees. A secondary analysis compared responses between in-person and virtual interviewees.

Results: Eighty-eight surgical residents were surveyed. Of the respondents, 57 (65%) identified a cisgender women. “Program Reputation'' ranked as the third most influential factor for cisgender men, while cisgender women ranked it seventh. “Program Atmosphere,” “Interview Experience,” “Personal Experience with House Staff,” and “Career Development Opportunities” were equally important for all applicants, regardless of gender. Notably, both groups ranked “Family Factors” similarly (8th and 10th, respectively). Data stratified by interview type revealed that 33 (37%) respondents had in-person interviews, while 55 (63%) had virtual. “Family Factors'' and “Opinions of Friends and Family” ranked higher for virtual than in-person interviewees (8th vs. 14th and 11th vs. 16th, respectively).

Conclusion: Our results reveal distinct differences in how cisgender men and women assess training programs, even among their highest priorities. Both emphasize the importance of family factors, making it an optimal target for programs seeking to enhance residents' quality of life. Additionally, we uncovered differences in applicant program assessments between the pre- and post-virtual interview era. In the virtual interview era, family considerations were prioritized more often in the decision-making process compared to in-person interviewees. Further research may discern factors impacting applicant rank lists in this new digital landscape.

Introduction: Selecting amongst qualified applicants for a formal residency interview can be a difficult process. Standardized video interviews (SVI) have been introduced into the selection process by other medical specialties to evaluate applicants, and have been shown to be useful from a program perspective. However, the effects and perceptions of the SVI from the applicant standpoint is largely unknown.

Methods: In 2022, we included a three-minute standardized video interview as part of the residency application to aid the program in selection. Individuals applying to an integrated vascular surgery residency program who completed an SVI were invited to complete an anonymous 18-question survey that focused on the applicants’ experience with the SVI. Questions captured demographics and applicant perceptions on the process as well as its potential impact. Perception questions were captured on a Likert scale with the opportunity to provide qualitative feedback. 

Results: 45/48 (94%) applicants who submitted an SVI for our program completed the survey. 32/45 (71%) felt it could positively enhance their application. However, 25/45 (56%) felt it made no difference or hindered their chances of receiving an interview invitation. 37/45 (82%) found it at least mildly stressful, 14/45 (31%) disliked the SVI, and 11/45 (24%) stated they are less likely to apply to a program that utilizes this tool. Overall, 33/45 (73%) felt an SVI cannot replace the personal statement. 

Conclusions: While we as a program found the SVI to be a useful addition to the residency application, applicant feedback to the SVI was significantly negative. Applicants felt the SVI was stressful and, in some cases, hurt their chances of obtaining a formal interview. Applicants felt the SVI could not replace a personal statement. Programs considering implementing SVI into their evaluation process should think critically about how to make it a more positive experience for applicants.

Introduction:

The geographical distribution of general surgery residency (GSR) trainees across their career plays a pivotal role in shaping healthcare institutions. This study investigates GSR graduate’s (GSRGs) preference and mobility during training transitions (T) from medical school to GSR (T1), GSR to fellowship (T2), or GSR to their first attending physician position (T3).

Methods:

We analyzed 757 graduates from 42.4% (151/356) of ACGME accredited GSR programs that offered 2022 alumni data on their websites. Geographic proximity was measured using crows fly distance in miles. The US was divided into 6 regions: Northeast, Southeast, Midwest, Southwest, West, and Northwest. 26.9% (203/756) of GSRGs attended top 50 NIH-funded allopathic MSs (T50MS). Our analysis excludes international medical graduate’s in T1.

Results:

GSRGs on average traveled significantly more miles in T1: 1111 compared T2: 738 and T3: 629 (p<.001). A majority of GSRGs trained more than 200 miles away from their last institution T1: 67.1% (508/757), T2: 69.1% (430/622), and T3: 53.3% (72/135). 22.63% (86/380) of T50MS GSRGs stayed at the same institutions in T1 and T2 compared to 14.8% (204/1378) of nonT50MS. The average regional retention rate for GSRGs during all transitions was 50.10% (716/1429). The Northeast US region (USR) had the highest rate of retention during all transitions T1: 64% (135/211), T2: 53% (114/215), and T3: 64.3% (18/28).

Conclusion:

GSRGs demonstrate a trend towards distant relocation for initial training phases, with a subsequent preference for proximity in their final transition to attending roles, suggesting a balance between the pursuit of diverse training experiences and eventual settlement in familiar environments. The inclination of T50MS graduates to continue at their original institutions underscores the magnetic pull of prestigious educational centers. The Northeast's prominent retention rates may reflect its educational and professional appeal. These migration dynamics are crucial for shaping surgical education, workforce distribution, and healthcare accessibility.

Background

The advent of large language model-based applications that produce text, popularly called artificial intelligence authorship (AIA), has provoked controversy throughout the educational spectrum.  Personal statements on medical students’ application to residency are a likely target for AIA use. Personal statements have been the topic of reports on content, structure, strategies to influence ranking, and impact in comparison to other elements of the application. We hypothesized that AIA was being used to prepare personal statements for applications to general surgery training programs.

Methods

All applications to a large, urban based, academic affiliated general surgery residency program during the 2023-2024 application cycle were included.  Appropriate IRB and organizational permissions were obtained for this study.  Personal statements were extracted, blinded, and analyzed using commercially available artificial intelligence detector software (Originality.ai).  For each statement the detector determines a level of confidence that AIA was involved in authorship.  A level of confidence over 50% was considered indicative for use of AIA in creation of the personal statement. Additional items from the applications analyzed included location of the medical school (United States / Canada vs. other), gender, and USMLE Step 2 CK scores.  Appropriate statistical analyses were applied, p-value < 0.05 was considered significant. 

Results

Of the 1455 personal statements 66% (957) were considered to involve AIA.  There was a significant difference in AIA usage between medical students graduating from US medical schools compared to non-US medical schools (496/824=60% vs. 461/631=73%; p<0.05).  No correlation in AIA usage was seen in gender or Step 2CK scores. 

Conclusions

This novel study reports the extensive use of AIA in creating the personal statements of applicants for general surgery training. The role and impact of personal statements on match ranking is poorly understood with the situation now further confounded by AIA usage. It is most probable that AIA usage will grow. A national conversation is warranted to develop policy regarding the use of artificial intelligence authorship in the generation of personal statements by residency applicants.   

Background: 

Artificial Intelligence (AI) in the selection of residency program applicants is a new tool that is gaining traction, with the aim of screening high numbers of applicants while introducing objectivity and mitigating bias in a traditionally subjective process. This study aims to compare applicants chosen by an AI software to applicants chosen by single Program Director (PD) selection.  

Methods:  

A single PD at an ACGME-accredited, academic general surgery program screened applicants. A parallel screen by AI software, programmed by the same PD, was conducted on the same pool of applicants. Weighted preferences were assigned in the following order: personal statement, research, medical school rankings, letters of recommendation, personal qualities, board scores, graduate degree, geographic preference, past experiences, program signal, honor society membership, and multilingualism. Statistical analyses was conducted by chi-square, ANOVA, and independent two-sided t-tests.  

Results:  

Out of 1235 applications, 144 applications were PD-selected and 150 AI-selected (294 top applications).Twenty applications (7.3%) were both PD and AI selected for a total analysis cohort of 274 prospective residents. We performed two analyses: 1) PD-selected vs. AI-selected vs. Both and 2) PD-selected vs. AI-selected with the overlapping applicants censored (Table).

For the first analysis, AI selected significantly: more White/Hispanic applicants (P<0.001), less signals (P<0.001), more AOA honors society (P=0.016), and more publications (P<0.001). For the second analysis that censored overlapping PD and AI selection, AI selected significantly: more White/Hispanic applicants (P<0.001), less signals (P<0.001), more US medical graduates (P=0.027), less applicants needing visa sponsorship (P=0.01), younger applicants (P=0.024), higher USMLE Step 2 CK scores (P<0.001), and more publications (P<0.001).

Conclusions:  

Surprisingly, there was only a 7% overlap between PD-selected and AI-selected applicants for interview screening in the same applicant pool. Despite the same PD educating the AI software for particular preferences, AI software favored younger applicants from US medical schools with higher STEP 2 CK scores and more research, with a bias toward White/Hispanic applicants, compared with manual PD review. In its present state, AI may be utilized as a tool in resident application selection but should not completely replace human review.

Background

A notable disparity exists in representation of women and those Underrepresented in Medicine (URiM) in surgery across the United States. Pipeline Programs (PPs) in surgery have been developed to potentially combat this disparity. However, overall outcomes and best practices remain unknown. This systemic review and meta-analysis aimed to evaluate the outcomes of pre-medical pipeline programs in surgical departments. 

Methods

Adhering to the Preferred Reporting Items for Systemic Reviews and Meta-Analyses (PRISMA) guidelines, we conducted a comprehensive literature search of the databases: PubMed, Embase, Web of Science, Scopus, and ERIC from inception to 2022. Studies in English reporting on PPs for high school and college students within surgery departments were included.  Programs were included if they introduced students to the field of surgery through workshops, shadowing, research, etc. Abstract screening, full article review, and data extraction was conducted by three independent reviewers. The primary outcome was reported medical school matriculation for participants.

Results

Of the 941 studies screened, 13 were included in the systematic review, from which 10 were included in the meta-analysis. Most (60%) of studies reported on surgical PPs geared towards high school students and 40% towards undergraduates.  Of the 847 students in these programs, a majority of the participants were female (n=697, 87%) and 36% participants were from URiM backgrounds. The duration of the programs ranged from 1 day (50%), 8-10 weeks (40%), and 1 year (10%). A majority (80%) of studies reported on the number of students (n=233, 28%) interested in pursuing a career in healthcare after attending the PPs, however, no study reported whether students successfully pursued a career in healthcare. Ten studies (77%) reported on the programs’ curriculum and design, and 4 studies evaluated and reported on the improvement of participant’s surgical skills after hands-on sessions.

Conclusion

Premedical PPs have been implemented in order to increase the diversity in the surgical field. Our systemic review found that while studies reported an increase in interest of students in pursuing career in healthcare, data on medical school matriculation and career choices was lacking, which are required to understand and improve best practices.

In response to the pandemic, alterative virtual methods for interviews were established. At an institution offering both in-person and virtual interviews, we sought to understand applicant motivations for picking one setting over the other, and further, identify the associated financial burden.

We conducted a mixed methods survey of all US general surgery categorical applicants who applied to one institution in the 2023 cycle. The primary outcome was the factors that impacted applicant decision to interview in-person vs virtual. Secondary outcomes included estimated costs associated with the entire residency application process, including away rotations, interviews, and 'second looks'.

162 of 727 US applicants completed the survey (22.3%). The average number of interviews completed by this cohort was 14.2 ± 6.25. Females completed more interviews than males (14.9, 12.6, p=0.047). Respondents accepted a mean of 89% of virtual interviews offered. 73% of applicants who were given an option, elected to complete at least one interview in-person. Top reasons cited for electing in-person interviews were to gain a stronger impression of the program (64%), to make a greater impression at the program (50%), and strength of interest in the program (64%). Top factors for virtual interviews were decreased costs (64%), less travel time (53%), and the ability to complete more interviews (53%). 60% of applicants planned to visit at least one program in-person post-interview. Mean total cost of virtual interviews was $122 and each in-person interview was $575. 'Second look' visit travel cost $567. The cohort completed a median of one away rotation, with an average $3,230 per away rotation.

Applicants who had the option, chose to interview in-person because of a mutual feeling of portraying and receiving a stronger impression. Candidates chose virtual due to cost reduction and the ability to complete more interviews. The financial savings of choosing virtual interviews should be examined in the context of increased applications, in-person post-interview travel or "second look", and costs associated with away rotations. Future work is needed to elucidate the role of in-person contact or away rotations on the likelihood of an applicant to interview and match in a largely virtual interview era.