Poster1-07: USING MOBILE DEVICES FOR INTRAOPERATIVE PERFORMANCE ASSESSMENT
Karan Grover, MD, PharmD, Amin Madani, MD, PhD, FRCSC, Charesa Smith, BA, Roman Nowygrod, MD, James Lee, MD; Columbia University Medical Center

Background: Surgical training programs are transitioning from the traditional Halstedian apprenticeship model to a competency-based model.  Nevertheless, there has been a paucity of effective methods to measure intraoperative performance, and current assessments are limited by either a narrow applicability or generic language prohibiting formative feedback. A need exists for a robust, comprehensive, universal, and objective assessment tool.

Objective: The objective of this study is to: 1) develop a novel assessment that comprehensively evaluates all intraoperative skills and is universally applicable to any procedure; 2) incorporate this assessment into a user-friendly, web-based, mobile application; 3) provide evidence for validity and feasibility for its incorporation into surgical training.

Subject Population: There are a total of 100 residents in 4 general surgery residency programs, all of whom will be invited to participate in this pilot study. 

Study Procedures: This is a nonrandomized, cross-sectional, prospective study approved by the Institutional Review Board.  Attending surgeons will rate surgical residents' operative skills (as either primary surgeon or first assistant) using the Intraoperative Performance Assessment Tool (IPAT) after any surgical procedure.  IPAT contains 41-items based on the previously published 5-axis intraoperative performance model, which includes: psychomotor skills, declarative knowledge, interpersonal skills, personal resourcefulness,  and advanced cognitive skills. Validity evidence will be obtained with respect to internal structure and relationship to other variables.  Scores will also be correlated to years of training and total case numbers for each procedure using the appropriate parametric or non- parametric statistical tests.  Internal structure will be evaluated by assessing inter-rater and intra-rater reliability on a random sample of 10 subjects using intra-class correlation coefficient. Feasibility of the IPAT will be evaluated by measuring time to completion and a questionnaire circulated to the raters and surgical trainees.

Significance: Competent surgeons must apply a complex fund of knowledge and skill to an endless variety of clinical situations. Feedback is the cornerstone for developing this level of professional expertise.  We explore better methods to objectively measure these skills to provide trainees with meaningful and longitudinal information.