PS1-09: PILOT IMPLEMENTATION AND EVALUATION OF A NATIONAL QUALITY IMPROVEMENT TAUGHT CURRICULUM FOR UROLOGY RESIDENTS: LESSONS FROM THE UNITED KINGDOM
Elena Pallari, Ms¹, Zarnie Khadjesari, Dr², Shekhar Biyani³, Sunjay Jain³, Dominic Hodgson⁴, James Green⁵, Nick Sevdalis¹; ¹King's College London, ²University of East Anglia, ³Leeds Teaching Hospitals NHS Trust, ⁴Portsmouth Hospitals NHS Trust, ⁵Bart’s NHS Trust

Background: Teaching residents quality improvement (QI) skills is a training requirement and a means to improve care. In SEW2017 we reported the developmental phase of a national QI-curriculum for UK urology residents. Here we report a further multi-phased detailed evaluation of the educational impact of the pilot curriculum on residents’ knowledge, skills and attitudes in QI.

Methods: This was a prospective multi-phased pre/post-training evaluation, grounded onto the Kirkpatrick evaluation framework. Phase-1: assessment battery development: knowledge was assessed via standardised multiple-choice-questions (MCQs), derived from the Institute of Healthcare Improvement knowledge item bank, adapted for UK use. Phase-2: pre-post training assessments: skills, intentions and attitudes regarding QI were self-reportedly assessed by the residents – attitudes via adapted validated scales. Residents also reported their overall satisfaction with the course. Scores were statistically analyzed pre/post-training using SPSS 21.0; all scales were psychometrically assessed for internal consistency via Cronbach alpha coefficients.

Results: Demographics: The half-day QI skills training was delivered to 2 national resident cohorts (October 2017; January 2018; total N=98), across all 13 urology training regions of the UK. Internal consistency: most scales showed acceptable reliability (Cronbach-alphas pre-training=0.485-0.904; post-training=0.727-0.924). Knowledge: Baseline objective MCQ scores (58%) did not improve post-training (59%, p>0.05). Residents’ subjective knowledge improved significantly (Mpre=2.71, SD=0.787; Mpost=3.97, SD=0.546, p<0.0001).  Intentions: Intentions to initiate/engage in QI in subsequent 6-month period increased significantly post-training (Mpre=3.65, SD=0.643; Mpost=4.09, SD=0.642, p<0.0001). Attitudes: Attitudes improved significantly post-training (all ps<0.0001) – including attitudes to conducting QI projects (Mpre=3.67, SD=0.646; Mpost=4.11, SD=0.591); attitudes towards QI at work (Mpre=3.80, SD=0.511; Mpost=4.00, SD=0.495); and attitudes towards influencing QI at work (Mpre=3.65, SD=0.482; Mpost=3.867, SD=0.473). Satisfaction scores: Residents reported high satisfaction with training content and delivery (median scores for content, delivery and global satisfaction=4, on a 5-point Likert scale (5=strongly agree).

Conclusion: The study supports feasibility and immediate educational impact of our evidence-based and user-informed QI-curriculum for UK urology residents; and offers evidence on the psychometric suitability of an attitudinal assessment battery for QI skills. Objective knowledge assessment needs further development. Ongoing research is evaluating the impact of the curriculum in residents initiating QI projects; and its large-scale implementability in the UK.