The impact of COVID-19 on School of Medicine students’ performance: an interrupted time series study

Background The COVID-19 pandemic has profoundly challenged the whole educational system. Shifting towards emergency remote teaching (ERT) has raised questions about its quality and effectiveness, especially for those university students enrolled on courses including interactive learning and clerkship modules as well. Therefore, we employed an interrupted time series analysis, to assess the impact of lockdown (March 2020) and subsequent shifting to ERT on exam enrollments and performance for students at the School of Medicine of the University “Federico II” of Naples, one of the largest universities nationwide, between January 2017 and February 2021. Methods We extracted data from 49 course modules within 14 courses of study offered by the School. Multilevel linear models were employed to account for the hierarchical structure of the data (multiple exam sessions within each module). Model covariates were time, term, course of study, and gender. Results In the summer term 2020 there was a 23% increase in students enrolled on exams, as compared with the previous year (2019: 16,237;2020: 20,091), with a 0.92% (SD 0.04) pass rate and an average mark of 26.39 (SD 2.64), as compared with the 0.89% (SD 0.78) pass rate and an average mark of 26.50 (SD 2.55) for the previous year. Considering the whole study period, we found that for the winter term following the national lockdown (March 2020) there was a 0.38 increase in the pass rate (coeff. 0.38;95%CI 0.04,0.71), as compared with the period before the lockdown, which however was not sustained over time (slope change: -0.04;95%CI -0.09, 0.00). No differences were found when considering changes in the average exam mark (step change: -1.38;95%CI -2.83,0.07, slope change: 0.03;95%CI -0.01,0.06). Conclusions Although students promoted the online examination method as part of the ERT considering the increase in enrollment and the initial increase in pass rate, this change was not sustained by an improvement in their performance. Key messages The shifting toward emergency remote teaching as educational system response to COVID-19 in Italy has increased exam enrollment but not students’ performance at School of Medicine. Ensuring high quality of teaching during emergencies is fundamental. Mixed-methods follow-up studies should be conducted to evaluate students’ perception and identify ways for quality improvement.