Exploring item discrimination in an online self and peer assessment of interprofessional teamwork

Abstract Prior research on interprofessional education has used item response theory to demonstrate that existing measures of attitudes are not capable of differentiating between learners. However, current literature does not speak to the discriminatory abilities of existing self or peer assessments of team behaviors. The purpose of this study was to use item response theory to examine the validity of teamwork items associated with the Comprehensive Assessment of Team Member Effective (CATME) online system. Rating data derived from 2731 students who participated in a collaborative longitudinal interprofessional service learning experience between 2014 and 2017. Students participating in interprofessional teams provided peer and self-evaluations using the CATME online system twice during the nine month experience. We evaluated the data using a generalized partial credit model finding evidence of factor multidimensionality and item miss-fit across all six items comprising the assessment. While items could identify those who had limited teamwork ability, the items could not discriminate between individuals at or above the estimated population mean. These results provide evidence that the CATME peer and self-assessment items may be most useful as a single component of a multimodal assessment on interprofessional teamwork behaviors.