Knowledge assessment: Squeezing information from multiple-choice testing

Knowledge assessment via testing can be viewed from two vantage points: that of the test administrator and that of the test taker. From the administrator’s perspective, the objective is to discover what an individual knows about a domain of interest. From that of the test taker, the challenge is to reveal what one knows. In this article we describe a procedure for administering and scoring multiple-choice tests that satisfies both of these objectives and we present experimental data that demonstrate its effectiveness. The method allows test takers to provide specific information about their confidence that each alternative for an item is the correct answer and makes guessing not only unnecessary but detrimental. From this information the administrator can derive measures of both knowledge and confidence, which, we argue, provides better estimates than systems that do not allow measurement of partial knowledge. The use of such measures for purposes of evaluation both of individual test takers’ knowledge of a subject of interest and of the effectiveness of instruction with respect to that subject is discussed.