First Equals Most Important? Order Effects in Vignette-Based Measurement

A vignette typically describes a hypothetical situation or object which respondents are asked to judge. The object is described as having different dimensions, the values of which are experimentally varied, so that their impact on respondents’ judgments can be estimated. We examine 1) whether the order in which dimensions are presented impacts estimates, and 2) under which conditions order effects are mostly likely. Using data from a web survey of students we analyze several possible conditions features of the vignette design, characteristics of respondents, and interactions between these. Our results show that strong order effects can occur, but only when the vignettes are complex.