Reliability and validity effects under mail survey conditions

Abstract In searching for means to reduce error in mail surveys researchers have focused largely on one component of total survey error—sampling error; means have been sought for its reduction by increasing mail survey response rates. The second component of total survey error, nonsampling error (e.g., response bias), has received relatively little attention. This study focuses on the second type of survey error through the development of indices of retest reliability and covergent validity. These indices were applied in a field experiment involving manipulations of survey administration conditions previously linked to response rate effects. The results indicate that in addition to influencing response rates, survey sponsorship, cover letter message, question order, and notification method can substantially affect the retest reliability and covergent validity of data obtained from mail questionnaires.