Synthesis of Validation Practices in Two Assessment Journals: Psychological Assessment and the European Journal of Psychological Assessment

The objective of this study was to conduct a research synthesis of validation practices based on the Standards for Educational and Psychological Testing (AERA, APA, and NCME, Standards for educational and psychological testing. American Educational Research Association, Washington, DC, 1999) in a random selection of 50 articles published in 2011 or 2012 in two premier assessment journals, Psychological Assessment and the European Journal of Psychological Assessment. Commonalities and differences in findings between the journals were identified. Reliability evidence was reported in similarly high percentages of studies from both journals, with internal consistency being the most commonly reported type of evidence. Internal structure and relations to other variables sources of validity evidence were strongly favored in our sample of studies, with little to no evidence presented related to test content, response processes, or consequences of testing. Weaknesses in validation practice and reporting are identified and recommendations for improvement are provided.