The Russian “Monitoring of Efficiency of Educational Organizations”: Consistency, validity and unintended consequences

Since 2006, Russian government made energetic attempts to boost academic performance of the country’s universities, to shut down underperforming institutions and to turn the survivors into world-class schools. Overall, the organizational population of the Russian HEIs shrank by some 40% as a result of these efforts. Since 2012, Russian policies in the higher education sphere were largely directed by the results of the Survey of efficiency of higher education organizations (Monitoring effectivnosti deyatelnosti orgnizatsij vysshego obrazovanija) – a set of quantitative performance indicators used to sort efficient from non-efficient organizations. In this paper we use statistical methods to, first, evaluate the consistency and reliability of the Survey as a measurement instrument; second, look into the structural factors which predict certain schools’ success in the Survey, and third, discuss some conditions such system is to meet to evaluate “quality” in a highly stratified university population and analyze the consequences of implementation of systems not meeting these conditions.