A new report questions the validity of the findings in The Efficiency Index, a study released earlier this month that ranked the U.S. 19th out of 30 countries in the outcomes it gets from its investment in education.

The latest study, published Tuesday by the Think Twice think tank review project of the National Education Policy Center at the University of Colorado Boulder School of Education, challenges the methodology used for the index, and its conclusions.

Professor Clive Belfield, an associate professor of economics at Queens College in New York whose research focuses on resource allocation and cost-effectiveness, reviewed the original study.

For the index, test scores from the Programme for International Student Assessment (or PISA) were used to measure educational outcomes, and each country’s efficiency was captured by using an econometric technique called stochastic frontier analysis, correlating test scores with educational inputs. 

Only two factors in the statistical model were found to have an impact on PISA results: changes in teachers’ salaries and class size. The United States would have to reduce teacher salaries about 5 percent below their current average of $41,460 and increase class size by 10 percent–to nearly 17 students per teacher–to be optimally efficient, the researchers said.

However, Belfield writes that the report’s methods are questionable, saying that “each of its three main assumptions—on outputs, inputs, and how they are modeled—is open to serious challenge.”

On outputs, Belfield questions whether PISA test scores capture the quality of an entire country’s education system. He is similarly concerned about whether the salaries of teachers with 15 years’ experience are the best gauge of an input, rather than the teachers themselves, and the use of a teacher-student ratio as an “input,” rather than simply an indicator of how many pupils there are per instructor. He also wrote that the econometric method by which they are correlated “does not have a straightforward economic interpretation.”

In his conclusion, Belfield said The Efficiency Index “may satisfy an apparent keenness for reports that rank countries … But it will not help advance our understanding of how to make the provision of education more efficient.”

The Review of The Efficiency Index report is available here.