For years, efforts to explore and compare the educational performance of impoverished countries—and by implication, their economic potential—have been stymied by a lack of useful data.

An ambitious new analysis by the World Bank aims to change that.

A “global dataset” unveiled by the international development organization uses statistical methods to put the results of much-publicized international tests like the PISA and TIMSS—which many poor nations do not take part in—on a comparable scale as regional exams commonly used by developing countries.

The result is a new method for comparing the test performance of rich and poor nations that World Bank researchers say hasn’t been accomplished before.

In a paper giving a detailed breakdown of the new tool, the authors describe it as the “largest and most current globally comparable dataset” in education, covering 90 percent of the world’s population, across 163 countries and regions.

The analysis by the World Bank, an international organization that aims to reduce poverty and promote economic growth, offers data on nation-by-nation performance over a half-century, from 1965 through 2015. The TIMSS and PISA data back only to the 1990s.

To date, “learning outcomes have not been measured in many developing countries, or the measures are quite poor compared to those of developed countries,” said Harry Anthony Patrinos, an economist at the World Bank and one of the authors of the dataset, in an interview.

That knowledge void is especially harmful for developing nations, because it leaves them with few options for benchmarking their academic progress, which is a key to driving future economic growth, Patrinos said.

He is one of the authors of the dataset, along with Nadir Altinok of the University of Lorraine, in France, and Noam Angrist, of Oxford University, who worked with the World Bank in creating it.

The new data, perhaps not surprisingly, shows an enormous learning gap between wealthy and impoverished nations. The top-performing students in many non-industrialized countries often perform worse than the lowest-achievers in developed nations, the World Bank found.

And overall, fewer than 50 percent of students in developing countries reach the minimum level of proficiency, compared with 86 percent in wealthier nations.

The authors use a series of research methods to link the test scores of countries taking part in the PISA and TIMSS with the scores of countries administering regional exams of their students’ educational progress. The regional exams the authors examined included the PASEC, taken by French-speaking countries in sub-Saharan Africa, and an exam arranged by the LLECE, a network of countries in Latin America and the Caribbean.

One of the authors’ methods is to use the test results of countries that take part in both international and regional exams as anchors to create comparability between the tests—and the nations taking part in them.

Ed. Policy Influences National Performance?

The World Bank analysis also provides a measure of changes in individual countries’ performance over time, from the 1960s to today, with some surprising results.

Mediocre U.S. test scores on PISA and TIMSS during recent testing cycles have fueled worries—overstated or not—that the western superpower is falling behind educationally. Yet according to the World Bank’s analysis, the United States’ overall performance made a “significant improvement” from 1975 to 1995, Patrinos says.

By contrast, one traditional high-flier on international tests, Finland, has seen its performance plateau somewhat after making strong gains in the 1970s and 1980s.

Another traditional power on international tests, Hong Kong, was performing at roughly the same level as Thailand through much of the 1980s, the World Bank data shows. But they diverged after that, with Hong Kong’s performance climbing and Thailand’s slipping, most likely a reflection of the latter country’s “two decades of unsuccessful reforms,” the authors say.

The nation-by-nation scores show that “transformational change in learning outcomes takes time,” said Patrinos, a practice manager for the World Bank’s East Asia and Pacific education team. It typically “won’t take place during a [single] government term,” but over many years.

Critics of the World Banks’ analysis have typically pointed to a number of limitations in the research, Patrinos said.

Some have argued that the analysis could be skewed because it is based solely on students who are enrolled in school, and who took the tests in question, when in fact many youth in impoverished nations are not in school. Another criticism is that there are not enough countries that serve as anchors for the results between tests. Still, others point out that the results are based solely on test scores—an imperfect proxy for the educational accomplishment of students and nations.

But Patrinos said the methods are robust. He acknowledges that academic progress is measured through more than tests scores. But he said test results are also a “fundamental building block” in tracking academic proficiency, and nations badly need that data.

Eventually, the World Bank researchers plan to put the dataset described in their paper on a website, where users can download data on countries’ performance, conduct searches, and perform their own analyses, Patrinos said.

Tough Measurement

The dataset is also being released as U.S. education companies increasingly seek to do business in foreign markets, including developing countries, a shift that in some cases is being accelerated by technology. In addition, ed-tech providers in developing countries are also seeking to create school products for their own markets, as evidenced by last year’s launch of Injini, which bills itself as Africa’s first ed-tech incubator program.

Eric Hanushek, a senior fellow at the Hoover Institution at Stanford University, said the new analysis is a move “in the right direction,” in its attempt to bring many more nations into a tent in which they’re being compared educationally by the same metrics.

“How can you think about judging the education [in different countries],” he asked, “if you don’t know where they are, and where they’re going?”

But he said the researchers are also trying to pull off a difficult feat, in comparing countries with vastly different levels of educational performance, on the same scale.

The value of the analysis, and how well it holds up, is likely to be seen over time with further research and greater test participation, said Hanushek, who has studied and written about the connection between countries’ education systems, their test scores, and economic growth.

“There’s substantial evidence that these test scores measure skills that are important economically” to nations, Hanushek said.

The World Bank’s research “will ultimately start to shape the policy debate in a number of these countries—or it should.”

Follow EdWeek Market Brief on Twitter @EdMarketBrief or connect with us on LinkedIn.

---

See also:

• U.S. Education Pressed by International Comparisons
• African Ed-Tech Incubator Aims to Set Companies, and Students, on Winning Path
• India’s Economy Is Soaring, But Breaking Into Its School Market Isn’t Easy