New Analysis Compares Costs, Effectiveness of K-3 Math Products
Noodle Markets Report, First in a Series, Delves Into Math, With ELA to Come
An ambitious new analysis uses district purchasing information, demographic data, and test results in an effort to create an easy-to-use tool for comparing what curricula K-12 systems are buying, what they’re paying for it, and academic outcomes.
The report by Noodle Markets is the first in a series of comparisons the school procurement platform says…
Sign up for Market Brief updates
To continue reading this story, please provide your email address. We will send you updates about new content that will interest you.
An ambitious new analysis uses district purchasing information, demographic data, and test results in an effort to create an easy-to-use tool for comparing what curricula K-12 systems are buying, what they’re paying for it, and academic outcomes.
The report by Noodle Markets is the first in a series of comparisons the school procurement platform says it will produce monthly.
Noodle Markets officials want K-12 officials to use its research to make comparisons and inform buying decisions.
Whether “The Noodle Market Reports” will gain traction in the school community or private sector remains to be seen. But the project comes as more organizations are trying to share purchasing and product information across districts, in the belief that those metrics will improve decisionmaking and bring transparency to the process.
The first analysis, released exclusively to EdWeek Market Brief, pairs data about K-3 mathematics curriculum purchases in more than 1,000 districts with 3rd grade math assessment scores in the hope of providing guidance for district leaders.
The aggregated information also seeks to provide an indicator of the market share of the companies that sell those products, and an indicator of specific products’ relative cost, as well as whether those products are trending up or down in purchasing activity over the seven-year period of the study.
The first Noodle Markets Report examines 20 K-3 math curriculum products that 1,094 districts bought from 2008 to 2015, according to purchase orders obtained through SmartProcure, a company that offers a searchable database of government orders for buying that covers about 60 percent of K-12 purchasing.
It found that Houghton Mifflin Harcourt, Pearson, and McGraw-Hill account for 85 percent of the market share for K-3 math curriculum. “Purchase data tells us there is ebb and flow in product market share,” the report says. “Relative cost is not always a factor.”
Data on District Demographics
The Noodle Markets analysis also incorporates socioeconomic information, in an attempt to show what products were purchased most often in communities with different income levels.
It finds that districts with large numbers of low-income families—where the median income is $18,000 to $50,000—bought Math Expressions Common Core; GO Math; Harcourt Math; Math in Focus; Singapore Math, and Bridges, while districts with a greater proportion of middle income households—with family incomes of $50,001 to $70,000—were most likely to purchase Envision Math; Math Expressions Common Core; GO Math; Growing with Mathematics and Math Triumphs.
Districts in the highest-income areas, where the median incomes were $70,000 to $222,000, purchased My Math; Ready Mathematics, Envision Math, Math Expressions Common Core, and Harcourt Math.
The authors say the analysis will provide districts with valuable information about purchasing across K-12 systems, and about student outcomes—even if test-score results can’t be definitively tied to specific curricula.
“While this research cannot conclude direct causal relationships between products and student achievement for purchasing leaders, it’s worth looking at what products might best serve a particular student population, especially considering socioeconomic status,” the report concludes.
Noodle Markets’ hope is that, by sharing the data with districts, K-12 leaders will be able to identify products that might best meet their needs, and consider them for pilots or purchases.
“When we started getting deep into the work, we realized it is not just about the efficiencies we can create, and the time and money districts can save,” said Nicole Neal, the CEO of Noodle Markets, a company focused on bringing transparency to the K-12 procurement process. “We wanted to bring out data that is currently buried for K-12 decisionmakers.”
The three considerations that drove their inquiry were: helping educators to see what products are popular among districts like theirs; an understanding of relative pricing of products, and what the correlation to student performance might be.
A school leader, for instance, might want to identify high-performing districts of similar size and demographics, to see what products they’re using.
Sharing Market Share Data
Knowing what products specific school districts are using can help competing companies “trying to break through in this textbook, OER [open-education resources], and digital space,” Neal said. “If I’m a small textbook provider in a state that’s not an adoption state, it gives me information about how I can sell into particular types of districts that my competitors are winning in.”
Researchers and industry representatives said they found value in the data—but also pointed to its limitations.
“I found the market share information to be the most interesting of everything” in the report, said Steve Leinwand, the principal research analyst for math at the American Institutes for Research, a nonprofit organization that performs basic and applied research.
However, he cautioned about drawing conclusions on the effectiveness of the products based on the 3rd grade test scores.
“They’re taking the database of what you bought and connecting it to achievement levels, when there are so, so many intermediate variables that are not being addressed,” he said.
Among Leinwand’s concerns was the fact that the performance metrics were based only on 3rd grade assessments, but the products span K-3. “You don’t really learn about the bridges being used” from one grade to the next, he said.
Another question:”So [the districts] bought a product. How did they use it?,” Leinwand asked. “It’s not what is bought; it’s what’s used.”
From the education industry standpoint, the report was hailed as a good conversation starter. Chris Lohse, the senior vice president and managing director of the education practice at the Software & Information Industry Association, said overall the report is “very interesting…and in many ways, it’s pushing the market in a positive direction.”
The idea of taking a look at different income bands will also be of interest to companies, Lohse said. If they have done any efficacy or impact studies, this research could convince them to see whether they have differentiated their findings by socioeconomic status, he said.
The study relies on a “convenience sample,” which means the information was what is readily available from SmartProcure’s database. While large districts like Long Beach, Calif. and Chicago are part of the study, the average district size is 9,648 students, and the average district percent of students receiving free and reduced-price lunches is 43 percent. Forty-six states are represented in the research.
“With all of the efforts around OER, and other factors, we think this list of top providers will absolutely look different in three years,” said Neal of Noodle Markets.
She said the next study will examine district buying of K-3 English language arts products.
Access the full report here.
See also: