By Liana Loewus
Cross-posted from the Curriculum Matters blog
A new website geared toward school administrators rates instructional programs for K-12 reading and math based on the criteria for judging their effectiveness that is laid out in the new federal education law.
Textbooks, professional-development models, computer-based curricula, tutoring programs, and other instructional methods and materials are designated as “strong,” “moderate,” or “promising” on the free site, which was created by a team of researchers at the Johns Hopkins University’s Center for Research and Reform in Education.
“My image of this is you’ve got a principal or a superintendent saying, ‘We’ve got a school where kids are not doing well in math. What can I do to improve math?'” Robert Slavin, the director of the center, said in an interview.
To label programs for the Evidence for ESSA site, the team used the research-base criteria from the Every Student Succeeds Act—the federal law signed just over a year ago. ESSA urges states and districts to use interventions that have been proven to help students learn. It recognizes three tiers for evidence:
- A program is considered as having “strong evidence” if at least one “well-designed and well-implemented experimental study” shows it has a statistically significant effect on improving student outcomes.
- It has “moderate evidence” if at least one “well-designed and well-implemented quasi-experimental” (or matched) study shows it improves student outcomes.
- And it has “promising evidence” if at least one “well-designed and well-implemented correlational study with statistical controls for selection bias” shows it works.
That’s a departure from the law’s predecessor, the No Child Left Behind Act, which simply called for using programs “based on scientifically based research.”
“We saw both enormous potential in the ESSA evidence standards and also enormous difficulty, because even though this is clearer about what strong evidence is, it isn’t clear enough for educators to really know what that means in terms of programs that a school or district might actually adopt,” said Slavin.
100 Programs So Far
The Johns Hopkins team combed through every study it could find on instructional programs. As of now, the website lists about 100 programs that fall within one of the three evidence categories. However, users can search almost any instructional program on the site. For those programs that aren’t deemed “strong,” “moderate,” or “promising,” the site will indicate that they either haven’t been studied or haven’t shown positive results.
One wrinkle in this sort of work is that different studies about a single program may have conflicting conclusions. Under the ESSA criteria, a program only needs one study with a positive outcome for the program to be considered evidence-based.
“I understand why they did it this way–they wanted to have a large number of programs qualifying, but there should have been some procedure to allow you to head off this particular problem,” Slavin said. “We’re going to learn from this process, but what we’re doing is following the law for what it is right now.”
The new site was developed in collaboration with more than a dozen key education organizations, including AASA, the National PTA, Chiefs for Change, and the National Education Association. It is funded by the Annie E. Casey Foundation.
The effort is somewhat similar to that of the What Works Clearinghouse, a federally run site that reviews existing research on educational programs and products. The standards differ, though, and Slavin notes that Evidence for ESSA will be updated more quickly than the WWC. “If we hear about a new study or program or a mistake, we’re going to have it up there in two weeks,” he said.
Another website, EdReports.org, has been getting attention recently for rating K-12 math and reading materials as well. But that group looks only at print and digital textbooks, and it uses an internal team to assess them for alignment to the Common Core State Standards, rather than reviewing outside research on their effectiveness.
Image: Screenshot from the Evidence for ESSA website
- Common-Core Materials Continue to Vary in Quality, According to Textbook Review
- Evidence-Based Programs: New Guidance Details What ESSA Means for Research
- Inside the Every Student Succeeds Act