Learning Loss Interventions Are Falling Short on Implementation, Study Finds
School districts’ intervention efforts aimed at making up for significant learning lost during the pandemic are frequently falling short of their original expectations, according to a new study.
A report by the National Center for Analysis of Longitudinal Data in Education, or CALDER, found that interventions often aren’t meeting the scale or intensity of treatment that K-12 officials intended, despite districts having access to federal relief money.
What’s more, the study — co-authored by the assessment organization NWEA, Harvard University, and Dartmouth College — found that interventions had few statistically or practically significant effects on students’ math and reading test scores.
Districts are running into big challenges in implementation, according to the study, which looked at a dozen mid- to large-sized districts across 10 states. They include difficulty targeting students in need, engaging families, allocating staff, and making room in already packed student schedules.
“The implementation challenges district leaders recounted suggest that the simple-sounding logic of academic intervention — identify students in need and provide them extra support — belies a host of complex design and implementation decisions,” the study says.
This information should be met with urgency, said Dan Goldhaber, director of CALDER.
“How well we do as an economy and as a society in the future is tied to what kids are learning today,” he said in an interview. “If there is a generation of kids that end up well behind … all of our futures are diminished because those kids would have been able to contribute more to society had schools and the community and parents been able to ameliorate the learning loss.”
Unintended Inequities
How intervention programs are implemented varies not only across districts, but across schools within a single district, researchers found.
For example, when it comes to virtual learning tools focused on academic recovery, the study shows that some schools use them during core instruction time while others expect students to use them outside of school.
Or, when it comes to tutoring, some schools offer in-person sessions while others rely on virtual meetings.
In every case, districts need to hire staff to support the new efforts, which — amid national school staffing shortages — often delays the rollout of services.
I worry that if schools are not saying transparently to parents, ‘Your kid is really struggling,’ that parents may not know to engage.Dan Goldhaber, Director, CALDER
That includes the districts who turn to vendors to help with learning recovery. While companies providing staffing may help start a program sooner, some K-12 officials who participated in the study said the lack of direct oversight makes it difficult to ensure staff quality and consistency.
And while interventions are all generally intended to target students who are behind academically, there is also variation in how those students are identified.
In some cases, there is tension between district-mandated eligibility criteria and the professional judgements of school leaders and teachers, the study found.
But when districts respond by de-centralizing decisions about who receives supports — leaving it up to educator’s judgements — that opens the door for inequities to who has access, Goldhaber said.
Some teachers feel students are too far behind for where interventions start, the administrators who were interviewed in the study reported. Others are resistant to having students pulled from their class for interventions because it meant they miss core instruction.
And some students are forced to choose between interventions and other critical supports, such as ELL or special education services, when scheduling conflicts arise.
“It comes down to access,” said one program leader, who was unnamed in the report, “How easy is it to pull a student [from class] and bring them back?”
More Targeted Help From Vendors
In response to the findings, Goldhaber encourages K-12 officials to make changes in the areas within their control. Engaging parents by more clearly communicating that their children are struggling is one place to start, he said.
Teachers tend to be optimistic about a student’s potential to catch up over the long-term and may lean toward rewarding progress with good grades, which could muddy whether or not a student has grade-level skills, Goldhaber said.
“I worry that if schools are not saying transparently to parents, ‘Your kid is really struggling,’ that parents may not know to engage.” he said.
Most of the districts included in the study are already working on plans to overcome some of these implementation challenges, the report noted.
Other suggestions from the CALDER report include districts prioritizing hiring more administrative staff to oversee the rollout of these programs — a task which typically now falls on just a couple of people. The study authors also pointed to the benefits of seeking additional support from vendors that districts are already working with when possible, rather than moving to hire new ones, to help smooth implementation.
“For both vendors and the system itself, targeting individual student needs is really, really important,” Goldhaber said. “And that’s hard without a great deal of coordination between the vendor and classroom teacher.”
Follow EdWeek Market Brief on Twitter @EdMarketBrief or connect with us on LinkedIn.
Image by Getty.
See also: