Marketplace K-12

Tech-Savvy Educators Often Use Vendors’ Info to Find Ed-Tech, But Doubt Their Research, Study Says

Michele Molnar
Associate Editor

The main way tech-savvy educators find out about ed-tech tools is from web searches, but nearly half of them also rely heavily on vendor reports for information on those products, according to a new survey.

The co-sponsors of the research—the International Society for Technology in Education and the Jefferson Education Exchange (JEX)—are working actively to change this dynamic.

In the study of 1,124 teachers, school administrators, district staff, and technology leaders, over half (57 percent) indicated that their local education leaders rarely or never discuss ed-tech research.

The respondents think research is important. In fact, three-fourths of them said it is part of their jobs.

When they talk about research, these educators generally do so under specific conditions. About 70 percent said it’s when they’re discussing tools recommended by colleagues or, collaboratively, during planning time (64 percent.)

Of those who reported being personally involved in procurement decisions, 65 percent said that they discuss ed-tech research with colleagues when working on a committee responsible for making procurement decisions about it, and 57 percent discuss such research with colleagues when seeking approval to use a new tool or product.

The respondents to the survey—from 50 states, D.C., and four territories—were more likely to be involved with ed-tech procurement than a general group of educators would be, according to Brandon Olszewski, director of research at ISTE, which distributed the survey with JEX to 18 organizations. That’s possibly because 60 percent of them are part of ISTE, and 28 percent from ASCD.

The strong representation of respondents who play a role in procurement “is both a limitation and strength of the survey,” Olszewski said, because a general population of educators is less likely to be involved with researching and purchasing ed-tech products. But since this sample did have a preponderance of those who are, the message from the research is: “They don’t have a place they can consistently go for evidence they consider reliable,” he said.

Dubious View of Vendor Research

Getting product information educators can trust is a challenge, according to the “Education Research Perspectives Survey,” which was fielded in May. More than three-quarters of respondents (76 percent) felt that ed-tech vendors were not qualified to conduct valid research about their products, yet 48 percent look at vendors’ sites to get information about those tools.

Interestingly, what researchers often consider the highest level of evidence from the U.S. Department of Education’s What Works Clearinghouse isn’t necessarily a “go to” source for them. Eighty-one percent of this “plugged-in” group does not regularly review research from the Institute of Education Sciences, the study found. The IES, part of the U.S. Department of Education, is often considered the “gold standard” of research, and its What Works Clearinghouse invites educators to “find what works” based on the evidence.

The overall findings echo elements of earlier research, which show a lack of willingness to seek out peer-reviewed research. In a study it commissioned three years ago, Google learned that educators are strongly influenced by their peers when they want information about ed-tech. Last year, another study found that only 11 percent of district administrators and teachers said they would outright reject buying or adopting an ed-tech product if it lacked peer-reviewed research behind it.

Educators—particularly those involved in procurement—believe that the organizations best equipped to conduct research are first and foremost, local school districts (67 percent), followed closely by research organizations (65 percent) and education non-profits (63 percent.)

“We continue to spend billions on these products without the information we need to make good decisions,” said Bart Epstein, the president and CEO of the Jefferson Education Exchange, in an interview about the survey. The survey’s findings “reinforce the need for new approaches to gathering and sharing reliable data about education technology,” he said in the release about it.

Without what they consider reliable information about ed-tech products, educators’ “number one source of data is peers, word-of-mouth,” said Joseph South, the chief learning officer for ISTE, in an interview. The result? Generally, ed-tech products “are chosen based on popularity, not efficacy,” he said.

South, like Epstein, called for a collaboration to fill the gap between what’s available to help schools make research-based decisions.

Suggesting Solutions

The Jefferson Education Exchange and ISTE are both working to address the issues raised in their research. The two organizations are collaborating with other groups, like Digital Promise, to elevate the issue of sharing reliable information about ed-tech products. Most recently, it was a topic of discussion at the EdClusters conference in Philadelphia.

ISTE also opened a community-driven review platform called the ISTE Edtech Advisor earlier this year. This resource is designed to give members of ISTE insight into which tools, technology, and apps are most likely to best meet their learning objectives.

And JEX, launched in February, is a nonprofit organization working to improve how schools choose, procure and implement ed-tech. It plans to pay teachers for reviews of ed-tech products.

Epstein said educators are right to be wary of vendor-sponsored research. When a company does research for the purpose of supporting the sale of its product, “it is naturally incentivized” to select among the schools where its product is implemented and working best, said Epstein. “As a result, educational technology purchasers are often led to believe—erroneously—that they can expect the same result in their local environment [which may be] wildly different.”

Denis Newman, whose company wrote the Guidelines for Conducting and Reporting EdTech Impact Research in U.S. K-12 Schools, an industry-backed best practices report, has said it can be challenging for school leaders to surface this kind of information even at the What Works Clearinghouse.

Educators searching the clearinghouse for research on subgroups that mirror their school’s demographics can find it difficult to discover a match, he said. They can’t easily find the differential impact of a program on a certain subgroup of students, because that information is buried in the report. In its summary reviews of research, the clearinghouse only shares the overall average effect, according to Newman, CEO of Empirical Education, Inc., a Silicon Valley-based research and development company that has conducted research that is also stored in the What Works Clearinghouse.

“I often point out a big project we did for a regional lab in the Southeast,” he said in an interview for a previous story. The study in Alabama found that a STEM program had a small overall effect on math. “But it didn’t work as well for the black kids as for the white kids,” he said, referring to the subgroup impact analysis. That information is included in the report, on page 299.

Epstein said further research is needed to understand more about how often—and what percentage of—educators review IES-supported research and use that research to inform their decision-making processes.  “Stay tuned for more on that,” he said.

Mark Schneider, the first permanent director of the IES in four years, seems to be aware of some of the limitations. In an interview with Sarah Sparks published last month, he said that the agency is exploring whether to use the What Works Clearinghouse to promote research that offers a strong cost-benefit or evidence of longterm results, rather than just looking over a year or two of evaluation.

Follow EdWeek Market Brief on Twitter @EdMarketBrief or connect with us on LinkedIn.  

See also:

Most Popular

Related Posts

48 thoughts on “Tech-Savvy Educators Often Use Vendors’ Info to Find Ed-Tech, But Doubt Their Research, Study Says”

  11. Additionally slated to make that big appearance at Nissan Stadium are Kelsea Ballerini, Brothers Osborne, Kane Brown, Luke Combs, Dan + Shay, Florida Georgia Line, Little Big Town, Tim McGraw, Maren Morris, Old Dominion, Pistol Annies, Rascal Flatts, Thomas Rhett, Carrie Underwood and Keith Urban. It isn’t indicated whether Pistol Annies will be a piece of Lambert’s set, just like the case at the current year’s Watershed Festival. The daily shows at Nissan Stadium are a ticketed occasion, and restricted passes are as yet accessible for buy at

  12. So you need to go to the 2019 Masters, yet you’re not a part, and you didn’t win the lottery? Well there are a lot of tickets accessible on the optional market, however it will cost you. Huge time.This year’s Masters is only a month and a half away and, based on the ridiculous costs for tickets to the occasion, enthusiasm during the current year’s competition is stratospheric.There are many 2019 Masters tickets available to be purchased right now on Stubhub.com. What’s more, they come in all shapes, sizes and costs.

  13. Samuelson was 21 years of age when she won the race in 1979 wearing her Bowdoin College singlet and a Boston Red Sox cap. It was the first of her two Boston Marathon triumphs. She asserted a second title in a then-world record time of 2:22:43 in 1983.

  14. Amid the NFL Scouting Combine, Murray was gotten some information about the likelihood of playing in Kingsbury’s offense. “If I somehow happened to play under him, I think it’d be a lot,” Murray told correspondents. “Me and him being as one, it’d be decent.” Despite the fact that Rosen is only 22 years and was drafted No. 10 generally speaking a year ago, Arizona terminated the head mentor who was with the group when he was picked (Steve Wilks).

  32. Tech-Savvy Educators Often Use Vendors’ Info to Find Ed-Tech, But Doubt Their Research. this fact is a valid point. i watched online training video regarding this fact on Cyberflix TV.

  33. Very Helpful Article Thank you Hello friends, what are you looking for? How to send money easily, so it can tell you some way from where you can send your money. more from moneyordernearme

  34. Thanks for your article. I have learned a lot about it. Do you want to keep your mailbox neat and clean? then use Temp Mail for junk registrations.

  41. Very Helpful Article Thank you Hello friends, what are you looking for? How to send money easily, so it can tell you some way from where you can send your money. more from moneyordernearme

  46. I would like to give a big hand to the International Society for Technology in Education for their dedication and research to find something more informative in the field of Technology. Thanks for sharing this kind of stuff with us. It would be very helpful for me as a student.

Leave a Reply