The advent of predictive analytics in educational technology brings with it a new set of questions about how the data it generates is being used to help students, what happens with that data, and what assumptions are built into the algorithms used to make decisions based on it.
“Where I see a big push is [to be] much more upfront” about the whole process, said Hawes, so individuals whose data is being collected understand how it is being used.
Beyond that, he said it’s important to have an awareness of the limitations of algorithms that make predictions.
“What is the potential for these algorithms to get it wrong?,” Hawes asked, and what decisions are being made based on them. “What recourse do individuals have to challenge those outputs?”
Hawes said the public and educators need to know and understand these issues so they can be discussed. “There’s no easy solution…but we shouldn’t use these issues to stop the good work being done,” he said to the audience brought together by the Education Technology Industry Network of the Software & Information Industry Association.
A Former Teacher Weighs In
Another panelist, David Koehn, who is now the vice president of product management at D2L Corporation, once taught high school English in Barrow, Alaska, which is the northernmost town in the United States. There, he used an adaptive-content system that established a student’s level of ability based on how he or she responded to an initial set of questions.
If the baseline question began with “Tommy stepped off the curb,'” his students were lost—regardless of their ability level—knew what a curb was.
Being unable to understand the first question “starts you as demoted from the baseline” in that program, and “you’re getting a misleading watermark,” Koehn said.
While that was a number of years ago and the algorithms have advanced since then, the panelists agreed that educators and students should understand assumptions built into the algorithms, and the potential biases that might exist within the data underlying those assumptions.
My colleague Ben Herold has written extensively about the use of adaptive platforms, including a major investment by Facebook CEO Mark Zuckerberg and others in AltSchool, a San Francisco-based school and laboratory that is putting big data and analytics to use. More than 50 engineers, data scientists, and developers there are designing tools that could be available to other schools by the 2018-19 school year.
Algorithms are everywhere, said Koehn, and they’re not going away. “From the medicine you’re prescribed by your doctor to your insurance premium and the the actuary predictor, the music you listen to on Pandora or Spotify,” he said. “From a provider’s standpoint, the way data is manufactured, the way it’s treated—we want to have the highest goals in mind” for educating children, he said.
A Higher-Ed Perspective
The rapid evolution of ed-tech poses questions about how different digital tools are influencing—and some argue, undermining—the delivery of instruction on campuses.
“Post-secondary education is being systematically de-resourced and something called ‘educational technology’ is rapidly becoming a worldwide industry,” said Mitchell Stevens, an associate professor at Stanford University who was also on the panel. “It’s a capital-driven industry, not a science-driven industry.”
Yet “we haven’t built the scientific infrastructure to establish ground truths on what works and what doesn’t,” which Stevens said makes it open to big promises, small deliver, and opportunities to get into trouble with “snake-oil salesman and lot of bad investments.”
Stevens said he is trying to develop a “vocabulary for what would count as ethically reasonable behavior in this sector,” one that makes as much sense to educational consumers and their families as it does to academics and researchers and the ed-tech providers themselves.
To that end, he is working on a project called the Responsible Use of Student Data in Higher Education, an effort launched within Stanford University’s Center for Advanced Research through Online Learning, the Graduate School of Education and Ithaka S+R. Their goal is to help define what it is to be “a good provider of educational services that make good, ethical sense and that have prima facie validity.”