By guest blogger Benjamin Herold
Cross posted from the Digital Education blog
Can big data and predictive analytics help school leaders hire better teachers?
That’s the new pitch being made by two companies: TeacherMatch, from Chicago, and Washington-based Hanover Research. Both claim that their new algorithm-driven teacher-selection tools can predict the impact teacher candidates will have on student test scores should they be hired, which they argue would be a significant upgrade over the screeners currently in use.
Between them, the companies say they have signed up nearly two dozen districts and charter organizations already.
“I’m seeing a shift in the way that [K-12] human resources departments are viewing themselves,” said TeacherMatch CEO Don Fraynd. “There’s a whole new breed of folks who are saying that we’re in charge of finding teachers…shouldn’t we care about student achievement?”
Fraynd and Peter Dodge, the founder and CEO of Hanover Research, which recently unveiled a product called Paragon K-12, both contend that the wealth and accessibility of educational data now available, combined with the advanced statistical modeling that is now widely used in other sectors and industries, have tremendous power to help improve teacher hiring.
“There’s no dearth of data in education, but we rarely put it to work,” Fraynd said.
But not everyone is sold on the concept.
“Overall, this is a good signal of districts getting on board and thinking more systematically about the hiring process,” said Jonah Rockoff, a Columbia University professor who has written extensively about teacher recruitment, hiring, and performance. “But is there a magic formula that can revolutionize teacher hiring? I’ll believe it when I see it.”
For years, many districts have used the Haberman Star Teacher Pre-Screener, Gallup’s TeacherInsight, and a handful of other teacher selection tools to screen prospective teachers. Those focus primarily on determining candidates’ attitudes and beliefs about teaching and their potential to fit well in a school environment.
More recently, some forward-thinking districts have begun taking a more comprehensive approach (my colleague Stephen Sawhuck of our TeacherBeat blog described these new “strategic hiring” practices back in 2011.)
Fraynd of TeacherMatch said he experienced first-hand the limitations of existing tools when he served as the chief school improvement officer for Chicago Public Schools.
“We evaluated a number of tools, and we could not find one that we were satisfied with,” he said. “We would press them for evidence that their screener was predictive of actual student achievement gains and not just [a candidate’s] ‘fit,’ and we were never satisfied with the evidence they presented us with.”
After leaving CPS, Fraynd and others decided to create their own company. They founded TeacherMatch three years ago, attracting private equity capital to get off the ground. Their first step was a $3 million research project, conducted in partnership with the University of Chicago, the Northwest Evaluation Association, and others. The study used years’ worth of data from NWEA, a Portland, Ore.-based assessment nonprofit, to determine what teacher characteristics are identifiable at the time of hiring and predictive of student achievement gains.
They found dozens of factors that could be lumped into four broad categories: Qualifications, including the selectivity of the college or university a candidate attended; attitude, including indicators related to a candidate’s penchant for persevering through difficult challenges; cognitive ability, as measured on tests of content and knowledge; and teaching skills.
Hanover Research, which works with hundreds of districts around program evaluations, data analysis and other research projects, concluded that a similar constellation of indicators can best predict teachers’ impact.
Fraynd said one key to TeacherMatch’s predictive ability is drilling down deeply enough to understand, for example, that senior-year college GPA seems to make a much bigger difference than overall college GPA.
It’s also important, he said, to have the statistical and technological horsepower needed to roll hundreds of variables up into a single meaningful score.
Peter Dodge, the founder and CEO of Hanover Research, agreed, adding that presenting that information back to district staff in usable ways is also key.
“Many of the [existing] instruments provide data that is really little more than an Excel spreadsheet,” Dodge said. “We offer a comprehensive and easy-to-understand dashboard around all the characteristics for which we’re screening.”
Just One Tool Among Many
Rockoff, the Columbia professor, agreed that tools like the Haberman STAR Teacher Pre-Screener are limited and that it takes a constellation of factors to predict whether candidates will positively impact student learning.
Those were essentially the findings of a 2009 research paper that he co-authored.
But in an interview, Rockoff also said he believes the most useful information for determining whether teachers will be any good is actually seeing them teach.
“Bring them in, have them teach three hours of summer school,” he said. “The closer you get to a real classroom observation, the [better].”
The new predictive tools are generally intended to be just one part of a more comprehensive hiring process. Both companies will also include ongoing research on how effectively their tools are predicting teachers’ impact, with plans built in to tweak their algorithms as new information is obtained.
That will be the case in the 29,000-student Chula Vista, Calif. school district, which recently became Paragon K-12’s first client. The district hires 40 to 60 new teachers each year, said superintendent Francisco Escobedo.
“Typically, we’ve had applications turned in, we meet with a pool of candidates, and we have a performance-based interview where [candidates] have to present a lesson in front of kids,” Escobedo said. “We’ll be using [the new screening tool] to determine which applicants are part of the eligibility pool, and this is an additional data point that principals and staff can use to make determinations between two or three similar candidates who have all done great on an interview.”
As it is, Esobedo said his district ends up pleased with about 70 percent of the teachers it hires. Now, Chula Vista is hoping to push that figure to 90 or 95 percent. The reasons are clear: Not only do ineffective teachers hinder students’ learning, they are also expensive to remove and replace.
Paragon K-12 and TeacherMatch are both banking that those kinds of concerns will be motivation enough for potential clients. Dodge said that while teacher selection will likely never become a big revenue-generator for Hanover Research, he hopes for a couple hundred district clients and about $10 million in annual business.
“Public schools are slowly being dragged into a more business-like state of mind,” he said. “That’s the direction things are headed.”