NAACP Calls for an End to For-Profit Charter Schools and Wider Rollback on Charter Sector Autonomy

Minnesota Educators Approve ‘Toolkit’ to Protect Transgender Youth From Discrimination at School

President Trump Donates $100,000 of His Salary to Education Department; Still Wants $9B Cut to ED Budget

New Fordham ESSA Report (Happily) Finds States Moving Beyond NCLB Limitations With New Education Plans

Exclusive: Data Show Charter School Students Graduating From College at Three to Five Times National Average

New Rankings: Most & Least Educated Cities — a Tale of Two Californias

NAACP to Deliver Blow to Charter Schools Today After Public Hearings Critics Say Were Setups

7 Things to Know About How a New Task Force in Los Angeles Is Looking to Improve the City’s Schools

Nobel Winner’s Research Shows Home Nurse Visits for New Moms Boost Children’s Cognitive Skills

‘This Is About Power and Control’: Advocates Push Back on Weingarten for Linking School Choice to Segregation

Editorial Board Lauds Louisiana Superintendent John White — as State Forces Him to Work Month to Month

Ohio’s ESSA Reboot: State School Leaders Make Big Pivot After Outcry Over Testing Requirements

‘It Gave Us a Choice When We Didn’t Have One’: Private School Choice Participants Flood Capitol to Tell Their Stories

This Week in ESSA: Ohio, Montana, Oklahoma, New York Advance Plans as Group Promotes Evidence-Based Rules

Weingarten Speech Tying School Choice to Racism Sets Off Firestorm

House Committee Rejects Democrats’ Bid to Restore Education Funding, Protect Teacher Training

How’s This for a Yarn? School Bus Driver Crochets a Personalized Toy for Every Student on Her Route

A Phoenix Breakthrough: How 3 Massachusetts High Schools Are Helping Dropouts Become College-Bound Grads

Teacher Groups Frustrated With California ESSA Plan’s ‘Loose’ Definition of Ineffective Teachers

The 74 Interview: You Don’t Think Your Child Is Average & Harvard’s Todd Rose Doesn’t Either

$8 Billion a Year on Ed Tech, but Does It Work? Experts Call for Better Research at Unique D.C. Symposium

May 4, 2017

Talking Points

Only about half of districts ask for evidence before purchasing #edtech #ShowTheEvidence

“Politics affect decision-making more than cost, effectiveness, or any other rational input” #ShowTheEvidence

Sign Up for Our Newsletter

Washington, D.C.

When it comes to proving their worth, education technology companies are struggling to put their money where their mouths are.

The $8 billion–plus ed tech industry has ballooned in recent years as tech tools flood K-12 and higher education to adapt classrooms now largely occupied by digital natives. The primary source for federal ed tech funding under the Every Student Succeeds Act is authorized at $1.65 billion. And while investments in the market slowed last year, private dollars continue to drive massive growth and innovation in the sector.

But in an industry that raised more than $1 billion across 138 venture deals last year, there exists an incongruence in evidence to support that explosion of investor and consumer confidence. Also missing is purchaser demand for proof that these innovations work once they make into the classroom. Experts believe, however, that better ed tech research would likely be driven by demand from schools and districts, and not because investors and tech companies seek evidence to grow business.

(The 74: #ShowTheEvidence: Building a Movement Around Research, Impact in EdTech)

In a new study of 63 ed tech companies, 90 percent of respondents reported that they were very or extremely confident that their product has its intended impact, while just over half of those who made claims of efficacy had research to back up their assertion. Overlaid is the finding that only 48 percent of consumers inquire about the relevant research.

“Most surprising to us – only about half of the school districts actually asked for evidence,” said Mahnaz Charania, director of strategic planning in research and program evaluation for Georgia’s Fulton County Schools. “We’re not asking the right questions. So hopefully this research will get us to say, ‘We need to look at this a little deeper and figure out what we need to do about this big gap in what we call quality and what we call evidence.’ ”

A graphic illustrating evidence and solutions presented at the EdTech Efficacy Research Academic Symposium in Washington, D.C.
Photo: Emmeline Zhao

The research was presented during the first day of the two-day EdTech Efficacy Research Academic Symposium Wednesday in Washington, D.C. The findings come from just one of 10 working groups identifying, analyzing, and prescribing recommendations for the role of efficacy research in ed tech development, implementation, and decision-making across sectors. The symposium – hosted by the University of Virginia’s Curry School of Education, Digital Promise, and the Jefferson Education Accelerator — brought together 275 researchers, teachers, entrepreneurs, professors, administrators, and philanthropists with a focus on actionable solutions.

Charania and David Daniel, a professor of psychology at James Madison University in Virginia, led the study. Also among their findings, 57 percent of ed tech developers reported conducting their efficacy research internally, while 20 percent said they use third-party evaluators. And in assessing their products, most companies don’t conduct efficacy research during product development, instead favoring user feedback and engagement data to improve a product over time. Just 42 percent of respondents said they have a standard process for evaluating the impact of their product.

“Our findings suggest that ed tech developers value research to inform their products but, for a variety of reasons, are not conducting rigorous research,” Charania and Daniel write. “Although most developers are focusing their outcomes on implementation fidelity and ease of use, these important aspects must be viewed as a start towards rigorous research and not the end goal.”

 

The rigorous research that Charania and Daniel call for still faces obstacles. The pace of the research cycle is slow — at times requiring years — and some of the best-researched technology often becomes obsolete after surviving the extended process. Researchers for a second working group found that nearly 90 percent of respondents to their survey did not insist on research evidence to make an ed tech purchasing or implementation decision.

“We need a paradigm shift in terms of how we’re working with schools,” said Michael Kennedy, an assistant professor at the University of Virginia and research lead on that study. A universal ratings system could be a solution, he said, to create more accountability in the sector.

Not only are those purchasing decisions often made without the basis of evidence, they’re also frequently the result of lip service paid by colleagues or counterparts in neighboring districts, a third working group found.

“Politics affect decision-making more than cost, effectiveness, or any other rational input,” working group lead Fiona Hollands said. Hollands is associate director and senior researcher for the Center for Benefit-Cost Studies of Education at Columbia University’s Teachers College in New York City.

The confidence and assumptions that ­trusted associates rigorously test recommended technologies are often misplaced, said Bart Epstein, founding CEO of Jefferson Education Accelerator and an associate professor at the University of Virginia’s education school.

Companies know that products multiply through and across communities as a result of that proximity, and Epstein said developers frequently ask him who key influencers are, or whether he can connect them with major school districts. If those mammoths pick up a particular product, it’s more likely to succeed.

Particularly in a torpid ed tech market with capricious investors in the driver’s seat, the onus is on researchers not to influence backers on the supply side but to develop educated mechanisms that foster collaboration and move the needle with schools and districts on the demand side, Epstein said.

“When you ask them publicly, every ed investor will only want to fund the things that work – they’re mission-driven,” Epstein said. “But if you ask them behind closed doors, they’ll tell you that unless they have specific social capital given to them by a foundation or family office, they invest in what provides returns. When research drives returns – meaning more sales – investors will do it.”

“Until then,” he added, “it’s not a ... productive use of our time to cajole or guilt investors into doing more research. What they’re doing is generally more rational. Investors tell us over and over what they want is smart consumers. We need better consumers. We need better tools to support our consumers so they make better decisions.”