Where Education Research, Politics & Policy Intersect: 3 States Reveal How Data Help Shape Their ESSA Plans

That was the topic of a panel at the Association for Education Finance and Policy conference in Washington, D.C., earlier this month. The discussion featured representatives from the Louisiana Department of Education, the Tennessee Department of Education, and a group of eight California districts — including Los Angeles, Oakland, and San Francisco— known as CORE. (A member of the Rhode Island Department of Education was also present, but his comments were off the record, so he cannot be quoted.)

The trio discussed the benefits — and challenges — of using research to inform ESSA accountability design. A few important themes emerged.

Like many states, Tennessee is wrestling with how to weigh students’ absolute achievement versus their growth when measuring their schools. Researchers generally say that the progress students make is preferable for isolating the impact of schools and holding them accountable.

That doesn’t mean that such a move is politically easy, noted Mary Batiwalla, the Tennessee Department of Education’s executive director of accountability.

“The lowest-achieving school could receive an ‘A’ under our [proposed] system — very low-achieving, but showing what we consider to be remarkable and life-changing growth,” she said. “It’s a tough conversation to have with folks because there is this very accepted notion that ‘If you say that school that is very low-performing is an A school, you are lying to parents.’ ”

In line with research, Batiwalla pointed to potential unintended consequences of judging schools by absolute performance or students’ raw standardized test scores.

“When we think about the downstream of impacts of labeling schools that are doing remarkable things for their students, even when those students start at very low places — [it’s] retention of high-quality teachers, recruitment,” she said.

A focus on growth may also lead to pushback from districts previously deemed high-performing. “Tennessee has decided we no longer want to reward simply having high absolute achievement,” Batiwalla said. “If you have students who come in at a certain level, the expectation is that you grow those students.”

“We haven’t released any of this modeling [of the new ratings] publicly, so I don’t think there’s been as big of an outlash as there might be if we were to actually release these lists,” she said. “There have been some case studies that we’ve provided that some of our higher-performing districts have been able to back into and say, ‘Hey, that C school, that’s me! Wait a second — get on the phone with the state legislator.’ ”

“We’ll see how this plays out,” she added.

California’s CORE districts have been among the pioneers in evaluating schools by measures that go beyond test scores and high school graduation rates. These include social-emotional or non-cognitive skills, such as motivation, focus, and confidence. States nationwide now have the opportunity to expand accountability systems along these same lines using the “fifth indicator” of performance under ESSA.

“We know these factors are incredibly important. The research tells us that, and so does our own felt experience,” said Noah Bookman, chief strategy officer for CORE, a coalition of some of California’s largest districts that joined forces to try to better measure and improve schools.

In addition to using chronic absenteeism and suspension rates for measurement, the districts have incorporated surveys that ask students various questions to gauge their “growth mindset, self-efficacy, self-management, and social awareness.”

The practice has proved somewhat controversial. Angela Duckworth, a University of Pennsylvania researcher who developed the idea of “grit,” — passion and perseverance in reaching a goal — specifically criticized CORE in a New York Times op-ed.

---

 

The CORE districts don’t measure grit specifically, as they see it as falling under the self-management measure.

Indeed, one study, co-authored by Duckworth, showed that students in Boston charter schools made large gains on standardized tests and had high attendance rates but actually saw dips in their self-reported non-cognitive skills. The researchers posit that this is due to reference bias.

One analysis of the California districts found that their measures of non-cognitive skills were correlated with schools’ achievement, attendance, and suspension rates, suggesting that they capture useful information.

As evidenced by the disappointing recent study on the federal government’s $7 billion school turnaround plan, figuring out how to improve a struggling school is difficult — and research doesn’t provide definitive answers on this front. The panelists’ comments reflected this reality, particularly on the questions of whether states should take control of low-performing schools.

Indeed, research on this question has come to mixed conclusions. In New Orleans, the state-driven expansion of charter schools and infusion of new resources post-Katrina has yielded large test-score improvements.

One study found that New Orleans’s closures of low-performing charter and district schools led to big gains for students.

But in Tennessee, as Batiwalla suggested, the state-run district has not produced any improvement in achievement, though a locally driven turnaround effort did. Still, Batiwalla noted, “The threat of state takeover perhaps energizes the district to respond in a way that leads to improvements.”

Bookman of California was largely silent on the issue of specific interventions for struggling schools. Last year, The 74 reported that the state had not spelled out what would happen in those cases.

A recent Los Angeles Times article describing California’s new dashboard of school ratings across a variety of measures noted, “It remains unclear how the dashboard will be used with regard to those schools that need help.”