One more chance to get it right

Critics of Oklahoma’s current A-F Report Card accountability system (including me) typically have two complaints about the system. First is that the methodology is poor. Half the score is simply a rendering of the percentage of students passing tests – which is closely tied to poverty; the other half is tied to growth. In measuring growth, we’re also double counting our lowest-performing students. In other words, our students who struggle the most count a total of three times.

The other issue many of us have is with the A-F label we ultimately attach to each school. It’s insulting to distill all the work schools do into a single indicator. Even though A-F is not explicitly a ranking system, much of the public assumes that an A school is better than a B school. Context be damned. A is better than B. Always.

For these reasons, the Legislature tasked the Oklahoma State Department of Education with leading a task force to make recommendations for improving school accountability prior to the end of this calendar year. As directed, the OSDE brought together “students, parents, educators, organizations representing students with disabilities and English language learners, higher education professionals, career technology educators, experts in assessment and accountability, community-based organizations, tribal representatives, and businesses and community leaders.”

The task force met four times and provided the OSDE with feedback. The last of these meetings was on November 9^th. On November 28^th, the US Department of Education changed one key piece of the federal requirements under which we have to operate. We no longer have to have a summative score. The task force never met after that one requirement was removed.

It’s important to note that we don’t know how the task force conversation would have sounded if they could have discussed revising our accountability system with this piece of information. I doubt the different constituency groups would have been unanimous in their feelings on A-F Report Cards, or any other type of summative score. We also can’t know what the OSDE would have done with the feedback, even if it had been given. Task force members gathered to provide feedback only. They were not a voting body with any kind of decision-making authority.

To be clear, this is not a criticism of the work that has been done or the leadership under which the task force has worked. Superintendent Hofmeister and her staff expertly led this process. The lead researcher, Marianne Perie, from the University of Kansas, was thorough and good at explaining statistical processes to an audience with a varied background in that kind of work. Ultimately, the methodology of the accountability system being presented to the State Board of Education this week is solid. It will likely yield results that are not singularly correlated to poverty.

The end product is good, and an improvement over the current accountability system. That solves half of the problem from paragraph one. The other half remains – that schools will still receive a summative score.

I’ve always bristled at the idea that we need to label our schools this way. By always, I mean from the moment I began writing this blog nearly five years ago. I don’t think a star rating system would be much better. School accountability isn’t Yelp, or this strange sign I found in China a few years ago.

I believe in accountability and transparency. Publish our schools’ test scores. Publish any data point you want. Just provide context. A summative score doesn’t do that. No matter how much detail is on the OSDE website for each school, the newspapers will skip to the end and publish the thing that’s easiest to consume. Calling A-F accountability, though, is like calling Velveeta cheese. It’s an accountability-like substance.

Recently, a couple of Mid-Del employees put together a list of all the schools in the five largest counties in the state and sorted them by grade span and by poverty. For example, one table showed all the elementary schools that have between 25% and 35% of student receiving free or reduced lunch (FRL). Of the 23 schools in that table, Mid-Del had one, and it had the highest numerical score on the current A-F system.

Compared with similar schools, Schwartz Elementary has outstanding academic performance, however you measure it. On the other end of the spectrum, there were 60 elementary schools on the list with an FRL rate at or above 95%. We had one such school, and it had a 68 on the report card. That’s a D+, and it’s higher than 53 of the other schools on the list.

Numerical Grade Distribution of Elementary Schools with at least 95% FRL
70-78	5
60-69	11
50-59	23
40-49	13
33-39	8

Let that sink in. No school in the five largest counties in the state with higher than 95% FRL had a numerical grade higher than a 78. Meanwhile, none of the schools in the 25% to 35% range had a score lower than 80. Does this mean that all schools serving mostly upper-middle class kids are better than all schools serving the kids with the highest levels of poverty? Of course not.

This is the thing I hate – the labels. If you provide most people with this entire view, they’ll get it. An A on a report card may be misleading. So may an F. Even though the new accountability system will do more to provide context, the summative grade will damage that effort.

A-F Report Cards feed a narrative. They are one of the most toxic pieces of the recent education reform agenda. They blur the difference between simple and simplistic.

Please understand that I hope the State Board of Education this week will recognize the hard work of the state superintendent, her staff, and the entire task force. Regardless of how they see the A-F labels, they need to recognize the quality of the work that is being presented to them. I hope the legislature and governor will recognize this too.