A-F: Flawed Now and Forever

This morning’s Oklahoman had a real surprise: an article by Ben Felder linking the state’s A-F Report Cards to the poverty level of schools. The connection between the two isn’t a surprise. It’s the placement of such a story.

Yep, that’s the front page of the Oklahoman. When I awoke this morning, I had messages and tweets telling me all about it. One tweet in particular pointed out that I had in fact made a similar connection on this blog – in 2012.

@benfelder_okc @TheOklahoman there has been some ed blogs highlighting poverty bias also https://t.co/arHRZhazJb @okeducation @edgeblogger

— Timmy Higgins (@timmyjhiggins) May 15, 2016

Yes, when Oklahoma issued the first A-F Report Cards in 2012 – using a formula that was even worse than the one we have now – I pointed out that the scores favored schools serving more affluent populations. Nonetheless, the Oklahoman supported the report cards on its editorial page.

Those who attribute good school grades to socio-economic factors are off the mark. Several A schools were in rural communities that aren’t concentrations of wealth and privilege. The poverty rate in Canton is higher than the statewide rate. Average household income is 14 percent lower than the statewide average. Yet Canton High School got an A.

Yes, you can always cherry pick the exceptions. Overall, though, poverty was a strong indicator of a school’s grade.

For that post, I also looked at the specific distribution by site, for the schools with the highest and lowest poverty rates.

Here is the grade distribution of the high-poverty schools:

Letter Grade Distribution of High-Poverty Schools
A	B	C	D	F
2	8	32	46	2

Here is the grade distribution of the low-poverty schools:

Letter Grade Distribution of Low-Poverty Schools
A	B	C	D	F
46	48	3	2	2

The truth is that this was also the case for the previous accountability system, API scores, as my very first blog post pointed out. It was still true the next year when the formula changed (and when the SDE had to recalculate scores a dozen or so times). Still, there was the Oklahoman criticizing the critics.

With the release of Oklahoma school sites’ A through F grades looming, opponents of accountability are predictably ramping up attacks. School officials should think twice before embracing one such tirade issued by a small group of college academics. To discredit A-F school grades, those researchers effectively argue that there is little correlation between a public school education and actual student learning.

The year doesn’t matter. Sometimes it’s preemptive. Sometimes it’s reactionary. The stance of the Oklahoman has been consistent. They don’t like it when superintendents, researchers, or anyone else points out the obvious truth that an A school isn’t necessarily better than a C school.

That’s why today’s article, along with its prominence, isso surprising. Felder is a good researcher and writer, as today’s piece illustrates:

The average poverty rate for an A school in the state is 45 percent, based on analysis of the 2015 letter grade report from the Oklahoma state Department of Education.

As you move down the grading list, the poverty rate grows bigger — B: 58 percent, C: 67 percent, D: 76 percent, and F: 84 percent.

In Oklahoma County, which is home to school districts in Oklahoma City, Edmond and Midwest City, the income gap between A and F schools is even starker. The average in Oklahoma County is A: 29 percent, B: 56 percent, C: 67 percent, D: 77 percent, and F: 83 percent.

The disparity highlights the challenges students living in poverty face when it comes to performance in school, at least performance on end of the year exams, which account for the majority of the state letter grade. It also highlights the potential challenge for low-income families to access many of the state’s highest performing schools.

Moving from a neighborhood with an F school to one with an A school could mean finding a residence where the income level is nearly three times higher.

Again, we know this pretty well. In November 2013, I made a list of factors that correlate to a school’s A-F grade:

Another reader pointed me to this spreadsheet showing all school districts in Oklahoma, their student counts, and the percentages of students eligible for free and reduced lunch. The table also has bilingual student counts, which is information I previously didn’t have. Last week, I ran correlations between school grades (and district grades) and poverty. Yet another reader suggested to me that I run correlations between the grades and poverty, this time only using districts with more than 1,000 students.

Comparison	Correlation
All District Grades to Poverty	-.52
Large District Grades to Poverty	-.80
Large District Grades to Bilingual	-.32
Large District Grades to Poverty + Bilingual	-.76
Small District Grades to Poverty	-.51
Small District Grades to Bilingual	-.10
Small District Grades to Poverty + Bilingual	-.45

Both factors – poverty and bilingual education – seem to impact large districts to a greater extent. Statistically speaking, there are a couple of factors here. One is that the data for bilingual counts include a lot of schools with none reported. Zeros in statistics skew results (as they do with student grades). Another factor is that there were 131 of the large districts (still a statistically significant sample) and 386 small ones.

My takeaway from this is that while the report cards tell the story of schools’ accomplishments only to a limited extent, and while my analysis from before built on that, there is always more to learn, if you’re willing to unpack the data and find out what is happening. Among our largest schools, we see more variance in socio-economic levels. We also know that urban poverty and rural poverty are not identical.

As always, I should point out that correlation does not equal causation. Nor does it equal forecast. Schools with high poverty rates do sometimes perform well on tests. They just don’t do it with the frequency of schools with low poverty rates. The explanations for this are myriad. Low-poverty schools get more applicants for open teaching positions. They are more likely (based on US Census data) to have parents who are college-educated. They get more parental involvement. The list of reasons goes on and on.

Nor is this simply an Oklahoma phenomenon. As Paul Thomas writes on his national blog, The Becoming Radical, today:

“Bad” and “good” contribute to our coded political and public discourse that reflects our collective unwillingness to do what is required: reform directly education so that all students have the sorts of opportunities that we do guarantee to the most fortunate children among us.

That’s all an A-F Report Card system does. It codes our schools. It labels enough of them as failures to extend the narrative that public education as a whole is failing. And I’ll go ahead and say what you’re thinking: it contributes to white flight.

Asked for his thoughts, Rep. Jason Nelson acknowledged the poverty linkage to the grades, but also advocated for more school choice:

Nelson views this income disparity as a reason to allow a student’s state appointed funding to be used for enrolling in a higher performing school.

“A lot of parents can’t really move from the inner city of Oklahoma City to Deer Creek, and even if they could afford to do it … their support system can’t all move with them to Deer Creek,” Nelson said. “The key is to give them options where they exist today so they aren’t forced to move if they can’t.”

That’s all true. People can’t just buy a house that’s 300 percent more expensive and move. What he doesn’t mention – what education reformers never mention – is that the school with a low grade may still be a good school. It’s also myopic to assume that families from the inner city even want to move to Deer Creek. Some people actually value their neighborhoods, and as an extension, their neighborhood schools. Maybe some would move, given the option. Some wouldn’t though.

Let’s frame it another way. If your kids are in a school with a low poverty rate, something like 20 percent free/reduced lunch participation, and the school gets a B, aren’t you going to wonder why? It rarely happens. Does that mean that every school with low poverty and an A has great teachers? Absolutely not. It’s easy to be shiny when you have resources. That doesn’t mean the teachers don’t work hard, though.

That’s been another one of my great concerns during this age of accountability. We don’t want to make any assumptions based on the letter grades. Some schools with an A are great. Some aren’t. The same is true for schools with lower grades. In most of them, you’re going to find teacher working really hard to help students succeed.

The Oklahoman recognizing that poverty impacts student achievement is like Mary Fallin acknowledging that fracking causes earthquakes. Admitting you have a problem is the first step, but it was obvious to the rest of us for years.

I’m happy for Felder’s coverage, but I now wonder what will follow on the editorial page. There’s long been a disconnect between the paper’s reporters (who tend to treat public school stories fairly) and its opinion writers.

For 2016, we’re still using the A-F Report Cards that hundreds of superintendents, as well as the state superintendent, have completely disavowed. Testing is over for the spring, and report cards won’t come out until this fall. If you want a preview, however, click this link showing current percentages of students served by free and reduced lunches in our schools. This will be pretty close to the final outcome.