Reason #9 to Pick a New State Superintendent: The 2013 A-F SNAFU
Since you’re reading a series about selecting a new state superintendent (and reasons not to keep the current one), it’s worth noting that six of the seven candidates showed up at a debate last night at Rogers State College in Claremore (Joy Hofmeister was not there). It was the first time I had seen any footage of Brian Kelly interacting with the other candidates.
There were several notable statements by the candidates – both good and bad. One confused the RSA with the Common Core. One continues to confuse A-F Report Cards with accountability. Several favor the ACT as the state assessment we will use for all things. Since I’m not going to go back through the entire hour and pick each candidate apart, however, I won’t single anyone out over this one event. If you have an hour, you should watch for yourself (or turn it on as background noise while you do other things.
As for Janet Barresi herself, I wish I had arranged my top 20 (and growing honorable mention) list into a bingo card. Between her responses and those of her opponents, we could have called several numbers during the debate, such as:
#20 – Oklahoma’s ESEA Waiver
#19 – Hiring CTB/McGraw-Hill in the First Place
#15 – In and Out of PARCC
#14 – Value-added Measurements
I would also include today’s post, as well as six of the eight remaining from the countdown (and three off the honorable mention list).
#9 – The 2013 A-F SNAFU
In October 2013, we learned that the state superintendent probably didn’t know what the acronym snafu means. Don’t get me wrong – it was the perfect word to describe the series of mistakes made by the SDE in rolling out last year’s A-F Report Cards. I just don’t think the tone of that word fits with how she usually speaks publicly.
To refresh your memory, let’s return to October 16, 2013 and go from there. Early that afternoon, I had this to write:
We already didn’t take A-F Report Cards seriously. They are statistically unreliable. They include calculations with arbitrary weights. And they are more subject to political narratives than they are to reality.
Today, after much fanfare, schools finally received their preliminary letter grades. There were some surprises, but nothing far off the expectations that had been communicated.
Thirty minutes later, that all changed. The SDE had applied their own formula incorrectly. Apparently, someone mistook the top quartile of students in math for the bottom quartile.
They recalculated grades, and schools saw drastic changes. Many dropped by more than a letter grade.
How can anyone continue expecting people to take this seriously? In thirty minutes, we saw the biggest problem with A-F Report Cards. They depend more on the formula than they do on the students. They are useless in highlighting school performance.
For many schools, grades changed again that evening. The next day explanations were in abundance.
Yesterday, when the grades were finally released, they looked a little high. Thirty minutes later, the SDE adjusted them, and they looked really low. Several schools received the explanation that the people plugging numbers into the formula had inadvertently mistaken the top quartile of last year’s test-takers for the bottom quartile. (Ironically, it was the math scores they had miscalculated.) And they had fixed the scores. And the grades were final.
The emails actually told administrators that the grades were final.
Then last night, I started receiving messages from people telling me that the grades had changed again. Now, they were somewhere in the middle of the first two iterations.
It takes months to calculate the grades, but only minutes to re-calculate them? And then a few hours to do it again – this time with no new explanation?
If anyone has ever doubted the idea that the biggest problem with A-F Report Cards is how easy they are to manipulate, this should be the day they stop. As Rob Miller says, the report cards are DOA.
That afternoon, the SDE released a memo explaining what had happened.
***SDE*** Report Cards Update
OK State Dept of Ed sent this bulletin at 10/17/2013 12:42 PM CDT Dear Superintendents, Principals and DTC’s, It came to our attention yesterday that the bottom 25 percent growth on the A to F report card was calculated incorrectly. A last-minute correction was made immediately before posting that inadvertently caused the errors. We are working to remedy this problem as swiftly as possible, and we will notify all districts once this has been corrected. The date for submitting Data Verification Forms for calculation errors is extended until 10:00 am, Oct 28th. I deeply regret the challenges you experienced yesterday afternoon. If additional calculation changes are needed, please submit the Data Verification Form and we will be happy to process it. Maridyth McBee, PhD |
As I mentioned last night, the legislature simplified the way the SDE was to calculate A-F Report Cards earlier in 2013. Still, the SDE couldn’t make it work. By the end of the second day, many districts reported being on the fifth draft of their report cards.
All of this happened over the week that most districts took Fall Break, which gave many of us ample time to analyze every version that would come out. Eventually, some schools would report seeing their scores change more than 10 times.
By day three, though, Superintendent Barresi had to assert that someone was in charge, so she released a statement of her own.
**SDE*** A-F Report Card Update
OK State Dept of Ed sent this bulletin at 10/18/2013 09:42 AM CDT School districts and schools now have access to review their A-F report card and double check their data and calculations. I understand the experience of the past few days has been frustrating for school and district administrators. I am deeply sorry for the resulting delay and confusion, as are all of us at the Oklahoma State Department of Education. A commitment to transparency can have an embarrassing downside. That was certainly the case when the grades were posted Oct. 16 even as they continued to undergo several versions. A last-minute correction in the calculation resulted in errors that subsequently had to be fixed. To ensure transparency in this process, the decision was made to leave the grades up as they were modified. Fortunately, the problems were addressed and corrected during the designated 10-day window for schools, districts and the OSDE to check for errors. That doesn’t excuse the snafu, but only explains it — and we thank you for your patience. Because of the delay, districts and schools have an extended period, until 10 a.m. Oct 28, to review and seek corrections on their prescribed grades. After that point, the proposed grades will be brought before the State Board of Education at the board’s Oct. 29 meeting for final approval. Some opponents of school accountability will no doubt seize on the recent delay as yet another reason to postpone, reconfigure or simply trash the A-F report cards. Oklahoma parents, students and all interested parties can rest assured that will not happen. The annual grades are critical to heightening accountability, arming parents with important information and furthering the simple proposition that all children can learn. Janet Barresi State Superintendent of Public Instruction |
Predictably, she blamed her opponents. She claimed that none of this would have happened if she – and her people – weren’t just so doggone transparent. She even faulted schools for missing deadlines.
And she called it a snafu. Rob could hardly contain his excitement. This word has been a source of hashtag fun for bloggers and social media magnates such as we are ever since.
The fun lasted until November. First, the SDE had to delay the release of the A-F Report Cards for the second straight year. Barresi complained that complaining district superintendents were engaging in “cheap political theater.” She may not know much about public education, but on that topic, she is a subject matter expert.
I’m intentionally separating last year’s debacle from the overall conversation about the utility of simplifying school performance into A-F grades. This span of time showed clearly that the SDE can’t even manage to compute its own formula correctly. For all the things we can rightfully blame on CTB (such as the #8 post in the countdown), we still have the right to expect competence out of the people running the show. Whether it’s the SDE’s failure to listen to stakeholders, poor hiring decisions, creating a working environment that talented people would rather leave, or simply choosing the wrong vendor, I struggle to find evidence of what these people can do right.
The snafus end June 24 – not a moment too soon.
As of one minute ago
- 850,955 informed readers and counting!
Follow okeducationtruths on Twitter
My TweetsRecent Posts
- Reason #1 to pick Dr. Grace over Mr. Walters: The future we’ve already seen
- Reason #2 to pick Dr. Grace over Mr. Walters: The why and the how
- Reason #3 to pick Dr. Grace over Mr. Walters: Bibliophilia
- Reason #4 to pick Dr. Grace over Mr. Walters: Respect for the profession
- Reason #5 to pick Dr. Grace over Mr. Walters: Fiscal management
Archives
- August 2022 (10)
- July 2022 (1)
- June 2022 (1)
- April 2021 (4)
- March 2021 (1)
- April 2020 (1)
- October 2018 (1)
- September 2018 (1)
- August 2018 (1)
- May 2018 (1)
- April 2018 (2)
- March 2018 (3)
- November 2017 (2)
- August 2017 (1)
- May 2017 (4)
- April 2017 (5)
- March 2017 (1)
- February 2017 (3)
- January 2017 (10)
- December 2016 (2)
- November 2016 (3)
- October 2016 (6)
- September 2016 (5)
- August 2016 (3)
- July 2016 (5)
- June 2016 (15)
- May 2016 (11)
- April 2016 (11)
- March 2016 (19)
- February 2016 (13)
- January 2016 (14)
- December 2015 (13)
- November 2015 (4)
- October 2015 (2)
- September 2015 (3)
- August 2015 (2)
- July 2015 (2)
- June 2015 (3)
- May 2015 (8)
- April 2015 (5)
- March 2015 (11)
- February 2015 (8)
- January 2015 (8)
- December 2014 (7)
- November 2014 (4)
- October 2014 (5)
- September 2014 (7)
- August 2014 (8)
- July 2014 (10)
- June 2014 (35)
- May 2014 (15)
- April 2014 (23)
- March 2014 (15)
- February 2014 (15)
- January 2014 (13)
- December 2013 (11)
- November 2013 (19)
- October 2013 (22)
- September 2013 (10)
- August 2013 (13)
- July 2013 (15)
- June 2013 (10)
- May 2013 (24)
- April 2013 (21)
- March 2013 (22)
- February 2013 (24)
- January 2013 (24)
- December 2012 (11)
- November 2012 (4)
- October 2012 (37)
- September 2012 (19)
- August 2012 (13)
- July 2012 (14)
- June 2012 (20)
- May 2012 (15)
- April 2012 (3)
Reading this site and Rob Miller is like watching a tag team in a wrestling match back when they really wrestled. As they say today, “Awesome.”
LikeLike
I can go with this analogy so long as I get to be “The Macho Man” Randy Savage. Oooh Yeah!
LikeLike
I was just inspired to title a post, “Headed straight into the turnbuckle.” I have no idea what it would be about, but I assume the page views would be out of this world.
Maybe you could use that to summarize last night’s debate, Rob!
LikeLike
I don’t want to grapple over it, but I will warn you that I can execute a rather mean “rolling toe hold.” I like the idea of tying the fake “theatrics” of pro wrestling to the performance of our state superintendent last night. I think Joy “pinned her down” on several occasions. And, on several occasions, I did feel like throwing a chair. I’ll be happy to try and take this down since I know you are busy working on number 8!
LikeLike
I am thinking now. I love this idea. I will write as if a commentator on the WWF network giving a play-by-play analysis of the debate. I am on it like a rat on a Cheeto.
LikeLike
If you’re doing this, I request you refer to me as the Junk Yard Dog.
Now to find some Cheetos for lunch.
LikeLike