Reason #4 to Pick a New State Superintendent: Changing Biology Cut Scores
The response to yesterday’s countdown post has been surprising – and mostly private. The direct messages and emails I have received have been appreciative, and in a couple of cases emotional. Think about that for a minute. The vast majority of parents never have to deal with special education issues. The ones who do are often a fixture in conference rooms with administrators. Yet when someone with a forum takes the time to speak about the issues this group faces, support is overwhelming and from all angles.
Whether you’ve walked a mile in the shoes of a parent of a special needs child or not, you probably still have a good understanding of why it’s important for policy-makers, the SDE, and school districts to possess understanding and compassion. Most parents can send their children to school, check progress periodically, help with homework at night, and expect for the best. I wrote last night’s post with my mind on the parents who never know what they’re going to hear when they talk to their child’s teachers. For those of you who got that, I appreciate you letting me know.
Here’s a recap of the Top Ten (so far):
#10 – Ignoring Researchers
#9 – The A-F Rollout
#4 – Changing Biology Cut Scores
Late last summer, after many districts around the state had already started holding classes, in fact, the SDE announced that test scores from the previous school year would be released soon.
Superintendents and District Testing Coordinators,
Please see the calendar below for a schedule for the release of test scores.
Test Scores Released August 23 Districts have 30-day window
to verify tests scores
August 29 to September 30 Districts have a 10-day window
to review A-F Report Cards
October 10 to 23 SDE staff presents A-F Report Cards
To SBE for approval
October 24 BOE meeting
Then the timeframe changed. The testing company pushed the release back another week. Yes, schools had been working with their preliminary data for two months. They had begun planning for improvement. They had set up schedules for remediation. They just didn’t have all the information they would need for the accountability reports that would become a fiasco two months later.
Oh, and they had no science scores. The SDE was late in releasing those because they were still manipulating the data. A few weeks later, when schools had that information, teachers – especially Biology teachers – were furious. Parents and administrators were angry and confused. Not only were scores much lower than they had been in the past; they did not reflect the opinions of the committees that were in place to set them. One reader sent me this:
Now that test scores have been released to districts, there has been a lot of discussion about the impacts of these scores. I’ve been asked for my opinion about these scores by several teachers, so I thought I would share my (rambling, sometimes incoherent) thoughts with you, the advocates for education in this state. (Also, be aware that I am approaching this from the perspective of my background as a high school biology teacher, although I suspect that many of these points apply to other disciplines and grades.)
First, has raising the cut scores for passing a test ever improved education? I don’t know of any studies that suggest this is true. If there was any evidence showing that raising cut scores alone, without providing additional supports to teachers, improves student achievement, then I would be more willing to accept the SDE’s justifications. Also, if I had confidence that the OCCTs and EOIs test student understanding of science–which I don’t– then I would be more apt to agree with the SDE’s decision. But this will only be a hardship to students, families, schools, and teachers. What is the purpose of making it more difficult to pass a test when you don’t help teachers become more effective? Just telling teachers to “do better or your test scores will be awful and your job will be on the line” is neither motivating nor effective.
When I heard the cut scores, I went through my previous years’ scores to determine how they would have effected my students– the students that I know. I know whether these students had mastered the biology curriculum. And I determined that many of my students, whom I– as a professional educator– deemed to be proficient in biology– would not have passed. And my school would have had to spend scarce resources to remediate these kids. I get that we want to raise the bar– I want that, too. But I don’t want to raise the bar for proficient students, students who “get it”. I want support to meet the needs of the struggling students.
Finally, I want to share my experience as a member of the committee that “set the cut score” for this year’s biology EOI. I put that phrase in quotation marks, because we didn’t actually set the cut score. We began by working through the actual EOI (I’m proud to say that I didn’t miss a single question, although I did struggle a LOT with three questions. I have a masters degree in science education; I’ve taken 40 hours of graduate-level biology courses. I’ve taken– and passed– every pre-med course offered at OU. I have earned a bachelor’s degree, a master’s degree, and I will earn a second masters in December before I begin a PhD program in spring. I have 248 hours of college credit under my belt. I taught biology for 8 years, and I served on several SDE assessment committees. AND I STRUGGLED WITH 3 QUESTIONS ON THE BIOLOGY EOI. Do you know what finally enabled me to answer those questions? I had to switch from the mindset of a person who is proficient in biology content, and instead think like a standardized test writers. What hope was there for our kids to answer those questions correctly?)
Like I said, if I believed in the ability of these tests to accurately gauge student understanding of biology, then I would not be writing this angry diatribe. But I’ve found that my own professional assessment of student understanding is far more reliable than the EOI.
I’ll skip all of the boring parts, but I will tell you that after we set our initial cut score recommendation, Meredith McBee from the SDE addressed us, and showed us data regarding how our cut score recommendations compare to ACT and NAEP data. Our cut scores did not align at all to the ACT or NAEP, but it was not sufficiently explained to us how the EOI comparable score was determined. We were also told that the legislature expected the biology test to be more rigorous than in the past. We were encouraged to reconsider our cut score based on ACT, NAEP, and the legislature’s intent. We did not deviate much from our original recommendation.
Now, here’s the part that should really concern teachers: When Meredith McBee presented cut score recommendations to the State Board of Education, she proposed a completely different cut score than the one that we came up with, and SHE TOLD THE STATE BOARD THAT THE CUT SCORE WAS DETERMINED BY A COMMITTEE OF TEACHERS. Now, I realize that the SDE has the ability to override the teacher committee’s recommendation. But it makes me steaming mad that they overrode our recommendation, and passed their own off as the recommendation of the teachers.
If I were still a biology teacher, I would be passing this information on to every parent of every student who did not pass the biology EOI. Our students should not be political pawns.
In addition to so many other misguided beliefs, Janet Barresi feels that if you don’t measure something, you don’t really value it. On the other hand, if you measure something, and you have no idea what unit of measurement you’re using, it’s hard to place much stock in the result. It was an insult to the students who took the class and the teachers who prepared them. It was a slap in the face for those who served on the standard setting committee (and evidence of why the SDE has difficulty recruiting people to serve on such committees).
Students who take the Biology EOI have to pass four of seven EOIs to graduate. When they went home in May 2013, they had a pretty good idea of how ready they were. When the results came in, a lot of those ideas were dispelled.
Meanwhile, Oklahoma has gone 10 years since completing a standards-adoption process in science; hence, schools are using textbooks that are falling apart. That doesn’t help.
These are the kinds of problems that linger to this day. The SDE shows little regard for what the schools tell them. Even when they ask our opinions, they don’t listen. This is evident in the deaf ear they are turning to the writing scores. Everything about this undermines the entire multi-million dollar testing process.
One thing we know as we await final scores for the 2014 tests this summer is that past is prologue. Later this month, the SDE will again host educators for standard setting for our social studies exams. At this point, I don’t expect a different outcome. The educators will go through the process to determine cut scores for the 5th and 8th grade tests, as well as the U.S. History EOI. The SDE will do whatever they want.