Biology Cut Scores

It’s important to know how the sausage is made, so I’m reposting this from the Effort SOS page on Facebook:

Now that test scores have been released to districts, there has been a lot of discussion about the impacts of these scores. I’ve been asked for my opinion about these scores by several teachers, so I thought I would share my (rambling, sometimes incoherent) thoughts with you, the advocates for education in this state. (Also, be aware that I am approaching this from the perspective of my background as a high school biology teacher, although I suspect that many of these points apply to other disciplines and grades.)

First, has raising the cut scores for passing a test ever improved education? I don’t know of any studies that suggest this is true. If there was any evidence showing that raising cut scores alone, without providing additional supports to teachers, improves student achievement, then I would be more willing to accept the SDE’s justifications. Also, if I had confidence that the OCCTs and EOIs test student understanding of science–which I don’t– then I would be more apt to agree with the SDE’s decision. But this will only be a hardship to students, families, schools, and teachers. What is the purpose of making it more difficult to pass a test when you don’t help teachers become more effective? Just telling teachers to “do better or your test scores will be awful and your job will be on the line” is neither motivating nor effective.

When I heard the cut scores, I went through my previous years’ scores to determine how they would have effected my students– the students that I know. I know whether these students had mastered the biology curriculum. And I determined that many of my students, whom I– as a professional educator– deemed to be proficient in biology– would not have passed. And my school would have had to spend scarce resources to remediate these kids. I get that we want to raise the bar– I want that, too. But I don’t want to raise the bar for proficient students, students who “get it”. I want support to meet the needs of the struggling students.

Finally, I want to share my experience as a member of the committee that “set the cut score” for this year’s biology EOI. I put that phrase in quotation marks, because we didn’t actually set the cut score. We began by working through the actual EOI (I’m proud to say that I didn’t miss a single question, although I did struggle a LOT with three questions. I have a masters degree in science education; I’ve taken 40 hours of graduate-level biology courses. I’ve taken– and passed– every pre-med course offered at OU. I have earned a bachelor’s degree, a master’s degree, and I will earn a second masters in December before I begin a PhD program in spring. I have 248 hours of college credit under my belt. I taught biology for 8 years, and I served on several SDE assessment committees. AND I STRUGGLED WITH 3 QUESTIONS ON THE BIOLOGY EOI. Do you know what finally enabled me to answer those questions? I had to switch from the mindset of a person who is proficient in biology content, and instead think like a standardized test writers. What hope was there for our kids to answer those questions correctly?)

Like I said, if I believed in the ability of these tests to accurately gauge student understanding of biology, then I would not be writing this angry diatribe. But I’ve found that my own professional assessment of student understanding is far more reliable than the EOI.

I’ll skip all of the boring parts, but I will tell you that after we set our initial cut score recommendation, Meredith McBee from the SDE addressed us, and showed us data regarding how our cut score recommendations compare to ACT and NAEP data. Our cut scores did not align at all to the ACT or NAEP, but it was not sufficiently explained to us how the EOI comparable score was determined. We were also told that the legislature expected the biology test to be more rigorous than in the past. We were encouraged to reconsider our cut score based on ACT, NAEP, and the legislature’s intent. We did not deviate much from our original recommendation.

Now, here’s the part that should really concern teachers: When Meredith McBee presented cut score recommendations to the State Board of Education, she proposed a completely different cut score than the one that we came up with, and SHE TOLD THE STATE BOARD THAT THE CUT SCORE WAS DETERMINED BY A COMMITTEE OF TEACHERS. Now, I realize that the SDE has the ability to override the teacher committee’s recommendation. But it makes me steaming mad that they overrode our recommendation, and passed their own off as the recommendation of the teachers.

If I were still a biology teacher, I would be passing this information on to every parent of every student who did not pass the biology EOI. Our students should not be political pawns.

Makes you want to shop on the organic aisle, no?