Home > Uncategorized > Reason #14 to Pick a New State Superintendent: Value-added Measurements

Reason #14 to Pick a New State Superintendent: Value-added Measurements

June 7, 2014

Friday’s reason to pick a new state superintendent – anybody but Janet Barresi! – centered around  the time and money that our state has spent on two of the central reforms that we’ve now abandoned – Common Core State Standards and PARCC. This tweet from a superintendent frames the situation nicely.

This week, both camps (the CCSS supporters and detractors) are trying to figure out where we go at this point. We just blew $4 million on a contract with Measured Progress, and now we won’t even get to keep any of the questions we field-tested.

Blog reader okteacher posted the following comment on Reason #15:

I feel like milkshake has been dumped all over my head. This is such a mess. We have worked so hard to meet these rigorous standards and now we are going backwards?! This is like a warped time machine. I did have some issues with some of my 6th grade ELA standards, like requiring 12 year olds to be able to type a minimum of 3 pages in one setting, but we were doing our best to make this happen. Will going back to PASS make our students fall behind everyone else in the nation that is continuing on with CCSS? This is such a joke. All we can do is laugh at this point and pray that Janet’s days are numbered to 17 from this moment.

This doesn’t resolve our main issue of testing. This just makes things more complicated. I am so embarrassed to be represented by these fools. Every time I think we may take a positive step in Oklahoma education, they continue to dump milkshake everywhere.

The comment is spot on, and thankfully, we’re keeping the milkshake metaphor going. I will make the effort to continue in this post. First, here’s a rundown of the last several reasons:

#18The Hearing no one Heard

#17 – 2K4T

#16 – Questionable Personnel Decisions

#15 – Pulling out of PARCC

#14 – Value-added Measurements

For two years, I have said that there are things far worse than the Common Core. If they just consisted of standards that resembled (without completely mimicking every other state), they wouldn’t be so bad. We’ve totally convoluted our testing in this state, and I’m not entirely sure the Common Core is to blame. No, the greatest insult to the profession we’ve ever seen (even more than Governor Keating calling teachers slugs in the 90s) is the idea that we can measure the contribution of every single teacher on every single student.

Here’s how the SDE introduced Value-added Measurements (and more specifically, the process of roster verification) to us in the fall:

Roster Verification Coming Soon!

In order to successfully collect data for the 35 percent quantitative portion of TLE, teachers will utilize a process called Roster Verification to properly link themselves to the students they teach.Why is Roster Verification important?  This process is important because no one is more knowledgeable about a teacher’s academic responsibility than the teacher of that classroom!  Rightfully so, teachers should have the opportunity to identify factors that affect their value-added results (e.g., student mobility and shared-teaching assignments).In order to assist teachers throughout this process, the Oklahoma State Department of Education (SDE) has partnered with Battelle for Kids (BFK), a non-profit school improvement organization. Together, SDE and BFK will provide teachers with an easy-to-use data collection instrument, Roster Verification training, and communication resources. During February, 2014 the Office of Educator Effectiveness is hosting webinars on Roster Verification.  The webinars will explain how to use the Batelle for Kids program to link students and teachers accurately.  Five sessions will be offered at various times.  We encourage administrators and/or data personnel to sign up for a session.  The same information will be covered at each session, and one session will be recorded and posted on the TLE Web page to access anytime.

The question we’ve all had since the SDE started developing the quantitative measurements for TLE has been, How will they measure growth from year to year, especially when we keep changing tests? And once again, we’re changing tests. Unfortunately though, we have no idea what next year’s tests will be. In the end, someone in the field of research or public relations will tell us that we have found a way to compare scores. On the other hand, these are the same people who keep telling us that the 3rd grade Language Arts test is a Reading test, and that it actually diagnoses reading level. They’ll tell us anything.

Another comment on Friday’s post was made by fellow blogger Mad Ramblings of a History Teacher.

I served on the working committee for VAM. I attended 2 of the 3 meetings (I missed the 3rd because of the weather). Later we were told there were 4 meetings (Where that 4th one came from? I never quite figured that one out.) Anyway, these meetings were a mess from the beginning to the end. And pardon my vulgarity, but where the HELL they came up with their “formula” is still a mystery to me. I don’t even understand the formula, and there, I believe, is the answer to your question about the rebellion over VAM. No one knows or understands exactly what will happen when VAM is calculated into their evaluations. I think the rebellion will begin after the adverse effects of the VAM formula start showing up on teachers’ evaluations. Just look at Houston; several teachers are suing over the use of VAM. One thing is for sure; it’s going to get even more interesting around.

Mad Ramblings had a nice post about this in February called My Shameful Secret.

I have a terrible confession to make. I was one of the teachers on working group #3 for Oklahoma’s TLE VAM sessions. In my defense I thought I was doing something good. I know I am not the smartest person in Oklahoma, but I was vain enough to believe my fellow teachers and I could bring some much needed sensible experience to this “groundbreaking” ill-advised venture Oklahoma’s DOE was determined to try. Boy was I mistaken!

They had Mathmatica. This is the same organization that gave D.C. such wonderful advice. It was so good that D.C. wrongly fired some teachers, and the method used to interpret the data so inherently flawed it can’t be fixed. No matter, Mathmatica just moved on to another state spewing the same propaganda and pocketing the big bucks. But I digress.

I thought the teachers would be treated as professionals at these meetings. .. wrong again. If we brought up a topic they were unwilling to touch, we were too political or going off topic. Then to explain what they were wanting from us, they sent us papers written with grants from the Gates Foundation. Talk about being too political!

I’m just going to throw out there that if it’s not ok to have academic standards that were written out of state, we probably don’t need talking points for VAM that were as well. Again, the whole idea is that we can input a whole bunch of variables into a formula and determine who is effective. For the teachers in subjects that don’t test, we’ll just make up things that we can measure and do that.

The thing with VAM is that it manages to fly under the radar. The few times I’ve written about it, I haven’t had a lot of page views or shares. That isn’t to say that it’s not worth my time to listen; rather it’s as if CCSS is an in-your-face affront to so many while VAM is going to remain so low-key that we miss it. Maybe it’s that we can read the Common Core and very few of us could read (and fewer comprehend) the VAM formula.

This is worse. And the SDE can’t explain it even reasonably well. For the first year that they were developing the quantitative piece of TLE, the two people directing the effort had never even been in charge of evaluating teachers. This year, they added a former principal, but largely, we still have an experience gap.

My fear is that we’ll miss it until it’s too late. Worse yet, we’ll borrow other states’ talking points for adoption without paying attention to their experiences, unintended consequences, and then eventual extraction.

In other words, we’re going to spend years and millions on another acronym only to toss it later. Once again, instruction will suffer as a consequence. This is what happens when the amateurs running the show make decisions without consulting the people doing the work.

Speaking of the amateurs at the SDE, please read Rob Miller’s research on the myth of Barresi cutting costs at the agency. It’s a good companion piece for my #16 reason.

I know that teachers try not to be political. I know many of us have worked in school districts that flinch at activism. We have nothing to be afraid of. We have the momentum. And we have right on our side.

We have 17 days. We must continue being political and speaking loudly. Use your outside voice. Do whatever it takes.

  1. OKMath Teacher
    June 7, 2014 at 10:58 pm

    I absolutely agree with your statement that VAM is flying under the radar. As an AP statistics teacher, I can tell you that I understand the theory behind most value added formulas but I absolutely disagree with the application. I am in good company as the American Statistical Association recently released a statement cautioning about the misapplication of VAM in high stakes situations. http://community.amstat.org/blogs/ronald-wasserstein/2014/04/09/asa-at-175-policy-asa-statement-on-value-added-models-for-educational-assessment
    If Oklahoma continues to pursue this misguided course not only will it be throwing good money after a bad idea (again!) but will continue to see an exodus of teachers. Why, why, why would any teacher (especially those of us who understand how the math is being abused) want to stay in this state??


  2. June 8, 2014 at 11:14 am

    This entire measurement concept is correctly attacked as bad statistical use as Ok Math Teacher has pointed out. Also, this whole large concept is bogus: Children cannot be measured as products for quality control with an industrial model.


  3. joeddins
    June 8, 2014 at 4:03 pm

    Will ALL candidates for THE JOB comment on your ideas?


  4. Anon
    June 8, 2014 at 5:24 pm

    Keep on talking about VAM. It is a fiasco waiting to gain SNAFU status. Its validity is indefensible, and if anyone bothered to look at the reliability coefficients for our state tests, they would know that using those to predict anything else is ridiculous. (Hint: we don’t have score bands around our state test scores because they would be so large, everyone would know that making high-stakes decisions on them is crazy.)


  5. Mona Hickerson
    June 19, 2014 at 9:25 am

    I CANT STOP READING!!!!! I am learning so much…thank you!


  6. June 22, 2014 at 12:46 pm

    Hmm is anyone else having problems with the pictures on this blog loading? I’m trying to find out if its a problem on my end or if it’s the blog. Any suggestions would be greatly appreciated.


  1. June 8, 2014 at 7:47 pm
  2. June 10, 2014 at 6:51 am
  3. June 10, 2014 at 5:25 pm
  4. June 11, 2014 at 9:32 pm
  5. June 12, 2014 at 5:32 pm
Comments are closed.
%d bloggers like this: