VAM: Better Never than Late

Let’s talk about teachers for a moment. Some are great, some are decent, and some are better suited for another career. We knew this when we were students. We see it as parents. It’s even obvious to us sometimes as colleagues down the hall. The vast majority of teachers fit into the first two categories. Of that, we should be quite proud. Just the same, nearly all schools have someone who brings us all down.

If there were an instrument – a valid, reliable one that I believed could tell me numerically who my best teachers are – I’d use it in a heartbeat.

There’s not.

When I was teaching Honors English II classes in Mustang, most of my students could have passed their end-of-instruction exams before they even had one day with me. There was no standardized instrument suited either to measure their learning or my effectiveness. They were already hitting their heads on the ceiling of every test they had ever taken. Another test showing the same thing meant nothing to them.

As a teacher, I received great evaluations. Those also may or may not have meant anything. I can only remember two negative comments, both from my first year of teaching, when I was in Muskogee.

The first (from an assistant principal) was a question about whether or not I noticed a certain student chewing gum while I was teaching. Yes, I had noticed it. No, I didn’t want to interrupt the momentum of instruction to draw attention to it. By the end of the hour, I had forgotten about it. Yes, he got away with breaking a rule, but it wasn’t worth stopping and starting again.

The second (from the university professor on my entry-year committee) was a comment about finding something instructional for students to do when they finish an assignment because – wait for it – all they’re doing is reading.  Claudia Swisher, I should have told you to turn away. Oh, the horror! Eighth graders reading, without anyone telling them to! I think his point was that I should have been teaching bell-to-bell. If it was something else, it was lost on me.

I think as a first year teacher, I had some very good moments. By the time I left the classroom, I think I was a very good teacher. I was never great, though. I didn’t have the years of experience (nine) or consistency to claim that. I loved it, but we’re not automatically good at the things we love. I love to sing in the car. I love basketball.

If you looked at my evaluations when I was in the classroom, though, you would have thought I was the very model of a modern master teacher. All of the check marks were in the far right column (the good side). Occasionally, I’d have a few encouraging comments like “try beginning class with an activity to engage prior knowledge.” Casually (not in writing), I would receive suggestions about classroom management or working with parents and colleagues – normal things that young teachers need to learn. Still, my evaluations would have all the check marks lined up in the right boxes.

That was the old teacher evaluation system. In 2011, the Legislature – acting in conjunction with then State Superintendent Janet Barresi – passed legislation creating the Oklahoma Teacher and Leader Effectiveness Evaluation System, or TLE. Even with later legislation amending it, the TLE law includes both qualitative and quantitative pieces. Since not all teachers give a state test, and since not all state tests are paired in ways that can show growth, two different quantitative measurements were created: Value Added Measures (VAM) and Other Academic Measures (OAM). The OAMs are no longer a thing. They have ceased to be. They are now ex-quantitative components.

Let me come back to VAM a few paragraphs from now. On the qualitative side, I have seen two of the evaluation models in action. When I was in Moore, we used the Tulsa Model. In Mid-Del, we use Marzano. I honestly have no preference. The language describing the different ranges of teachers is about the same.

With the way we used to do teacher evaluations, as with TLE, what really matters is how committed principals are to improving instruction. Do they have the resolve to have difficult conversations with teachers? Do they use the evaluation model with fidelity? Or, to borrow from Garrison Keillor, is the school the kind of place where “all the [teachers] are above average”?

We can have a well-researched qualitative teacher evaluation system, and we can make districts pay for training in the summer so that principals learn to calibrate their scores for teachers. It’s like shooting free throws in practice. When you have the pressure of giving a teacher a low score, even though you personally like that person, or even though his last 10 principals gave him a good evaluation, what will you do?

I’ve jumped in with both feet, and I know many other principals who have too. It’s not an easy thing to do, but at least you’re doing what seems right based on what you actually see. Then there’s VAM.

To date, no teacher in Oklahoma (that I know of) has had a VAM score added to his/her evaluation. No principal or superintendent I talk to has faith in them. It also sets up a two-track system for evaluating teachers – one for those with a VAM score, and one for those without. It’s inequitable on its face.

That is why I was less than enthused to see this in my email yesterday:

Value-Added Results Now Available

Value-added results demonstrating student academic growth during the 2014-15 school year are now available for teachers and administrators through the SSO2 portal. Guidance documents about how to access and distribute these reports can be found on the Teacher and Leader Effectiveness (TLE) page on the OSDE website.

If, by chance, I cared about VAM scores, I would probably want them sooner. Why would I attach scores that ostensibly show a teacher’s effectiveness last year to this year’s evaluation? Since I don’t care and I wouldn’t use them, we need them to go away. In Mid-Del, I don’t even know how many certified and support employees in my district spend how many hours preparing for the Roster Verification process. It’s all a waste of their time. Furthermore, the SDE spends over $600,000 on contracts with out-of-state vendors so we can verify who had this kid for what part of that month and how to calculate VAMs that nobody uses. Every penny of that is a waste of resources that continue to melt away.

One bill that we can still support to end this madness is HB 2957. In its current form:

• Districts would have the option to use quantitative measures in their evaluation process; but it would no longer be required;
• Teachers and administrators would develop a yearly individualized program of professional development;
• This would be a collaborative effort between the evaluator and the teacher/principal.
• The focus would be on components from the qualitative framework, but not necessarily on low areas;
• This is not intended to increase the amount of required PD hours, but rather to focus professional learning on areas that lead to higher student achievement;
• VAM would no longer be required by the state (and hopefully no longer purchased by it either); and
• Career teachers receiving a district rating of “highly effective” or “superior” would only need to be formally evaluated once every three years.

It passed the House by a vote of 94-0. It passed the Senate with amendments 46-0. Now the House needs to approve the changes and send it on to Governor Fallin. Easy, right?

Not this year. Nothing is easy this year. Nor is it logical.

Here’s hoping…

 

 

 

 

Ten Observations on Three Years Blogging

This blog turns three today, which is 21 in dog years. How do we celebrate 21^st birthdays? I forget. Besides, I don’t own a dog.

I don’t know that there’s much significance to the blog turning three. A lot has changed during the last 36 months, and no, I’m not particularly taking credit for any of it. Here are 10 observations from my first 535 posts.

1. Three years ago, Oklahoma educators were fed up with policy makers who were ripping apart our education system. They’ve had to slow the pace of implementing corporate reform, but they’re still on the move. We’re still fed up. The agenda is still moving forward.
2. Parents are the best voice for public education. As many educators as there are blogging and contacting legislators, we only impact policy to a point. Parents move the needle. Even better is when parents and educators band together to advocate for children.
3. Electing a state superintendent who respects teachers is a game-changer. There’s been a change in the mood among educators since January, but there is only one meaningful difference in terms of the elected leaders of this state. We still have the same governor. We still have the same senators and representatives dredging up the same bills. We still have RSA and ACE; A-F Report Cards; TLE and VAM (though maybe with a delay); and funding for public education is still critically low. The difference is that we have replaced the state superintendent who blames teachers for everything with one who goes to bat for them. Joy Hofmeister understands that teachers aren’t bad people. Rather they’re the people who spend all day with our children. They deserve respect.
4. High-stakes testing is unpopular with most students, parents, and educators. It’s only certain politicians and “philanthropists” who love it. This seems obvious now, but remember that my first post was filled with frustration that we were sorting and ranking schools by test scores, without regard to poverty. Over time, okeducationtruths has become one voice among many expressing anger over this. Those of us calling for testing reform don’t always agree on solutions, but when it comes to the harmful effects of using tests to label people and schools, we’re together.
5. I enjoy reading blogs probably more than I enjoy writing them. This isn’t a humble-brag statement. If I didn’t think I could write, I wouldn’t. I just know that I’m not the only game in town. At various times, I’ve tried to capture a list of Oklahoma education blogs and national blogs I read regularly. That list is sadly out-of-date. I’ll probably work on it again when the school year ends. Among my fellow Oklahoma educators are writers who say it better, and bloggers who are more popular. There are also some who are just getting started. I try to read them all.
6. I treasure the friends I’ve made from blogging. These aren’t just shallow acquaintances who happen to share a common interest in saving public education. These are real people with students and families and stories and histories that make them who they really are.
7. Sometimes I just can’t tell what’s going to be a hit. For example, last week I wrote two posts. In the first, I described how I would introduce poetry to my students 15 years ago. I spent hours on it. In the second, I heaped praise on Hofmeister for acting quickly to find a solution to a tough problem. I wrote that in 15 minutes while waiting to pick up my daughter from play rehearsal. The second post has been viewed five times as many as the first one. I’ve received several comments – both privately and publicly –stating that the first was one of my best, which is how I feel as well. That isn’t to say that people are wrong. I am probably just a poor judge of what will stick.
8. Teachers will band together to protect their content areas. There’s a reason the APUSH legislation in both houses of the Legislature fizzled into a joint resolution with all the impact of a greeting card. My Save AP post from February is sixth for page views all time on this blog. It’s the most-viewed post that doesn’t talk about the third-place finisher in last summer’s Republican primary. Well except for one…
9. Teacher pay in Oklahoma still hovers around the bottom of the country. My January post discussing teacher pay jumped to number three when it made another viral run around social media in March. In 1970, Oklahoma teachers made 80% of the national average. In 2013, Oklahoma teachers made 80% of the national average. In between, there’s been little fluctuation. At the rally in March, we heard every excuse imaginable from our elected leaders about why teachers can’t have raises right now. This from the same crowd who don’t want to hear excuses from legislators. What they’re really lacking is resolve, and it’s apparently a generational problem that spans decades and knows no partisan preference.
10. Blogging anonymously was fun, but getting to know my readers has been better. At edcamp in February, I was able to participate in a roundtable discussion about advocacy and blogging with the likes of Joy Hofmeister, Jason James, Rob Miller, Kevin Hime, and Claudia Swisher. At this year’s education rally, I had many candid conversations with people about what they’re dealing with at their own schools. I wondered how taking off the mask would impact the blog. It’s more popular than ever. Page views, Twitter followers, and Facebook likes affirm that. I just wish I had more time to write.

As Rob explained this morning, we still have much to keep us angered. We don’t fight for self-interest. If that were our motivation, many of us would have changed careers years ago. We fight because we want our schools to be places that help children thrive rather than places that demoralize them. We want teachers to be taken more seriously than tests. Thanks for reading; here’s to another year!

 

About the Governor’s Race

I’ve been quiet the last couple of weeks, mainly just enjoying my summer. I go to work. I come home and do things not related to my job or education policy. I catch up a little on Twitter. Otherwise, I’ve been staying low key regarding politics, and enjoying every minute of it.

In June, I was the blogger who wouldn’t shut up, and it wore me out. Before work, I was researching and writing. After work, it was more of the same. I was tired, but it was worth it. As David Blatt pointed out today, the rise of activists on social media probably contributed something to the defeat of Janet Barresi in the Republican primary.

The anti-Barresi movement was united by frustration with high-stakes testing and inadequate funding of public education. The A-F school grading system, mandatory third-grade retention and efforts to expand charter schools all stoked the feeling that the superintendent and her supporters were bent on implementing an ideologically driven agenda at the expense of teachers, students and parents.

The movement, which identifies itself by the Twitter hashtag #oklaed, includes many strands playing different roles. Statewide organizations of superintendents, school board members and teachers spread information to their members across the state. Civic groups like the Parents Legislative Advocacy Committee, the PTA, and Voice effectively educate parents and bring them to the Capitol to lobby their legislators.

This year, these advocates showed their organizing muscle by mobilizing 25,000 Oklahomans for a rally at the Capitol. They showed their political muscle by defeating legislation to expand charter schools and getting the Legislature to override the governor’s veto of a bill to give parents and educators more control over retention of third-graders. And of course they delivered their knockout blow to Barresi in June.

When I started this blog in 2012, it was never my intent to focus so much on one individual. I’m still more pro-public education than I am anti-Barresi. In most political races, I have no desire to endorse candidates. When I’m not blogging, I’m quite free with my political views – much to the chagrin of family, friends, and colleagues. On the blog, however, I don’t think I need to endorse candidates. I’m not a newspaper with an editorial board. I’m an individual with strong views about my profession and the children we serve. On the other hand, when the preponderance of evidence shows – as it has with Janet Barresi – that a public official has actively harmed public education, I have no problem stating the case that we should elect someone else.

At the same time, I’m not a single-issue voter. Public education is probably the biggest focus I have when it comes to state politics, and with the state superintendent’s race, it’s an easy focus to maintain. With our legislators and governor, however, we have to ask ourselves how much our passion for public education matters when we look at the big picture. When I ask myself, “Is Mary Fallin the best possible governor for Oklahoma,” the analysis is much more complicated than one issue.

Over the next few months, I will occasionally break down the race between Democrat challenger Joe Dorman and Fallin. Today though, I want to start with yesterday’s news that Fallin and Joy Hofmeister – the Republican who ousted Barresi – have pledged support for each other in this November’s elections.

“Joy Hofmeister is a teacher, small business owner and a mother who cares deeply about public education in Oklahoma, which is why I was proud to appoint her to the Oklahoma State Board of Education. I know Joy will work tirelessly to unite parents, teachers, employers and lawmakers as we work to support and improve our schools. I am proud to support her in her race for superintendent.” – Governor Mary Fallin

“Governor Fallin has always said that improving education is the most important thing we can do to support the long term growth and prosperity of our state. She should be applauded for highlighting the importance of public education, not just in the individual growth of our students, but for Oklahoma’s long term economic well-being. I encourage Oklahomans to get behind Governor Fallin to ensure we have a pro-education governor for the next four years.” – Joy Hofmeister

These are both very nice statements, but as many in the print media and social media have noted over the last few weeks, Fallin has actively distanced herself from Barresi. I noticed this late last fall when the state superintendent always seemed to mention the governor’s name, but with no reciprocity. It’s clear that attaching herself to Barresi’s toxic personality would not benefit Fallin politically. Surrounded by many astute handlers, the governor kept putting more space between the two of them.

While Mary Fallin may not be tight with Janet Barresi anymore, however, their education policies remain intertwined. As chairperson of the National Governor’s Association, Fallin has pushed strongly for the Common Core. She opposed HB 2625 which gave parents a voice in the retention decision of third-graders – in lock-step with Barresi, who called the Legislature’s override of Fallin’s veto pathetic and outrageous.

By the way, it was after that override (by a combined 124-19 margin) that I realized the power of the #oklaed movement. Apparently Fallin did too. She flipped her support for the Common Core into a signature of HB 3399, which eliminated it in Oklahoma (a change of heart that could have major unintended consequences in terms of increased federal oversight). Even her campaign website still proclaims her love of all things Common Core.

Though Fallin received good press after speaking to the state PTA last week for backing off the third-grade reading test, her actual words do not show much of a change. And her website still shows she supports high-stakes testing for eight- and nine-year olds. Here’s how Rob Miller explained it.

In her prepared remarks to the PTA delegates, Governor Fallin said, “If we can get to a system where we are measuring a student throughout the progress of their education versus one test — one high-stakes test — we are better serving the children.”

As you recall, just two months ago the Governor made waves with her controversial veto of House Bill 2625. This legislation allows districts to implement “probationary promotion” by incorporating a committee of school personnel and parents in making final determinations on student retention. Her veto came despite the fact that the bill was passed by large majorities in both the Oklahoma House and Senate. At the time, the Governor was adamant that the RSA law should remain unaltered, saying HB2625 “returns us to a system that has failed Oklahoma children for decades.” Despite her strong objectives, the House and Senate voted overwhelmingly to override Fallin’s veto.

The “thump thump” sound you may have heard later in the Governor’s remarks was the sound of Superintendent Janet Barresi being thrown under the bus.

This happened when Governor Fallin remarked that testing accommodations should be restored for children in special education or English language learners. This year, all students but the most severely disabled were required to take the same standardized tests as their peers despite their disabilities.

When asked to clarify her remarks on special education students, Fallin said she always felt they should be accommodated and attributed the current Education Department policy to State Superintendent Janet Barresi.

“That’s been her position. Now I’m telling you what my position is as governor. The superintendent is an independently elected official. She has her ideas. I have my ideas,” Fallin said.

Thump thump!

She also still supports the A-F Report Cards and Value Added Measurements for teachers. These are positions far more insidious than the Common Core. I don’t care what standards are in place; if we insist on using flawed tests (or any tests, for that matter) to measure teacher quality and make critical decisions for students, our path is sorely misguided. Fallin is part of the reason that we will have to remain vigilant against the expansion of charter schools and voucher programs. She has done even less than Barresi to restore funding to public education.

In the end, I don’t know how much the other issues impacting our state matter to you. I’m not a straight-party voter, and some of the things I support would probably surprise you. When I consider the state of public education in Oklahoma, though, I cannot in good conscience support Mary Fallin. She has damaged public education. Sure, I understand that these two Republicans supporting each other is a political thing. I am also pretty sure it helps Fallin a lot more than it does Hofmeister. Yes, Joy would work well with Fallin, but based on my own meetings with her, I think she’d work pretty decently with Dorman too. Besides, there are two other state superintendent candidates, and once they sort out their own differences, Hofmeister will have to demonstrate why she is better than the one who remains. Oklahoma may be the reddest state in the country, but that doesn’t mean we vote with our eyes closed.

I want a governor who supports public education. Since we can’t bring back Henry Bellmon, I’m looking for the one who is close.

Reason #14 to Pick a New State Superintendent: Value-added Measurements

Friday’s reason to pick a new state superintendent – anybody but Janet Barresi! – centered around  the time and money that our state has spent on two of the central reforms that we’ve now abandoned – Common Core State Standards and PARCC. This tweet from a superintendent frames the situation nicely.

#oklaed Its not the scrap of #CCSS that I'm in fear of, it's what our leg's will approve when written. Will politics win again or education?

— Tyler Bridges (@bridgestyler) June 7, 2014

This week, both camps (the CCSS supporters and detractors) are trying to figure out where we go at this point. We just blew $4 million on a contract with Measured Progress, and now we won’t even get to keep any of the questions we field-tested.

Blog reader okteacher posted the following comment on Reason #15:

I feel like milkshake has been dumped all over my head. This is such a mess. We have worked so hard to meet these rigorous standards and now we are going backwards?! This is like a warped time machine. I did have some issues with some of my 6th grade ELA standards, like requiring 12 year olds to be able to type a minimum of 3 pages in one setting, but we were doing our best to make this happen. Will going back to PASS make our students fall behind everyone else in the nation that is continuing on with CCSS? This is such a joke. All we can do is laugh at this point and pray that Janet’s days are numbered to 17 from this moment.

This doesn’t resolve our main issue of testing. This just makes things more complicated. I am so embarrassed to be represented by these fools. Every time I think we may take a positive step in Oklahoma education, they continue to dump milkshake everywhere.

The comment is spot on, and thankfully, we’re keeping the milkshake metaphor going. I will make the effort to continue in this post. First, here’s a rundown of the last several reasons:

#18 – The Hearing no one Heard

#17 – 2K4T

#16 – Questionable Personnel Decisions

#15 – Pulling out of PARCC

#14 – Value-added Measurements

For two years, I have said that there are things far worse than the Common Core. If they just consisted of standards that resembled (without completely mimicking every other state), they wouldn’t be so bad. We’ve totally convoluted our testing in this state, and I’m not entirely sure the Common Core is to blame. No, the greatest insult to the profession we’ve ever seen (even more than Governor Keating calling teachers slugs in the 90s) is the idea that we can measure the contribution of every single teacher on every single student.

Here’s how the SDE introduced Value-added Measurements (and more specifically, the process of roster verification) to us in the fall:

Roster Verification Coming Soon!

In order to successfully collect data for the 35 percent quantitative portion of TLE, teachers will utilize a process called Roster Verification to properly link themselves to the students they teach.Why is Roster Verification important?  This process is important because no one is more knowledgeable about a teacher’s academic responsibility than the teacher of that classroom!  Rightfully so, teachers should have the opportunity to identify factors that affect their value-added results (e.g., student mobility and shared-teaching assignments).In order to assist teachers throughout this process, the Oklahoma State Department of Education (SDE) has partnered with Battelle for Kids (BFK), a non-profit school improvement organization. Together, SDE and BFK will provide teachers with an easy-to-use data collection instrument, Roster Verification training, and communication resources. During February, 2014 the Office of Educator Effectiveness is hosting webinars on Roster Verification.  The webinars will explain how to use the Batelle for Kids program to link students and teachers accurately.  Five sessions will be offered at various times.  We encourage administrators and/or data personnel to sign up for a session.  The same information will be covered at each session, and one session will be recorded and posted on the TLE Web page to access anytime.

The question we’ve all had since the SDE started developing the quantitative measurements for TLE has been, How will they measure growth from year to year, especially when we keep changing tests? And once again, we’re changing tests. Unfortunately though, we have no idea what next year’s tests will be. In the end, someone in the field of research or public relations will tell us that we have found a way to compare scores. On the other hand, these are the same people who keep telling us that the 3^rd grade Language Arts test is a Reading test, and that it actually diagnoses reading level. They’ll tell us anything.

Another comment on Friday’s post was made by fellow blogger Mad Ramblings of a History Teacher.

I served on the working committee for VAM. I attended 2 of the 3 meetings (I missed the 3rd because of the weather). Later we were told there were 4 meetings (Where that 4th one came from? I never quite figured that one out.) Anyway, these meetings were a mess from the beginning to the end. And pardon my vulgarity, but where the HELL they came up with their “formula” is still a mystery to me. I don’t even understand the formula, and there, I believe, is the answer to your question about the rebellion over VAM. No one knows or understands exactly what will happen when VAM is calculated into their evaluations. I think the rebellion will begin after the adverse effects of the VAM formula start showing up on teachers’ evaluations. Just look at Houston; several teachers are suing over the use of VAM. One thing is for sure; it’s going to get even more interesting around.

Mad Ramblings had a nice post about this in February called My Shameful Secret.

I have a terrible confession to make. I was one of the teachers on working group #3 for Oklahoma’s TLE VAM sessions. In my defense I thought I was doing something good. I know I am not the smartest person in Oklahoma, but I was vain enough to believe my fellow teachers and I could bring some much needed sensible experience to this “groundbreaking” ill-advised venture Oklahoma’s DOE was determined to try. Boy was I mistaken!

They had Mathmatica. This is the same organization that gave D.C. such wonderful advice. It was so good that D.C. wrongly fired some teachers, and the method used to interpret the data so inherently flawed it can’t be fixed. No matter, Mathmatica just moved on to another state spewing the same propaganda and pocketing the big bucks. But I digress.

I thought the teachers would be treated as professionals at these meetings. .. wrong again. If we brought up a topic they were unwilling to touch, we were too political or going off topic. Then to explain what they were wanting from us, they sent us papers written with grants from the Gates Foundation. Talk about being too political!

I’m just going to throw out there that if it’s not ok to have academic standards that were written out of state, we probably don’t need talking points for VAM that were as well. Again, the whole idea is that we can input a whole bunch of variables into a formula and determine who is effective. For the teachers in subjects that don’t test, we’ll just make up things that we can measure and do that.

The thing with VAM is that it manages to fly under the radar. The few times I’ve written about it, I haven’t had a lot of page views or shares. That isn’t to say that it’s not worth my time to listen; rather it’s as if CCSS is an in-your-face affront to so many while VAM is going to remain so low-key that we miss it. Maybe it’s that we can read the Common Core and very few of us could read (and fewer comprehend) the VAM formula.

This is worse. And the SDE can’t explain it even reasonably well. For the first year that they were developing the quantitative piece of TLE, the two people directing the effort had never even been in charge of evaluating teachers. This year, they added a former principal, but largely, we still have an experience gap.

My fear is that we’ll miss it until it’s too late. Worse yet, we’ll borrow other states’ talking points for adoption without paying attention to their experiences, unintended consequences, and then eventual extraction.

In other words, we’re going to spend years and millions on another acronym only to toss it later. Once again, instruction will suffer as a consequence. This is what happens when the amateurs running the show make decisions without consulting the people doing the work.

Speaking of the amateurs at the SDE, please read Rob Miller’s research on the myth of Barresi cutting costs at the agency. It’s a good companion piece for my #16 reason.

I know that teachers try not to be political. I know many of us have worked in school districts that flinch at activism. We have nothing to be afraid of. We have the momentum. And we have right on our side.

We have 17 days. We must continue being political and speaking loudly. Use your outside voice. Do whatever it takes.

Roster Verification: The Pilot

In this episode of Roster Verification, Janet and her friends experience some wacky shenanigans and unfortunate misunderstandings. Mr. Roper comes in and makes everybody feel terribly uncomfortable. In the end, everybody learns a valuable lesson about hubris.

In the television universe, production companies develop single episodes of new shows to try to sell a series to a network. This is called a pilot. In a typical year, about three pilots are developed for every show that airs.

An actual television pilot

In education reform, we only tend to pilot programs to which we have already committed, either through policy or contract (or both). That’s why teachers and administrators were excited this week to receive the following email from the SDE:

Roster Verification Coming Soon!In order to successfully collect data for the 35 percent quantitative portion of TLE, teachers will utilize a process called Roster Verification to properly link themselves to the students they teach.Why is Roster Verification important?  This process is important because no one is more knowledgeable about a teacher’s academic responsibility than the teacher of that classroom!  Rightfully so, teachers should have the opportunity to identify factors that affect their value-added results (e.g., student mobility and shared-teaching assignments).In order to assist teachers throughout this process, the Oklahoma State Department of Education (SDE) has partnered with Battelle for Kids (BFK), a non-profit school improvement organization. Together, SDE and BFK will provide teachers with an easy-to-use data collection instrument, Roster Verification training, and communication resources.During February, 2014 the Office of Educator Effectiveness is hosting webinars on Roster Verification.  The webinars will explain how to use the Batelle for Kids program to link students and teachers accurately.  Five sessions will be offered at various times.  We encourage administrators and/or data personnel to sign up for a session.  The same information will be covered at each session, and one session will be recorded and posted on the TLE Web page to access anytime. TLE Roster Verification Webinars Feb. 24, 1:00 – 2:30 PM;  Feb. 25, 9:00 – 10:30 AM;  Feb. 26, 3:00 – 4:30 PM; Feb. 27, 11:00 AM – 12:30 PM; Feb. 28, 9:00 – 10:30 AM To register for a webinar session, go to:  https://oksdetraining.webex.com and click on the “upcoming” tab.  Select one of the webinars titled TLE Roster Verification. As required by state statute, mandatory roster verification is scheduled for the spring of 2014 and should be completed by all districts. To learn more about roster verification, please access the following link: http://ok.gov/sde/tle-roster-verification

The catch is that 2013-14 is a pilot year. Districts must participate at all of their school sites, but they can select which teachers to use. They can use one teacher, one department, or the whole school. They are testing, more or less, how well the information tracks.

When Roster Verification is in full effect, we will eventually be able to calculate how much time each student spent with each teacher in each grade. That way, as the email suggests, we will know which teachers add the most value.

I’ve made my opinions on VAM clear before. We’re going to be making personnel decisions based on test scores. In some cases, these decisions will impact teachers in non-tested grades and subjects. Roster verification is a process by which we assign a percentage of responsibility to different teachers for a student’s growth. By responsibility, of course, I mean credit and blame.

If you’re a first grade teacher, eventually we will be able to tell you what percentage of the students you’ve taught passed the third grade reading test, took accelerated math classes in middle school, and graduated high school on time. We’ll also be able to tell you how many of your students were retained in third grade, struggled in math down the road, and dropped out.

To conduct Roster Verification (and VAM), the SDE has contracted with Batelle for Kids. Here’s how BFK describes themselves:

Battelle for Kids is a national, not-for-profit organization that provides counsel and solutions to advance the development of human capital systems, the use of strategic measures, practices for improving educator effectiveness, and communication with all stakeholders in schools.

Those who have read this blog for a while know I get twitchy around the words nonprofit or not-for-profit. Essentially, I loathe the idea that you can count as a charitable donation money you have given to an organization that really isn’t a charity.

Looking up their most recent tax form 990 on Guidestar, I found out a few interesting things about BFK. Here is some basic financial information from 2011:

Total Revenue	$21,398,999
Total Expenses	$18,761,469
Revenue Less Expenses	$2,637,530
Beginning Fund Balance	$8,896,988
Ending Fund Balance	$11,534,518

With such a healthy ending fund balance, I do hope they gave all of their employees a $2,000 raise!

This clearly is a non-profit on the rise. As I’ve said before, I don’t mind people making money. Profit is a good thing. I just abhor the doublespeak of non-profits making so much money. Where they make and spend their money is also interesting. They are heavily funded by the Bill and Melinda Gates Foundation. They have taken money from Race to the Top. Their top 11 executives all have six-figure salaries (as do two additional consultants).

Imagine the outrage if a school district in Oklahoma with a $21 million budget had 11 employees making over $100,000 (with the leader earning about $421,000). Imagine the outrage if Governor Fallin’s supporters knew that Oklahoma’s teacher evaluation system was entangled with Race to the Top, the hallmark of federal interference.

Roster verification does not benefit students. It does not give parents or teachers more information to make decisions. It simply creates additional work for already overextended teachers and principals while lining the pockets of out-of-state companies that are beholden to the corporate reform agenda.

Unfortunately, we know the network has picked up this pilot and bought several seasons worth of episodes.

Two Year Delay for TLE?

Yesterday, the SDE sent out a media release stating that they would be requesting a two year delay for full implementation of the Teacher/Leader Effectiveness system. The content of the release was linked to the SDE website, but is now down, probably due to a technical problem. The release reads as follows:

OKLAHOMA CITY (March 19, 2013) – State Superintendent Janet Barresi announced today that she will ask the authors of Senate Bill 426, Sen. John Ford and Rep. Earl Sears, to consider a two-year delay for full implementation of the Teacher and Leader Effectiveness reform that was originally enacted in 2010 in Senate Bill 2033.

In making the announcement, Superintendent Barresi stated, “After listening to input from teachers and superintendents across the state as well as teachers serving on working groups for the TLE Commission, I have concluded that this extra time is necessary to assure the entire TLE system is implemented with fidelity and to the high standards we expect of such a critical reform.”

“Nothing is more important than assuring that each child in our state has the opportunity to be taught by an effective teacher and school principal. We will continue to work with the TLE Commission and the State Board of Education to build a model program and quality technology infrastructure to support the program. I appreciate Gov. Fallin’s support in this decision and our work,” Barresi added.

Governor Mary Fallin said, “Studies show that the most important driver of student success in the classroom is high quality teachers. That’s why it’s so important that we get these reforms right. Giving Oklahoma schools adequate time to properly prepare for TLE implementation is in the best interest of everyone. I strongly support TLE and look forward to full implementation so we can utilize performance pay options and other compensation models tied to the system.”

Superintendent Barresi suggested the timeline for implementation of the qualitative or observational component of the system is currently being piloted this year and will be fully implemented in districts for the 2013-14 school year. If SB 426 passes, the Other Academic Measures portion of the quantitative component will be piloted next school year and implemented in 2014-15. The 35 percent accountability measures of the quantitative component will be implemented in the 2015-16 school year, making TLE complete.

Per state statute, the Oklahoma State Department of Education is working in conjunction with the TLE Commission and working groups of educators throughout the state to develop a robust professional growth tool known as the Teacher and Leader Effectiveness evaluation system. When fully implemented and utilized properly, TLE will identify the direct cause-and-effect relationship between teaching practices and student achievement using both qualitative and quantitative measures.

A teacher’s evaluation will be based on 50 percent of qualitative measures such as classroom observations and 50 percent quantitative measures. Of the quantitative measures, 35 percent will be based on student test scores for tested grades and subjects and the remaining 15 percent on Other Academic Measures as defined by the TLE Commission and educators. How teachers are evaluated in non-tested grades and subjects for the quantitative portion is still being discussed.

SDE staff have been very direct for months now that they would be asking for this delay, which will be appreciated by most teachers and administrators. If they need more time to work out the details, that’s fine. The cynic in me believes that something else is happening here, however.

Two things are missing at this time. First, as we heard back in January, the SDE is getting input from SAS to develop a model for creating a VAM model. (Yes, that was three acronyms in a single sentence.) Without a mathematical equation that includes factors both within and outside of a school’s control, there won’t be a value-added measure. This would make the recommendations of the various working groups meaningless at this time. The other piece in development is a more refined student data system – one that can effectively track where students were and for how long and which teachers impacted their learning and for what percentage of the time.

That brings us to roster verification, which I wrote about last week. This is a new experiment that the SDE wants to run before full implementation of TLE. If we start calculating the quantitative portion of the evaluation without these pieces, it will be harder to add them in.

A delay will ensure that VAM and roster verification will be a piece of the enacted system. It will also guarantee that we will have conversations like those taking place in Florida right now – ones in which successful teachers get low ratings because of the students they do not teach. Call me ungrateful, but rather than waiting to get it right, we should instead acknowledge that the entire concept is fatally flawed.

*****

Update: the SDE press release is back.

Teachers Respond to the TLE Commission

A group of Jenks elementary school teachers sent a response to the SDE this afternoon, expressing their concerns over the recommendations for quantitative measures for evaluating teachers that I wrote about this morning. This group includes art, music, and physical education teachers. I’ll let their words speak for them, first with an excerpt, and then with a link to the full letter:

We the undersigned and highly qualified specialists at Jenks East Elementary School urge our legislators to seriously explore the quantitative component of TLE before 2013/2014 implementation. Below you will find our “real time” experiences and “real voices” speaking facts which must be considered before Oklahoma implements a “one-fits-all” approach for evaluating educators and determining their compensation.

This insightful four-page letter didn’t just go to the SDE, however; it went to the 250 members of the TLE working group. And they told two friends. And they told two friends. And so on. And so on. And then it found its way to your friendly neighborhood blogger. It’s the gift that keeps on giving.

Later in the afternoon, the Jenks teachers (and the entire working group) received a response from the SDE:

Ms. Riggin and Special Area Teachers (ART/MUSIC/PHYSICAL EDUCATION) of Jenks East Elementary,

Thank you for your careful thought, consideration, and time in preparing the document you provided for us. We appreciate your input and respect your perspectives to this challenging work. The working groups’ final recommendations were presented to the TLE Commission for initial consideration on Tuesday, March 12, 2013. I will share your additional input with Superintendent Barresi, who is also the chair of the TLE Commission. If she has any questions or follow-up requests, I will get back in touch with you.

I’m glad that our state has some teachers not taking this quietly. Read the whole letter. It’s worth your time.

And Then There Was Roster Verification

I received an email last night with a 30-page document attached showing the different recommendations to the Teacher and Leader Effectiveness (TLE) Commission by the various working groups that developed them. In all, the Value Added/Student Growth Measures for Teachers of Non-Tested Grades/Subjects and Teachers Without a Teaching Assignment. In all, this document contains suggestions for how to quantify the effectiveness of 18 different classifications of certified teachers. These include:

CareerTechnology	Counselors
Early Childhood/PreK	Elementary (1-6) Non Tested
English Language Learners	Fine Arts
Gifted/Talented	Instructional Coaches
Library Media Specialists	Nurses
Physical Education	Reading Specialists/Response to Intervention
School Psychologists	Secondary: Non-Tested Subjects
Special Education	Speech Language Pathologists
Technology	World Languages

I am repulsed by the idea that we have to come up with some sort of a quantitative measure to evaluate some of these groups of teachers (and nurses, really!?!), but I decided to play along and read through the sections. What I found were some drastically disparate ways to calculate teacher effectiveness. All include some level of new training for both the teachers and the principals who would evaluate them. And most ask for more time to come up with a workable plan.

The different recommendations include some similar language that we rarely use when talking about education reform initiatives. We are going to have to learn these terms in the same ways that our students learn the academic vocabulary. I have already discussed Value-Added Measures (VAM) on this blog; I am not a fan. I do not believe that an agency incapable of developing a statistically-sound report card can develop VAM in a way that is fair to teachers. I’m not convinced that it is achievable in the first place.

Several of the proposals also call for some kind of a matrix, portfolio, or rubric to assess teachers. Principals would have to become familiar with all of these instruments. They also call at various times for different weights on the quantitative pieces of the evaluation. Imagine keeping track of all of that!

The newest term for educators, however, is Roster Verification. The only group that mentioned this process in the report to the Commission was Special Education. In an email to superintendents and principals this week, here’s how the SDE described Roster Verification:

Roster Verification – Voluntary, yet Valuable! The Oklahoma State Department of Education is offering Roster Verification as a service to school districts this spring. The OSDE will be completing value added analysis for all teachers of TESTED grades and subjects after testing occurs this spring. Value added analysis will be used for INFORMATIONAL purposes so that teachers and administrators have the opportunity to learn about the process and can use data to inform instructional practices during the 2013-2014 school year. This is a NO STAKES process meaning NONE of the value added calculations will be used in evaluations. Because there are so many different teaching scenarios that occur throughout the year, Roster Verification allows teachers to account for who they taught, for which months during the year, and for what percentage of the instructional time. For example, when I taught 5th grade, we were departmentalized. I was responsible for MANY students’ instruction in mathematics and science, but my team member was responsible for their instruction in reading and social studies. Without completing Roster Verification, my value added analysis would be based on my HOMEROOM roster (unless someone uploaded that information differently into the Wave.) As a teacher, I would want to be held accountable for the growth of the students I instructed in math and science, but I would want my partner to be responsible for their growth in reading and social studies. Roster Verification gives teachers the ability to account for such scenarios, therefore value added analysis reports are much more accurate for teachers who were able to complete the Roster Verification process.

The SDE provides even more detailed information on this flyer, which includes training dates, a shout out to the Gates foundation for funding, and a picture that would lead you to believe this is about children. One line even promises that Roster Verification will provide “much more accurate value added reports which will be extremely useful as a professional growth tool.”

This is not remotely about professional growth. This is about continuing down the path of assigning blame, and trying to find a mathematical formula for doing so. In ten years, we will be able to look at the students of two second-grade teachers and see which ones are better prepared for college. We will be able to assign partial credit/blame for the success/lack-thereof to all the teachers those students have ever had. Over time, we’ll have all kinds of data pointing back to that second grade classroom.

Think back to that bizarre oak tree analogy. The disembodied voice in the video tells us that countless factors go into two farmers raising their oak trees. It also tells us that some of those factors are out of the control of the farmers. (By the way, I still don’t know any oak tree farmers).  Removing all of those factors, the argument is that we can tell which farmer is adding more value. Extending the analogy, we can remove the factors out of the control of teachers and look at results, and ascertain value-added that way as well. This too makes me uncomfortable.

The factors out of a teacher’s control are too many to count. We will be assigning value-added sometimes ten or twelve years after the fact. Would you want part of your evaluation during your eleventh year in the profession to be based on something you did your first year?

Many districts have chosen not to participate in Roster Verification at this time. Others, for the sake of curiosity, are joining in the trial run. I understand both positions. While I can’t think of one school administrator who wants to see this happen, many want to see what the process looks like since it will happen eventually anyway.

The recommendations to the TLE Commission are non-binding. Commission members can act to accept, revise, or reject the proposals at a later date. Meanwhile, the SDE is wisely pushing for more time to implement the quantitative piece of the evaluation system. While they would be even wiser to scrap it altogether, that won’t happen. Too much money –taxpayer and corporate money – is invested at this point. The agency is philosophically entrenched in this process.

January Review / February Preview

School Choice Week is ending with a whimper. Other than another sad editorial in which Brandon Dutcher of OCPA begs the state’s taxpayers to subsidize his choice to homeschool toddlers, it has been pretty uneventful. Fortunately for this blog, January as a whole has been more interesting. Statistically speaking, this has been the second busiest month since I started writing, with almost 6,000 page views. February will see the beginning of the legislative session and more questions raised by the email trail between the SDE and its puppet masters. I can hardly wait.

Here’s a look back at the top five blog posts for January:

1. Mid-term Adjustments: Virtually Unbelievable – School leaders came back from Christmas Break to find that their mid-term adjustments had been sent to a handful of charter and virtual schools. And they were pretty frustrated about it. That fact – compounded with the discretionary spending decisions made by the SDE have led us to the point where schools have to have a supplemental allocation at the beginning of the legislative session just to fully fund employee benefits and the Reading Sufficiency Act.
2. In Defense of Superintendents – When reformers bash the education establishment in general, I barely flinch anymore. When the Oklahoman directly rips two Tulsa-area superintendents – one on his way out the door, I actually look at my screen with mouth agape. I don’t want to sound like a cheerleader for Jenks Public Schools, but there’s a reason everybody hates them. They take pride in being good at everything. At a time in which we are obsessed with measuring everything, you would think reformers would love Jenks. This just goes to show that those in power have long memories and hold grudges.
3. I am a Teacher; I Add Value – Sometimes, when I’m not refuting the baseless lie that public education stinks, I retreat to a more creative state. While I think I’m pretty good at connecting the dots from fact to fact to fact and surrounding the false rhetoric out there, sometimes, I just have to speak passionately about the many professionals I know personally. Each paragraph in this post is someone I know. Like me, they’ll remain anonymous.
4. Tie an Incomprehensible Ribbon … – The popularity of this post surprised me a little. Usually, there’s a negative return on investment when I drop too many pop culture references into a post (especially one from the 70s). I decided to watch the recorded videoconference discussing the procedures for adopting OAM and VAM and VARC, and there was an analogy about an Oak Tree, and zzzzzzzzzzzz………. Moral of the story, don’t watch SDE videoconferences if you don’t have to.
5. Testing: To Profit or Not? – Whether it’s testing or charter schools, the line between companies making money and those not is very thin. In fact, it doesn’t exist. Even the non-profits exist to make money. College Board and ACT wouldn’t be who they are if they were losing money. They advertise and develop new product lines, just like companies that make a profit. And our state is in deep with them.

Come on February, let’s follow that up with something spectacular. I know you can do it!

I Dare You to Bring CornNuts

I mentioned a couple of weeks ago that the SDE was accepting volunteers to be on the TLE working group that would inform the TLE Commission in its task to select Other Academic Measures to use in the evaluation of teachers in non-tested subjects. According to emails forwarded to me by readers, over 200 people signed up. I can’t take credit for all of that, but maybe an icebreaker activity for the working group can be developing a formula for evaluating the impact of this blog.

One can dream, right?

Seriously, though, the group has gotten so big that the SDE is dividing them up into three smaller groups. The first meeting is today. Participants have been told three things and three things only:

1. The meeting is at Langston-OKC
2. Dress comfortably
3. Bring your own snacks and drinks

One can infer from the second point that a dress Snuggie™ (I haven’t checked, but you know they have to make one like a tuxedo) would be appropriate. The third point raises several possibilities, however. Clearly there will be no chocolate fountain and crabcakes. I do suggest bringing something, though. A full day without nourishment can be brutal.

I had a college class with a guy who always ate CornNuts. It didn’t matter where you sat in the room, you could hear them and smell them. It’s not a horrible snack if you are the one eating them, but I have a thing about sounds and smells. 

Overall, the variety of snacks is likely to be more interesting than the exchange of ideas. It will certainly be more fulfilling. Hopefully the snacks people bring with them will be less irritating than the topic at hand.

The truth is, your ideas are merely a prop. The real work on VAM is being done by a company called SAS out of Cary, NC. Fortune Magazine just ranked them 2^nd on the list of top 100 companies to work for in the United States – right behind Google. Business analytics are a growth industry, after all.

In all seriousness, if you go today, speak up. In all likelihood, the SDE staff that will be there have their mind made up about what measures will be used. That doesn’t mean that collectively, we can’t make it interesting.