VAM: Better Never than Late

Let’s talk about teachers for a moment. Some are great, some are decent, and some are better suited for another career. We knew this when we were students. We see it as parents. It’s even obvious to us sometimes as colleagues down the hall. The vast majority of teachers fit into the first two categories. Of that, we should be quite proud. Just the same, nearly all schools have someone who brings us all down.

If there were an instrument – a valid, reliable one that I believed could tell me numerically who my best teachers are – I’d use it in a heartbeat.

There’s not.

When I was teaching Honors English II classes in Mustang, most of my students could have passed their end-of-instruction exams before they even had one day with me. There was no standardized instrument suited either to measure their learning or my effectiveness. They were already hitting their heads on the ceiling of every test they had ever taken. Another test showing the same thing meant nothing to them.

As a teacher, I received great evaluations. Those also may or may not have meant anything. I can only remember two negative comments, both from my first year of teaching, when I was in Muskogee.

The first (from an assistant principal) was a question about whether or not I noticed a certain student chewing gum while I was teaching. Yes, I had noticed it. No, I didn’t want to interrupt the momentum of instruction to draw attention to it. By the end of the hour, I had forgotten about it. Yes, he got away with breaking a rule, but it wasn’t worth stopping and starting again.

The second (from the university professor on my entry-year committee) was a comment about finding something instructional for students to do when they finish an assignment because – wait for it – all they’re doing is reading.  Claudia Swisher, I should have told you to turn away. Oh, the horror! Eighth graders reading, without anyone telling them to! I think his point was that I should have been teaching bell-to-bell. If it was something else, it was lost on me.

I think as a first year teacher, I had some very good moments. By the time I left the classroom, I think I was a very good teacher. I was never great, though. I didn’t have the years of experience (nine) or consistency to claim that. I loved it, but we’re not automatically good at the things we love. I love to sing in the car. I love basketball.

If you looked at my evaluations when I was in the classroom, though, you would have thought I was the very model of a modern master teacher. All of the check marks were in the far right column (the good side). Occasionally, I’d have a few encouraging comments like “try beginning class with an activity to engage prior knowledge.” Casually (not in writing), I would receive suggestions about classroom management or working with parents and colleagues – normal things that young teachers need to learn. Still, my evaluations would have all the check marks lined up in the right boxes.

That was the old teacher evaluation system. In 2011, the Legislature – acting in conjunction with then State Superintendent Janet Barresi – passed legislation creating the Oklahoma Teacher and Leader Effectiveness Evaluation System, or TLE. Even with later legislation amending it, the TLE law includes both qualitative and quantitative pieces. Since not all teachers give a state test, and since not all state tests are paired in ways that can show growth, two different quantitative measurements were created: Value Added Measures (VAM) and Other Academic Measures (OAM). The OAMs are no longer a thing. They have ceased to be. They are now ex-quantitative components.

Let me come back to VAM a few paragraphs from now. On the qualitative side, I have seen two of the evaluation models in action. When I was in Moore, we used the Tulsa Model. In Mid-Del, we use Marzano. I honestly have no preference. The language describing the different ranges of teachers is about the same.

With the way we used to do teacher evaluations, as with TLE, what really matters is how committed principals are to improving instruction. Do they have the resolve to have difficult conversations with teachers? Do they use the evaluation model with fidelity? Or, to borrow from Garrison Keillor, is the school the kind of place where “all the [teachers] are above average”?

We can have a well-researched qualitative teacher evaluation system, and we can make districts pay for training in the summer so that principals learn to calibrate their scores for teachers. It’s like shooting free throws in practice. When you have the pressure of giving a teacher a low score, even though you personally like that person, or even though his last 10 principals gave him a good evaluation, what will you do?

I’ve jumped in with both feet, and I know many other principals who have too. It’s not an easy thing to do, but at least you’re doing what seems right based on what you actually see. Then there’s VAM.

To date, no teacher in Oklahoma (that I know of) has had a VAM score added to his/her evaluation. No principal or superintendent I talk to has faith in them. It also sets up a two-track system for evaluating teachers – one for those with a VAM score, and one for those without. It’s inequitable on its face.

That is why I was less than enthused to see this in my email yesterday:

Value-Added Results Now Available

Value-added results demonstrating student academic growth during the 2014-15 school year are now available for teachers and administrators through the SSO2 portal. Guidance documents about how to access and distribute these reports can be found on the Teacher and Leader Effectiveness (TLE) page on the OSDE website.

If, by chance, I cared about VAM scores, I would probably want them sooner. Why would I attach scores that ostensibly show a teacher’s effectiveness last year to this year’s evaluation? Since I don’t care and I wouldn’t use them, we need them to go away. In Mid-Del, I don’t even know how many certified and support employees in my district spend how many hours preparing for the Roster Verification process. It’s all a waste of their time. Furthermore, the SDE spends over $600,000 on contracts with out-of-state vendors so we can verify who had this kid for what part of that month and how to calculate VAMs that nobody uses. Every penny of that is a waste of resources that continue to melt away.

One bill that we can still support to end this madness is HB 2957. In its current form:

• Districts would have the option to use quantitative measures in their evaluation process; but it would no longer be required;
• Teachers and administrators would develop a yearly individualized program of professional development;
• This would be a collaborative effort between the evaluator and the teacher/principal.
• The focus would be on components from the qualitative framework, but not necessarily on low areas;
• This is not intended to increase the amount of required PD hours, but rather to focus professional learning on areas that lead to higher student achievement;
• VAM would no longer be required by the state (and hopefully no longer purchased by it either); and
• Career teachers receiving a district rating of “highly effective” or “superior” would only need to be formally evaluated once every three years.

It passed the House by a vote of 94-0. It passed the Senate with amendments 46-0. Now the House needs to approve the changes and send it on to Governor Fallin. Easy, right?

Not this year. Nothing is easy this year. Nor is it logical.

Here’s hoping…

 

 

 

 

Two Bills to Support

Yesterday was pretty anti-climactic. The morning was loud and at times, contentious. The afternoon was like a balloon with a small pin prick. Slowly, and noticeably, the air went out of the Senate first, and then the House.

I won’t spend a whole lot of time on that. Before I discuss two bills that warrant your support, I want to share with you the experiences of Oklahoma teacher and English/language arts standards writing team member, Kelli Anglley.

I had the unique opportunity to go to the state capitol today and speak with our legislators about the Oklahoma ELA standards that I helped author.

As I teacher, I often wonder why our legislators make the decisions they do. Today I gained some insight. Teachers obviously cannot go and lobby because we are teaching. However, other groups seem to have more time on their hands.

This group (ROPE – Reclaiming Oklahoma Parent Empowerment, formerly Restoring Oklahoma Public Education) was there in force. They were holding red signs that read “FIX AND VERIFY” in reference to our new standards. Some members of this group had no clue why they were there. I heard a lady say to another, “Why are we here again?” All she had done was answer a robocall plea to be at the capitol. It took all I had not to walk up with my copy of the standards and say, “Which one would you like me to fix and verify” because I am almost positive most have never even read them.

As legislators would walk past them, they would chant and and grab some for conversations about the bills they were interested in.

As members of the writing team walked by to enter the House Republican Caucus, where we were invited as guests, this group was chanting “STOP COMMON CORE” the whole time we walked down the long hallway.
1. Our standards are NOT Common Core.
2. I’ve never been on either side of a protest before, so that
was very odd.

My opinion is that this is why we get some of the crazy legislation we get – because there are crazy people up at the capitol bending our legislator’s ears. I feel that my presence there today, shaking hands, putting a face to the standards, and answering questions helped. However, I am very happy to be going back to my classroom tomorrow.

As parents and teachers, we need to get more involved. I’ll post a group in the comments that you can join if interested in current educational legislation.

I was there for a little while in the morning too, but I missed that scene. That’s probably a good thing.

1. Senate Bill 1170 – This bill would repeal End-of-Instruction testing and give districts control over testing and graduation requirements for high school students. This bill does nothing for grades 3-8 testing, which is fine with me. That’s more complicated, and I’m still not sold on anything we’ve seen to replace those tests. It’s a good start and would save the state money (and high schools valuable time).

2. House Bill 2957 – This bill would end the Teacher and Leader Effectiveness Evaluation System and place the job of teacher evaluation back into the hands of districts. As with SB 1170, it’s a bill that provides flexibility and local control.

(By the way, this is a stark contrast to SB 1187 which rolls back years of progress on teacher protections – get this…as a REWARD for being successful! That’s not the local control we’re looking for.)

HB 2957 also would save districts money. Time, flexibility, and cost-savings. These are always upgrades.

As hard as we’ve worked these last few weeks fighting madness, let’s keep that energy focused, and fight for things we want. These bills passed their chamber of origin unanimously this month. As I found talking to people at the Capitol today, there are more elected leaders trying to help us than hurt us. Let’s thank them and let them know we support good legislation.

Guest Post on the Teacher Shortage from a POed Parent

I received the following this week from UCO professor and #oklaed advocate, Dr. Dan Vincent. I present it to you, unedited. 

---

I’m a public school parent and I’m pissed off. I keep hearing that our state has a teacher shortage but I don’t see it this way anymore. I see an unusually high causality rate from the WAR ON TEACHERS.

Let me explain….

As a parent with two kids in public school I try to keep informed on issues related to education. I read the news, follow legislation and even research topics to be more informed. For the past few years, at the start of the schoolyear, I have read stories about the growing number of vacancies in Oklahoma classrooms—vacancies that districts cannot fill. Class sizes get larger and courses get cancelled. This number has gradually been creeping up and it has hit larger urban districts particularly hard. Now, even large suburban districts, where there has historically been an abundance of qualified applicants, are being hit by this shortage.

Over the past several years I have also observed waves of educational reforms crashing into the doors of classrooms and onto the desks of students—reforms initiated and passed into law by our state legislature. If you are a student or teacher, you’ve felt it; my kids have felt it. The changes included things like the A-F, the RSA, the ACE and the TLE to name a few. These have been widely recognized by educational leaders in our state as doing more harm than good, especially when it comes to teacher morale and student engagement. Professional associations, parent groups, blogs and personal anecdotes have documented how these reforms are negatively impacting Oklahoma districts, classrooms and kids. There has also been much written about how these reforms are DRIVING GOOD TEACHERS OUT OF THE CLASSROOM. Legislators have been told this over and over. Personally, I have had civil discussions about the issues I see; I have written umpteen letters to lawmakers pleading for change. I have friends who written many more.

So what I fail to understand with the ‘teacher shortage’ in our state is why –  WHY – legislative leaders have stood by and allowed this to happen. The teacher shortage is not an unforeseen consequence of a poorly timed tax cut, but the steady attrition of teachers who have HAD ENOUGH of nonsensical educational reform policy and poor pay. The teacher shortage is not an unavoidable crisis caused by federal laws, but a compounding of state-level educational policies that fly in the face of what is known about learning. And as a parent, I hold legislative leaders responsible; they have created a WAR ON TEACHERS and our teacher shortage is a sad result of this war. It is a moral failing by our state leaders in not taking seriously their job of properly supporting a free public education.

We know that money matters and we know that teaching climate matters. Legislative leaders have tremendous power over both and have done little to nothing to create REAL SOLUTIONS for teachers. In fact, I am not big on conspiracy theories but I am now seriously thinking our legislative leaders are purposefully making a teacher’s life miserable so they can justify their own policies meant to ‘help’ the problems in education—problems they have created with the war on teachers. And this is all being done TO OUR KIDS.

Imagine if we had a shortage of qualified STEM candidates to fill the jobs in our state. Do you think our current legislative leaders would do anything to attract quality candidates? Do you think they would initiate policy to help the STEM industry will those positions? Do you think they would be advocating for the STEM industry? Would our leaders actively seek out leaders in the STEM industry for ideas on how to attract applicants? Would they try to fill the STEM pipeline with qualified applicants?

You bet. In fact, Gov. Fallin says there is a STEM shortage in our state, and our leaders have already done the things above (in fact, our governor’s 3^rd annual STEM Summit is a few weeks away). But not for our teachers. Not for our kids. WAR ON TEACHERS continues.

A few weeks ago, I felt a glimmer of hope when I read House Speaker Jeff Hickman and House Republican education leaders calling for a “more cooperative approach” to address the teacher shortage. Not three weeks later however, Speaker Hickman wrote an opinion piece for the Daily Oklahoman blasting district administrators for not doing more themselves to pay teachers a higher salary; I also suspect School Boards felt targeted. I wonder if Hickman cooperated with any Oklahoma administrators on the ideas for this OpEd? I doubt it. WAR ON TEACHERS continues.

Just this week, the Republican leadership offered up a plan to allow retired teachers $18,000 per year to come back to the classroom and teach. On the surface, this sounds admirable, but honestly, how many retired teachers would be willing to work for that pay under the same educational environment that drove many to retire in the first place? Does this address the current issues our teachers face—pay and climate? Sounds like a Band-Aid solution to a war-time wound. WAR ON TEACHERS continues.

In short, the solutions offered up by republican leaders thus far only deepens my suspicions of how serious they are about addressing our state’s desperate need to put well-qualified teachers in EVERY classroom. My kids deserve better. Our state’s kids deserve better. So here are some things I would offer as solutions. I would encourage every parent, grandparent and relative that has a kid in school to write their legislator and tell them to end the WAR ON TEACHERS with some of these bullet points (no pun intended):

1. First and foremost, do your part to fix the educational climate in Oklahoma. Stop the blame game and be real about solutions to our teacher shortage. Ask the educational leaders in our state (who are really informed about the issues they see firsthand) for input and take it seriously.
2. Stop the High Stakes Testing (found in the RSA, the ACE, the TLE, the A-F). This would also save some money on administrative overhead and ink for signing RSA documents.
3. Seriously rework the TLE. It is well known that value added measures are junk science yet our state leaders insist they can work. This could also save money by reducing administrative overhead.
4. Stop the A-F charade. OU and OSU put together a pretty good summary of the charade. And this also could reduce administrative overhead.
5. Publicly support teachers, but more importantly seek out educational leaders so your public support can be turned into fully-informed legislative action.
6. Develop a workable plan to increase teacher pay. Money matters. Our state invests public money to support the STEM industry and others. Let’s get real about how to invest in the profession that can support all industry.
7. Either UNMANDATE or FULLY FUND. There are many unfunded mandates placed on schools and this solution could both create a better climate in schools AND free up money that could be used on teacher salaries. One good example would be to eliminate the ACE graduation requirement.

In closing, I honestly hope our legislative leadership can do something soon to refresh the souls of educators in our state. I hope parents will a) get pissed off with me and b) constructively express their frustration to leadership in our state. Their current attempts are a far cry from the real, workable solutions needed to address the root causes of our teacher shortage. With the upcoming session being near an election cycle, I think more ears will be open to listening.

Let’s end this war.

---

Dan Vincent is a former public school teacher, turned university professor who has two kids in elementary school. Although he was not teaching in the K-12 schools when many of these educational reforms were passed into law, he and his wife are frequent volunteers in schools and have seen firsthand the impact of these reforms on classrooms, teachers and kids.

Barresi Issues a TLE Warning

Oklahoma’s Teacher/Leader Effectiveness System (TLE) is highly flawed. Ask anybody in a school, and you will hear that. Sure, some like the qualitative part that will eventually comprise 50 percent of a teacher’s overall rating. They say it has improved the language of the evaluation process. Unfortunately, it has also increased the extent to which teachers and principals are over-burdened with paperwork. It is a thorough process, but it is also terribly cumbersome.

This disruption to the status quo, however, has nothing on the impending disaster of the other 50 percent. When the quantitative component of TLE becomes reality, the bottomed-out morale of teachers will find a new low. Anybody who teaches or supervises teachers understands this. The future former state superintendent does not. Last night, Janet Barresi posted one of her final missives (at least in an official capacity), this time defending the TLE and refuting some of the concerns we’ve voiced for years. As usual, though, she misrepresented many, many things. I will attach a few excerpts and then respond.

When properly implemented by districts, TLE is not an excuse to fire teachers. We cannot and will not fire our way to a better education. TLE allows for focused professional development. It is a carefully designed system that helps good teachers become great, and struggling teachers become good.

Actually, this sounds like the justification of someone who hasn’t read the statutory language associated with the process. I understand – the relevant section doesn’t appear until pages 13-14. By then, most politicians have stopped reading to learn and commenced handing the document off to the underlings with instructions to brief me at a later time. Here’s the short version. Both career and probationary teachers who receive a less-than-effective TLE score for consecutive years can lose their jobs. Even if the principal observes good instruction happening in the classroom, an algorithm can override human judgment. Also, as I discussed Sunday, teachers who have the opportunity to make their own assessments (pre- and post-tests) will have a huge advantage over their counterparts. Still, Barresi warns us against the perils of abandoning evaluation by test score.

Some critics contend that TLE gives too much weight to student performance on assessments, but I believe the system we have designed strikes a good balance. It is important to recognize that student data is valuable. How can school leaders make informed decisions without indicators and data to guide them? How can parents feel assured they have an impartial measure of their school’s success if they only hear qualitative observations? Removing student data from TLE would threaten Oklahoma’s waiver from disastrous No Child Left Behind regulations, but even worse, it would usher in an accountability system that lacks measurable accountability itself.

Remember, Barresi and her ilk share the belief that anything you don’t measure doesn’t matter. As for me, I count two negatives in the previous sentence. It matters.

Seriously, though, Barresi still believes school leaders need her help to make informed decisions. We do use data, even if she won’t give us credit for it. As for assuring parents, I guess that’s what disembodied algorithms developed by out-of-state non-profits that have taken millions from our state are for. I’ve seen too many examples from this year’s VAM data that show great teachers with low scores. Even in cases where every student passes the state tests and most are advanced, the teachers are being labeled ineffective. Explain that to parents. Furthermore, we’ve lost the waiver once. If we lose it again, we’ll cobble something together and get it back. I’ve seen us do it.

Our work in school turn around has shown that as the hard work moves forward to improve instructional processes and practices, change the culture of the school and initiate the use of data as an integral component of improving instruction, that TLE scores also improve.

Rob Miller effectively took down this talking point recently. The SDE thinks they’ve discovered how to turn schools around. As Rob showed, they’ve also effectively discovered how not to turn schools around. Essentially, in any ranking system, there will be winners and losers. The system can’t help it; it was born that way. This is true for schools, for teachers, and for kids. Some will score high, and some will score low. Left to their own devices, some will rise, and some won’t. Placed under intensive scrutiny from the state, some will rise, and some will fall. It is a natural by-product of the system; often, what appear to be gains (or losses) are merely statistical corrections. No state agency deserves credit for schools that regress to the mean.

I don’t believe that the “sole purpose” of TLE is to fire people. I know that it will happen, though. Good teachers will lose their jobs because of bad data. Whether or not the intent of TLE is to shame teachers and schools, this will be the outcome. No amount of spin from Janet Barresi, Arne Duncan, Jeb Bush, or anyone else will change that. As Superintendent-elect Hofmeister has traveled the state, she has heard some version of these concerns again and again. Our legislators have heard them too, and most seem to understand that something has to give. In policy terms, it probably will come down to a choice between delaying implementation of the quantitative score or tossing the entire TLE system.

The timing of this letter is curious. It makes me wonder if Barresi has a last-minute surprise for us at tomorrow’s State Board of Education meeting. This will be her last one (unless they do not choose a vendor for spring testing, in which case there may be a special SBE meeting early next month), and the agenda for it should post this afternoon. We can only wonder right now if this is a clue to what we’re going to see on it.

March Review / April Preview

I pretty much took Easter weekend off from blogging but made a quick appearance on Twitter tonight while working on my end-of-month post. Once again, Representative Jason Nelson (R-OKC) has taken the position that he can’t support more funding for schools until he has a number of how much is enough. Several Oklahoma educators have pointed out to Nelson that with rising enrollment, reduced funding, and a whole slate of reforms, we’re nowhere near enough. I think he gets that, but I honestly don’t have an answer. At no point have we done an honest analysis of the “true cost” of public education. If we did, we’d have to admit that it’s not the same in Tulsa as it is in Poteau or Guymon. So while he has a point, it’s a pathetic excuse for the inertia of the growing school funding crisis.

With that said, here’s a look back at the top five blog posts from March:

1. CCOSA Call to Action – Parent Trigger – Oklahoma educators continue to have concerns about this legislation. It will pit parents against schools and even against each other. More importantly, there doesn’t seem to be any discernible push from a recognized parent group for the law. In short, it is a classic example of a solution in search of a problem. That won’t stop our legislature though, unless parents from around the state remind them that they are here to serve Oklahomans, rather than ALEC and FEE.
2. Senate Bill 1001 – Parent Trigger – In case I didn’t mention it clearly, I’m not a fan of the Parent Trigger bill. It seems blog readers aren’t either. The top two posts this month were ones that were critical of the proposal. In this one, I also linked to an article about a Florida school where the parents wanted to fire the charter school company, which then was taking them to court. Seriouly. This is where we’re headed if we don’t stop following Jeb Bush.
3. Two Year Delay for TLE? – Readers were also enthusiastic this month about the possibility of the SDE getting a two year reprieve from trying to figure out how to calculate value. I hope they get it. And in the meantime, I hope we can have a discussion about how to calculate votes. There has to be a better option out there.
4. And Then There Was Roster Verification – We added to our vocabulary this month, as the state announced a plan to pilot a program to calculate the percentage of student time spent with each teacher, pretty much from day one of school. This way, we can hold Pre-K teachers accountable for the number of students who graduate 13 years later. I am told, however, that roster verification will not calculate how many days each student came to school hungry or traumatized from some event that caused an amygdala hijacking. As always with things that are tied to TLE, we should remember that the SDE staff over this program have never had to evaluate a single teacher or principal.
5. Teachers Respond to TLE Commission and Senator Mike Mazzei’s Response to a Patron (tie) – A group of Jenks teachers wrote a spirited response to the TLE Commission early in March and got a cursory response. I am unaware if there has been any follow-up, however. There was a response given by a state senator to a patron about SB 1001, however. In it, Mazzei (R-Tulsa) made it clear that this law would never apply to suburban districts anyway. It’s really targeting the urban schools. And no, that’s not at all patronizing.

April is going to be an important month to be active. Once again, everything from funding, to ALEC/FEE based policy decisions is on the table. There will be another push for school consolidation. One thing we know is that there are hundreds of superintendents, thousands of principals, tens of thousands of teachers, and hundreds of thousands of parents in Oklahoma. If we can be a little more well-informed and a lot more vocal, maybe this will be the month it all turns around.

Two Year Delay for TLE?

Yesterday, the SDE sent out a media release stating that they would be requesting a two year delay for full implementation of the Teacher/Leader Effectiveness system. The content of the release was linked to the SDE website, but is now down, probably due to a technical problem. The release reads as follows:

OKLAHOMA CITY (March 19, 2013) – State Superintendent Janet Barresi announced today that she will ask the authors of Senate Bill 426, Sen. John Ford and Rep. Earl Sears, to consider a two-year delay for full implementation of the Teacher and Leader Effectiveness reform that was originally enacted in 2010 in Senate Bill 2033.

In making the announcement, Superintendent Barresi stated, “After listening to input from teachers and superintendents across the state as well as teachers serving on working groups for the TLE Commission, I have concluded that this extra time is necessary to assure the entire TLE system is implemented with fidelity and to the high standards we expect of such a critical reform.”

“Nothing is more important than assuring that each child in our state has the opportunity to be taught by an effective teacher and school principal. We will continue to work with the TLE Commission and the State Board of Education to build a model program and quality technology infrastructure to support the program. I appreciate Gov. Fallin’s support in this decision and our work,” Barresi added.

Governor Mary Fallin said, “Studies show that the most important driver of student success in the classroom is high quality teachers. That’s why it’s so important that we get these reforms right. Giving Oklahoma schools adequate time to properly prepare for TLE implementation is in the best interest of everyone. I strongly support TLE and look forward to full implementation so we can utilize performance pay options and other compensation models tied to the system.”

Superintendent Barresi suggested the timeline for implementation of the qualitative or observational component of the system is currently being piloted this year and will be fully implemented in districts for the 2013-14 school year. If SB 426 passes, the Other Academic Measures portion of the quantitative component will be piloted next school year and implemented in 2014-15. The 35 percent accountability measures of the quantitative component will be implemented in the 2015-16 school year, making TLE complete.

Per state statute, the Oklahoma State Department of Education is working in conjunction with the TLE Commission and working groups of educators throughout the state to develop a robust professional growth tool known as the Teacher and Leader Effectiveness evaluation system. When fully implemented and utilized properly, TLE will identify the direct cause-and-effect relationship between teaching practices and student achievement using both qualitative and quantitative measures.

A teacher’s evaluation will be based on 50 percent of qualitative measures such as classroom observations and 50 percent quantitative measures. Of the quantitative measures, 35 percent will be based on student test scores for tested grades and subjects and the remaining 15 percent on Other Academic Measures as defined by the TLE Commission and educators. How teachers are evaluated in non-tested grades and subjects for the quantitative portion is still being discussed.

SDE staff have been very direct for months now that they would be asking for this delay, which will be appreciated by most teachers and administrators. If they need more time to work out the details, that’s fine. The cynic in me believes that something else is happening here, however.

Two things are missing at this time. First, as we heard back in January, the SDE is getting input from SAS to develop a model for creating a VAM model. (Yes, that was three acronyms in a single sentence.) Without a mathematical equation that includes factors both within and outside of a school’s control, there won’t be a value-added measure. This would make the recommendations of the various working groups meaningless at this time. The other piece in development is a more refined student data system – one that can effectively track where students were and for how long and which teachers impacted their learning and for what percentage of the time.

That brings us to roster verification, which I wrote about last week. This is a new experiment that the SDE wants to run before full implementation of TLE. If we start calculating the quantitative portion of the evaluation without these pieces, it will be harder to add them in.

A delay will ensure that VAM and roster verification will be a piece of the enacted system. It will also guarantee that we will have conversations like those taking place in Florida right now – ones in which successful teachers get low ratings because of the students they do not teach. Call me ungrateful, but rather than waiting to get it right, we should instead acknowledge that the entire concept is fatally flawed.

*****

Update: the SDE press release is back.

Teachers Respond to the TLE Commission

A group of Jenks elementary school teachers sent a response to the SDE this afternoon, expressing their concerns over the recommendations for quantitative measures for evaluating teachers that I wrote about this morning. This group includes art, music, and physical education teachers. I’ll let their words speak for them, first with an excerpt, and then with a link to the full letter:

We the undersigned and highly qualified specialists at Jenks East Elementary School urge our legislators to seriously explore the quantitative component of TLE before 2013/2014 implementation. Below you will find our “real time” experiences and “real voices” speaking facts which must be considered before Oklahoma implements a “one-fits-all” approach for evaluating educators and determining their compensation.

This insightful four-page letter didn’t just go to the SDE, however; it went to the 250 members of the TLE working group. And they told two friends. And they told two friends. And so on. And so on. And then it found its way to your friendly neighborhood blogger. It’s the gift that keeps on giving.

Later in the afternoon, the Jenks teachers (and the entire working group) received a response from the SDE:

Ms. Riggin and Special Area Teachers (ART/MUSIC/PHYSICAL EDUCATION) of Jenks East Elementary,

Thank you for your careful thought, consideration, and time in preparing the document you provided for us. We appreciate your input and respect your perspectives to this challenging work. The working groups’ final recommendations were presented to the TLE Commission for initial consideration on Tuesday, March 12, 2013. I will share your additional input with Superintendent Barresi, who is also the chair of the TLE Commission. If she has any questions or follow-up requests, I will get back in touch with you.

I’m glad that our state has some teachers not taking this quietly. Read the whole letter. It’s worth your time.

And Then There Was Roster Verification

I received an email last night with a 30-page document attached showing the different recommendations to the Teacher and Leader Effectiveness (TLE) Commission by the various working groups that developed them. In all, the Value Added/Student Growth Measures for Teachers of Non-Tested Grades/Subjects and Teachers Without a Teaching Assignment. In all, this document contains suggestions for how to quantify the effectiveness of 18 different classifications of certified teachers. These include:

CareerTechnology	Counselors
Early Childhood/PreK	Elementary (1-6) Non Tested
English Language Learners	Fine Arts
Gifted/Talented	Instructional Coaches
Library Media Specialists	Nurses
Physical Education	Reading Specialists/Response to Intervention
School Psychologists	Secondary: Non-Tested Subjects
Special Education	Speech Language Pathologists
Technology	World Languages

I am repulsed by the idea that we have to come up with some sort of a quantitative measure to evaluate some of these groups of teachers (and nurses, really!?!), but I decided to play along and read through the sections. What I found were some drastically disparate ways to calculate teacher effectiveness. All include some level of new training for both the teachers and the principals who would evaluate them. And most ask for more time to come up with a workable plan.

The different recommendations include some similar language that we rarely use when talking about education reform initiatives. We are going to have to learn these terms in the same ways that our students learn the academic vocabulary. I have already discussed Value-Added Measures (VAM) on this blog; I am not a fan. I do not believe that an agency incapable of developing a statistically-sound report card can develop VAM in a way that is fair to teachers. I’m not convinced that it is achievable in the first place.

Several of the proposals also call for some kind of a matrix, portfolio, or rubric to assess teachers. Principals would have to become familiar with all of these instruments. They also call at various times for different weights on the quantitative pieces of the evaluation. Imagine keeping track of all of that!

The newest term for educators, however, is Roster Verification. The only group that mentioned this process in the report to the Commission was Special Education. In an email to superintendents and principals this week, here’s how the SDE described Roster Verification:

Roster Verification – Voluntary, yet Valuable! The Oklahoma State Department of Education is offering Roster Verification as a service to school districts this spring. The OSDE will be completing value added analysis for all teachers of TESTED grades and subjects after testing occurs this spring. Value added analysis will be used for INFORMATIONAL purposes so that teachers and administrators have the opportunity to learn about the process and can use data to inform instructional practices during the 2013-2014 school year. This is a NO STAKES process meaning NONE of the value added calculations will be used in evaluations. Because there are so many different teaching scenarios that occur throughout the year, Roster Verification allows teachers to account for who they taught, for which months during the year, and for what percentage of the instructional time. For example, when I taught 5th grade, we were departmentalized. I was responsible for MANY students’ instruction in mathematics and science, but my team member was responsible for their instruction in reading and social studies. Without completing Roster Verification, my value added analysis would be based on my HOMEROOM roster (unless someone uploaded that information differently into the Wave.) As a teacher, I would want to be held accountable for the growth of the students I instructed in math and science, but I would want my partner to be responsible for their growth in reading and social studies. Roster Verification gives teachers the ability to account for such scenarios, therefore value added analysis reports are much more accurate for teachers who were able to complete the Roster Verification process.

The SDE provides even more detailed information on this flyer, which includes training dates, a shout out to the Gates foundation for funding, and a picture that would lead you to believe this is about children. One line even promises that Roster Verification will provide “much more accurate value added reports which will be extremely useful as a professional growth tool.”

This is not remotely about professional growth. This is about continuing down the path of assigning blame, and trying to find a mathematical formula for doing so. In ten years, we will be able to look at the students of two second-grade teachers and see which ones are better prepared for college. We will be able to assign partial credit/blame for the success/lack-thereof to all the teachers those students have ever had. Over time, we’ll have all kinds of data pointing back to that second grade classroom.

Think back to that bizarre oak tree analogy. The disembodied voice in the video tells us that countless factors go into two farmers raising their oak trees. It also tells us that some of those factors are out of the control of the farmers. (By the way, I still don’t know any oak tree farmers).  Removing all of those factors, the argument is that we can tell which farmer is adding more value. Extending the analogy, we can remove the factors out of the control of teachers and look at results, and ascertain value-added that way as well. This too makes me uncomfortable.

The factors out of a teacher’s control are too many to count. We will be assigning value-added sometimes ten or twelve years after the fact. Would you want part of your evaluation during your eleventh year in the profession to be based on something you did your first year?

Many districts have chosen not to participate in Roster Verification at this time. Others, for the sake of curiosity, are joining in the trial run. I understand both positions. While I can’t think of one school administrator who wants to see this happen, many want to see what the process looks like since it will happen eventually anyway.

The recommendations to the TLE Commission are non-binding. Commission members can act to accept, revise, or reject the proposals at a later date. Meanwhile, the SDE is wisely pushing for more time to implement the quantitative piece of the evaluation system. While they would be even wiser to scrap it altogether, that won’t happen. Too much money –taxpayer and corporate money – is invested at this point. The agency is philosophically entrenched in this process.

February Review / March Preview

I hoped at the end of last month’s (p)review post that February would be exciting. I wasn’t disappointed. With over 6,100 page views, it was the second-best month since this blog started last April. The A-F Report Card issues flared up again – almost to the point that I didn’t get to write about anything else. Hopefully that will change in March. There are issues with the quantitative portion of TLE, incremental steps being taken towards school vouchers (with two more private schools approved to accept LNH scholarships yesterday), and of course budgeting concerns that have just been made worse by Sequestration. Meanwhile, the parent trigger, measures to arm teachers, and instant transfer policies are all moving forward.

Here’s a look back at the top five blog posts for February:

1. I’m Not Making This Up (But That Would Be Allowable, I Suppose) – Surprisingly, with all the writing I’ve done on the different A-F events, the top post for the month was one about science. Legislation is moving forward to allow students to opt out of any science content that they find objectionable. That’s tough to swallow. I don’t want public schools to be a place where children are told their beliefs lack significance. But it also shouldn’t be a place where they are allowed to bury their heads in the sand and ignore science. Faith and facts are not mutually exclusive. The fact that this post resonated so strongly with readers buoys my confidence in people.
2. Misunderstanding? Hardly! – And now we’re back to form with a post about the A-F Report Cards. Superintendent Barresi told a group of parents that the OU/OSU researchers had changed their mind and now supported the system. They hadn’t. It was an inexplicable statement that for which she hasn’t been held accountable.
3. The Silence is Broken – It took the Oklahoman almost three weeks to comment on the Washington Post article discussing the ties between Jeb Bush, his Foundation For Excellence, and the SDE. When they did, they glossed over all the important links showing how corporate influences are the real forces behind state policy. As usual, the Tulsa World was much more thorough. The frustrating thing about this is that the Oklahoman has good reporters capable of the work.
4. These are not the Rules You’re Looking For – Last week, after a legislative committee voted to throw out the existing rules for A-F Report Cards, the SDE quickly issued rules that change precious little. They seem to have been hastily constructed and create more problems than they solve. I have a hard time believing that these are the rules that will be in place for next year.
5. Get Serious, People – Much of February saw the legislature wasting time on issues that have nothing to do with helping kids or helping schools. They want to keep students from being bullied, but they don’t want to protect everybody. They also want to make sure a teacher can paddle children, even if the principal or school board do not allow it. What could be more important than that?

My sincere hope for March is that we will see the conversation turn more serious and constrictive. In lieu of that, I’ll surely be here, filling this space.

Tie an Incomprehensible Ribbon…

Does anyone besides me have an older relative who just says crazy, sometimes inappropriate things that make you shake your head? You always want to say, no, it’s not ok to say that or you can’t talk like that anymore. You love them because they’re family but you still just flinch when they say those things.

That’s how I feel when the SDE talks about things like VAM, and poverty, and … well … children. I just want to say, it’s not ok to say that. The difference is of course that I don’t find them all that endearing, like an aunt or a grandfather. Come to think of it, that was a pretty bad analogy.

Speaking of bad analogies (and segues), have you heard the one about how teaching is like gardeners raising oak trees? This very idea is central to the SDE’s argument for VAM in Oklahoma. I’ll explain more in a few paragraphs.

By the way, if you’re humming a Tony Orlando song right now, you’re welcome! It’s just because I want you thinking about anything related to Florida – the promised land of education reform.

I decided to watch yesterday’s videoconference for prospective participants of the TLE working group. Watch it if you’d like, but if you’d rather not, I completely understand. I kept a running log as I was watching.

• 4:28 The presenters tell us that if the overview is old news for us, we should feel free to check our email.
• 4:48 The presenters explain the widget effect. Teachers think they’re irreplaceable. We’re told they’re not.
• 5:37 We hear that teachers don’t want to be told that they’re excellent.
• 8:00 We hear about research from 1996 that in no way is based on the current methods being proposed for judging the extent to which teachers add value.
• 9:00 We see a quote from Lee Shulman, that reads in part, “classroom teaching…is perhaps the most complex, most challenging, and most demanding, subtle, nuanced, and frightening activity that our species has ever invented.” (By the way, if this is the justification for VAM, I’m completely lost. Why would you take something subtle and nuanced and try to find the most standardized way to evaluate it? You’re admitting that teaching is an art and then applying spreadsheet logic to it.)
• 11:00 We see that 37 states have or are revising teacher evaluation systems and a list of states that our SDE considers leaders in this area. Predictably, Florida and Indiana are at the top.
• 14:04 We see a breakdown by percentage of the components that will eventually make up teacher evaluation: 50% qualitative, 35% test scores, 15% other quantifiable academic measures.
• 16:12 The presenters explain that the SDE – along with the “education coalition across this state” – is proposing to delay the implementation of the quantitative part of TLE for one year. This would make them go into effect the year we start testing over Common Core. While I appreciate the delay, I don’t think they’ve entirely thought this through.
• 20:27 The presenters explain that most TLE Commission and State Board of Education members are not professional educators by trade. As if we were somehow unaware of this.
• 24:15 The presenters explain the purpose of the working group.
• 25:15 The presenters explain that working group members will be expected to have positive attitudes and that their input will be valued.
• 32:15 The first attempt is made at showing a 10 minute video on how teaching is like raising oak trees.
• 34:52 The presenters receive technical assistance and get the video to work.
  • The analogy uses starting points and ending points to evaluate the performance of each gardener.
  • The environmental factors such as soil condition, average temperature, and total rainfall are considered as contributors to the annual growth.
  • Predicted growth is then adjusted for these conditions to level the playing field for the gardeners.
• 45:20 The video mercifully comes to an end, at which point the presenters begin to explain to us why poverty is not one of the environmental factors that impacts student performance.
  • At this point, I pause the video and Google VARC – the producers of the video. They are a grant-funded project housed at the University of Wisconsin.
  • I read on their website that VARC’s methodology includes controls for “the likelihood that school effects are not random, but may be correlated with school-level variables such as race, ethnicity, and income status.”
• 55:00 The presenters ask whether test scores from other subjects should be used in part to evaluate other teachers. My answer to that is absolutely not.

The rest of the video is fairly uneventful. After viewing it, I still haven’t decided if I want to participate in the working group. I’d rather not lend my name to this effort, but at the same time, whether I join the discussion constructively or not, VAM is happening. Art and music teachers (in schools still fortunate enough to have them) will be evaluated, at least in part, by the test scores of students. Whether this happens by 2013-14 or 2014-15, it’s going to happen. I need to be in a position to keep teachers and administrators informed so they can work to protect themselves.

I don’t like to picture a world in which teachers are going to be jumping in front of each other to claim some credit for the successes of students. I also don’t want to see them throwing each other aside when students fail to make gains. If salaries are relatively close around the state, working conditions are not. The introduction of VAM only amplifies this disparity. The result – as happens with all reforms – is fewer teachers wanting to teach in the schools with the neediest kids.

As usual, we are left with a reform that subtracts value from the profession.