Posts Tagged ‘TLE’

And Then There Was Roster Verification

March 15, 2013 3 comments

I received an email last night with a 30-page document attached showing the different recommendations to the Teacher and Leader Effectiveness (TLE) Commission by the various working groups that developed them. In all, the Value Added/Student Growth Measures for Teachers of Non-Tested Grades/Subjects and Teachers Without a Teaching Assignment. In all, this document contains suggestions for how to quantify the effectiveness of 18 different classifications of certified teachers. These include:

CareerTechnology Counselors
Early Childhood/PreK Elementary (1-6) Non Tested
English Language Learners Fine Arts
Gifted/Talented Instructional Coaches
Library Media Specialists Nurses
Physical Education Reading Specialists/Response to Intervention
School Psychologists Secondary: Non-Tested Subjects
Special Education Speech Language Pathologists
Technology World Languages

I am repulsed by the idea that we have to come up with some sort of a quantitative measure to evaluate some of these groups of teachers (and nurses, really!?!), but I decided to play along and read through the sections. What I found were some drastically disparate ways to calculate teacher effectiveness. All include some level of new training for both the teachers and the principals who would evaluate them. And most ask for more time to come up with a workable plan.

The different recommendations include some similar language that we rarely use when talking about education reform initiatives. We are going to have to learn these terms in the same ways that our students learn the academic vocabulary. I have already discussed Value-Added Measures (VAM) on this blog; I am not a fan. I do not believe that an agency incapable of developing a statistically-sound report card can develop VAM in a way that is fair to teachers. I’m not convinced that it is achievable in the first place.

Several of the proposals also call for some kind of a matrix, portfolio, or rubric to assess teachers. Principals would have to become familiar with all of these instruments. They also call at various times for different weights on the quantitative pieces of the evaluation. Imagine keeping track of all of that!

The newest term for educators, however, is Roster Verification. The only group that mentioned this process in the report to the Commission was Special Education. In an email to superintendents and principals this week, here’s how the SDE described Roster Verification:

Roster Verification – Voluntary, yet Valuable!

The Oklahoma State Department of Education is offering Roster Verification as a service to school districts this spring. The OSDE will be completing value added analysis for all teachers of TESTED grades and subjects after testing occurs this spring. Value added analysis will be used for INFORMATIONAL purposes so that teachers and administrators have the opportunity to learn about the process and can use data to inform instructional practices during the 2013-2014 school year. This is a NO STAKES process meaning NONE of the value added calculations will be used in evaluations.

Because there are so many different teaching scenarios that occur throughout the year, Roster Verification allows teachers to account for who they taught, for which months during the year, and for what percentage of the instructional time. For example, when I taught 5th grade, we were departmentalized. I was responsible for MANY students’ instruction in mathematics and science, but my team member was responsible for their instruction in reading and social studies. Without completing Roster Verification, my value added analysis would be based on my HOMEROOM roster (unless someone uploaded that information differently into the Wave.) As a teacher, I would want to be held accountable for the growth of the students I instructed in math and science, but I would want my partner to be responsible for their growth in reading and social studies. Roster Verification gives teachers the ability to account for such scenarios, therefore value added analysis reports are much more accurate for teachers who were able to complete the Roster Verification process.

The SDE provides even more detailed information on this flyer, which includes training dates, a shout out to the Gates foundation for funding, and a picture that would lead you to believe this is about children. One line even promises that Roster Verification will provide “much more accurate value added reports which will be extremely useful as a professional growth tool.”

This is not remotely about professional growth. This is about continuing down the path of assigning blame, and trying to find a mathematical formula for doing so. In ten years, we will be able to look at the students of two second-grade teachers and see which ones are better prepared for college. We will be able to assign partial credit/blame for the success/lack-thereof to all the teachers those students have ever had. Over time, we’ll have all kinds of data pointing back to that second grade classroom.

Think back to that bizarre oak tree analogy. The disembodied voice in the video tells us that countless factors go into two farmers raising their oak trees. It also tells us that some of those factors are out of the control of the farmers. (By the way, I still don’t know any oak tree farmers).  Removing all of those factors, the argument is that we can tell which farmer is adding more value. Extending the analogy, we can remove the factors out of the control of teachers and look at results, and ascertain value-added that way as well. This too makes me uncomfortable.

The factors out of a teacher’s control are too many to count. We will be assigning value-added sometimes ten or twelve years after the fact. Would you want part of your evaluation during your eleventh year in the profession to be based on something you did your first year?

Many districts have chosen not to participate in Roster Verification at this time. Others, for the sake of curiosity, are joining in the trial run. I understand both positions. While I can’t think of one school administrator who wants to see this happen, many want to see what the process looks like since it will happen eventually anyway.

The recommendations to the TLE Commission are non-binding. Commission members can act to accept, revise, or reject the proposals at a later date. Meanwhile, the SDE is wisely pushing for more time to implement the quantitative piece of the evaluation system. While they would be even wiser to scrap it altogether, that won’t happen. Too much money –taxpayer and corporate money – is invested at this point. The agency is philosophically entrenched in this process.

February Review / March Preview

March 1, 2013 Comments off

I hoped at the end of last month’s (p)review post that February would be exciting. I wasn’t disappointed. With over 6,100 page views, it was the second-best month since this blog started last April. The A-F Report Card issues flared up again – almost to the point that I didn’t get to write about anything else. Hopefully that will change in March. There are issues with the quantitative portion of TLE, incremental steps being taken towards school vouchers (with two more private schools approved to accept LNH scholarships yesterday), and of course budgeting concerns that have just been made worse by Sequestration. Meanwhile, the parent trigger, measures to arm teachers, and instant transfer policies are all moving forward.

Here’s a look back at the top five blog posts for February:

  1. I’m Not Making This Up (But That Would Be Allowable, I Suppose) – Surprisingly, with all the writing I’ve done on the different A-F events, the top post for the month was one about science. Legislation is moving forward to allow students to opt out of any science content that they find objectionable. That’s tough to swallow. I don’t want public schools to be a place where children are told their beliefs lack significance. But it also shouldn’t be a place where they are allowed to bury their heads in the sand and ignore science. Faith and facts are not mutually exclusive. The fact that this post resonated so strongly with readers buoys my confidence in people.
  2. Misunderstanding? Hardly! – And now we’re back to form with a post about the A-F Report Cards. Superintendent Barresi told a group of parents that the OU/OSU researchers had changed their mind and now supported the system. They hadn’t. It was an inexplicable statement that for which she hasn’t been held accountable.
  3. The Silence is Broken – It took the Oklahoman almost three weeks to comment on the Washington Post article discussing the ties between Jeb Bush, his Foundation For Excellence, and the SDE. When they did, they glossed over all the important links showing how corporate influences are the real forces behind state policy. As usual, the Tulsa World was much more thorough. The frustrating thing about this is that the Oklahoman has good reporters capable of the work.
  4. These are not the Rules You’re Looking For – Last week, after a legislative committee voted to throw out the existing rules for A-F Report Cards, the SDE quickly issued rules that change precious little. They seem to have been hastily constructed and create more problems than they solve. I have a hard time believing that these are the rules that will be in place for next year.
  5. Get Serious, People – Much of February saw the legislature wasting time on issues that have nothing to do with helping kids or helping schools. They want to keep students from being bullied, but they don’t want to protect everybody. They also want to make sure a teacher can paddle children, even if the principal or school board do not allow it. What could be more important than that?

My sincere hope for March is that we will see the conversation turn more serious and constrictive. In lieu of that, I’ll surely be here, filling this space.

Tie an Incomprehensible Ribbon…

January 10, 2013 8 comments

Does anyone besides me have an older relative who just says crazy, sometimes inappropriate things that make you shake your head? You always want to say, no, it’s not ok to say that or you can’t talk like that anymore. You love them because they’re family but you still just flinch when they say those things.

That’s how I feel when the SDE talks about things like VAM, and poverty, and … well … children. I just want to say, it’s not ok to say that. The difference is of course that I don’t find them all that endearing, like an aunt or a grandfather. Come to think of it, that was a pretty bad analogy.

Speaking of bad analogies (and segues), have you heard the one about how teaching is like gardeners raising oak trees? This very idea is central to the SDE’s argument for VAM in Oklahoma. I’ll explain more in a few paragraphs.

By the way, if you’re humming a Tony Orlando song right now, you’re welcome! It’s just because I want you thinking about anything related to Florida – the promised land of education reform.

I decided to watch yesterday’s videoconference for prospective participants of the TLE working group. Watch it if you’d like, but if you’d rather not, I completely understand. I kept a running log as I was watching.

  • 4:28 The presenters tell us that if the overview is old news for us, we should feel free to check our email.
  • 4:48 The presenters explain the widget effect. Teachers think they’re irreplaceable. We’re told they’re not.
  • 5:37 We hear that teachers don’t want to be told that they’re excellent.
  • 8:00 We hear about research from 1996 that in no way is based on the current methods being proposed for judging the extent to which teachers add value.
  • 9:00 We see a quote from Lee Shulman, that reads in part, “classroom teaching…is perhaps the most complex, most challenging, and most demanding, subtle, nuanced, and frightening activity that our species has ever invented.” (By the way, if this is the justification for VAM, I’m completely lost. Why would you take something subtle and nuanced and try to find the most standardized way to evaluate it? You’re admitting that teaching is an art and then applying spreadsheet logic to it.)
  • 11:00 We see that 37 states have or are revising teacher evaluation systems and a list of states that our SDE considers leaders in this area. Predictably, Florida and Indiana are at the top.
  • 14:04 We see a breakdown by percentage of the components that will eventually make up teacher evaluation: 50% qualitative, 35% test scores, 15% other quantifiable academic measures.
  • 16:12 The presenters explain that the SDE – along with the “education coalition across this state” – is proposing to delay the implementation of the quantitative part of TLE for one year. This would make them go into effect the year we start testing over Common Core. While I appreciate the delay, I don’t think they’ve entirely thought this through.
  • 20:27 The presenters explain that most TLE Commission and State Board of Education members are not professional educators by trade. As if we were somehow unaware of this.
  • 24:15 The presenters explain the purpose of the working group.
  • 25:15 The presenters explain that working group members will be expected to have positive attitudes and that their input will be valued.
  • 32:15 The first attempt is made at showing a 10 minute video on how teaching is like raising oak trees.
  • 34:52 The presenters receive technical assistance and get the video to work.
    • The analogy uses starting points and ending points to evaluate the performance of each gardener.
    • The environmental factors such as soil condition, average temperature, and total rainfall are considered as contributors to the annual growth.
    • Predicted growth is then adjusted for these conditions to level the playing field for the gardeners.
  • 45:20 The video mercifully comes to an end, at which point the presenters begin to explain to us why poverty is not one of the environmental factors that impacts student performance.
    • At this point, I pause the video and Google VARC – the producers of the video. They are a grant-funded project housed at the University of Wisconsin.
    • I read on their website that VARC’s methodology includes controls for “the likelihood that school effects are not random, but may be correlated with school-level variables such as race, ethnicity, and income status.”
  • 55:00 The presenters ask whether test scores from other subjects should be used in part to evaluate other teachers. My answer to that is absolutely not.

The rest of the video is fairly uneventful. After viewing it, I still haven’t decided if I want to participate in the working group. I’d rather not lend my name to this effort, but at the same time, whether I join the discussion constructively or not, VAM is happening. Art and music teachers (in schools still fortunate enough to have them) will be evaluated, at least in part, by the test scores of students. Whether this happens by 2013-14 or 2014-15, it’s going to happen. I need to be in a position to keep teachers and administrators informed so they can work to protect themselves.

I don’t like to picture a world in which teachers are going to be jumping in front of each other to claim some credit for the successes of students. I also don’t want to see them throwing each other aside when students fail to make gains. If salaries are relatively close around the state, working conditions are not. The introduction of VAM only amplifies this disparity. The result – as happens with all reforms – is fewer teachers wanting to teach in the schools with the neediest kids.

As usual, we are left with a reform that subtracts value from the profession.

Categories: Uncategorized Tags: , ,

They want your help. They really, really do.

January 9, 2013 Comments off

What is value? And how do you add it?

These are the key questions the State Department of Education and its contributors will attempt to answer in working groups over the next six weeks. That is why, this week, the SDE issued a call for all hands on deck. They want anyone and everyone – especially teachers who don’t have a regular classload (such as counselors and media specialists) or teachers in non-graded subjects (the majority of the profession) – to help with creating a methodology for the portion of the Teacher Leader Evaluation system that falls to “other academic measures.” In reformer terms, they are talking about Value Added Measurements.

According to the SDE’s notice for the working group:

The quantitative portion of teacher and leader evaluations under the TLE System, which will comprise 35% of the evaluation, will be calculated based on Value-Added or Student Growth Data. In addressing those teachers and leaders in grades and subjects for which there is no state-mandated testing measure to create a quantitative assessment, the TLE Commission and the Oklahoma State Board of Education have chosen to:

Conduct more research to determine the appropriate measure(s) of student achievement taking into account a combination of multiple measures and including teacher, leader, and specialist input.

A list of options needs to be drafted for presentation to the TLE Commission and the Oklahoma State Board of Education. In addition, consideration must be given as to whether or not percentages for qualitative/quantitative measures should be adjusted for teachers of non-tested grades/subjects and those without a teaching assignment.

Any TLE Commission Member or other interested party may participate in the working group, but it is essential that attendees register so that we have enough space, handouts, facilitators, etc.

The appearance of stakeholder input is important to the image the SDE wants to project. In the past two years, educators have participated on committees to develop the state’s waiver to NCLB, only to find upon arrival that the major decisions have already been made; they have shown up to speak to SDE officials about rules for the A-F Report Cards only to find the room filled with tape recorders; and they have served on the TLE Commission only to have the state superintendent disregard their 12-5 vote for the Tulsa Model because she prefers a different one.

Given this track record, it is easy to understand why professional educators might want to skip this call to serve. That’s not happening, however. No, the SDE claims to be a little surprised by the size of the response. They are trying to manage the size of the crowd that they’re going to have. Sure, they can spin this response to claim that it shows support for the junk science that is VAM. In my mind, it shows that educators are beginning to rise up to take back the profession.

Want to make it interesting? More people should sign up. Tell your friends. Tell your co-workers. Tell the parents in your community.

Send them here to register.

The SDE says it’s important that we tell them how to measure the value of teachers whose instructional content isn’t tested. I say if you need test scores to know whether your art teacher has value, then you have no business supervising anybody.

But I’m just one voice. They need to hear it from the choir.

Categories: Uncategorized Tags: , ,

A little bit of a REAC3H

October 18, 2012 2 comments

A year ago, the SDE launched a new initiative designed to support public schools: REAC3H (sometimes REAC3H; sometimes REAC3H). The acronym stands for Regional Educators Advancing College, Career, and Citizenship Readiness Higher.

The launch of REAC3H came with a summit last fall for more than 100 districts that were chosen to serve in lead roles. Actually, it was two summits – one for rural districts and another for urban/suburban districts. At the end of the day, superintendents in attendance were asked to come forward and sign a commitment letter that they had just received.

At that time, the stated purpose of REAC3H was to provide districts with collaboration opportunities to assist with the transition from PASS to the Common Core State Standards (CCSS). Problems with this were immediate. Some of the districts not selected to be leaders never heard from those that were. Some had robust curriculum departments, while others relied on teacher leaders to lead the implementation in their spare time. What was true during the implementation of PASS over 20 years ago was still true: districts are differently equipped to move through the change process. As such, implementation would not and could not be standardized.

To assist districts with the transition, the SDE began producing a series of toolkits. The first explained CCSS. The second discussed steps for aligning curriculum. The third then focused heavily on the soon-to-be announced Teacher/Leader effectiveness program.

This shift in focus led to schools asking questions during the second REAC3H summit in January. The third summit – held in the spring – was downright unruly. Schools kept asking questions – about all the reform initiatives – and not getting answers. There were some nice breakout sessions put on by schools, but nothing to help with systemic change. It was sort of a mini preview of Vision 2020.

Over the summer, REAC3H took on another meaning as 60 instructional coaches were hired to work with schools around the state. They are being paid this year with federal money that is set to expire, and Superintendent Barresi has included $5 million in funding in next year’s budget request to maintain the program.

Interestingly, the coverage areas for these coaches are not aligned at all with the REAC3H consortium. They operate in pairs, and for the most part, use office space in Career Tech centers around the state. Some serve only one or two districts.  Other pairs serve more than 20. They have been well-received in some places and kept at arm’s length in others.

At first, REAC3H coaches were going to help with every reform initiative. Now they are focusing on K-2 reading. Since many of the coaches were secondary teachers and may not even be certified in English/Language Arts, their impact may be questionable. (Though to be clear – many schools are reporting satisfaction with their REAC3H Coaches at this time.)

Last month, the SDE released the fourth REAC3H toolkit, providing insight to the testing process that will accompany full implementation of CCSS. Maybe I just find this amusing since the SDE can’t even seem to select a testing company. And they’re now set to hold the fourth REAC3H summit on Election Day (at a yet-to-be-determined venue – in Oklahoma City – probably). This one will have a different format again. Starting early in the morning, participants will again choose breakout sessions – few of which are related to the topic of the fourth toolkit. Then they will spend an hour at the end of the day in a keynote session with David Coleman, who is the head of College Board. The meeting has no built-in time for interaction with SDE staff or for collaboration.

Between the networks, toolkits, conferences, and coaches (and time and resources spent developing and supporting all of them), there are moments where people from disparate groups actually arrive at the same place at the same time. Unfortunately, they are quite rare.

Categories: Uncategorized Tags: , , ,

Four Quick Thoughts

Yesterday’s State Board of Education meeting had little public drama. It was long, and they did spend a lot of time in closed session. I was able to keep up with the meeting from Oklahoma Watchdog via Twitter. After digesting notes from various sources, I have four quick thoughts.

1. At least they didn’t publish more students’ names. The agenda listed students by case number as should have happened all along. Even though the SDE continues defending the previous practice, doing things differently now must mean something. And I wonder if this has anything to do with the change in legal counsel for the department.

2. I’d hate to be a student whose appeal came up one vote short, knowing that one board member was absent. There’s no way of knowing if that would’ve turned the tide, but you have to wonder. With 72 appeals left to be heard, we can only hope that this is as important to board members as it is to the students.

3. For some reason, the SDE decided to withhold the names of the five companies bidding on state testing. They recommended the contract be awarded to “Company A.” The documentation provided lists the prices of the contract, but not the names of the vendors. Note: it is permissible to publish the names of multi-national corporations to which you are awarding multi-million dollar contracts.

4. Bids for training for TLE came in at roughly $4.6 million. The SDE has a budget of $1.5 million. They say they negotiated the cost down to $1.7 million, but since they don’t have the money to fill that gap, they’re releasing funds to schools to use on training as they see fit. My problem is this: will the vendors make training available to schools at the $4.6 million rate or the $1.7 million rate? That will be worth watching.

Categories: Uncategorized Tags: , ,

June 28 State Board of Education Meeting

June 27, 2012 Comments off

The agenda for the June 28 State Board of Education meeting is online, and it includes more student graduation appeals. I notice that this time, the agenda lists each student with a numerical identifier – a case number if you will. Three weeks ago, I was one of a number of people who suggested that would be more appropriate than either listing student names or publishing their academic records. Meanwhile, the SDE insisted that what the had done was perfectly fine.

For whatever reason, they’ve now changed their tune. With students fighting for their futures, no one should complicate the important board discussions with FERPA violations. (Incidentally, signing a waiver to have the state board members view your records does not mean the SDE gets to publish them.)

This board meeting also happens to come on the heels that the SDE has failed to procure affordable training for school districts to implement new teacher and principal evaluations. That will be discussed late in the meeting.

I’ve heard from a number of people who plan to speak during the public comment period. This could get interesting.

Categories: Uncategorized Tags: , ,
%d bloggers like this: