Archive

Archive for July, 2013

Editorial Shows Facts Not Taken Seriously

July 31, 2013 Comments off

How dare they? How dare the Education Establishment pick on friend of the Oklahoman Janet Barresi, what with her words and actions and such? It’s just not right! As yesterday’s editorial shows, they’re not going to just lie back and take that:

School testing settlement shows issue was taken seriously

The Oklahoman Editorial • Published: July 30, 2013

STATE schools Superintendent Janet Barresi and Department of Education staff have negotiated a $1.2 million settlement with CTB/McGraw-Hill in response to that vendor’s failures, which disrupted standardized testing at schools this spring.

The settlement is three times the amount allowed under the contract’s provisions. It includes a $376,205 cash payment and $860,000 in donated services.

Indiana, which experienced similar problems with CTB, is reportedly seeking around $7.68 per affected student. Oklahoma’s cash settlement alone equals $41.34 per affected student. When donated services are included, Oklahoma’s settlement averages nearly $136 apiece.

Oklahoma clearly got its pound of flesh. The settlement shows Barresi and agency officials took this issue seriously. Now if only Barresi’s critics would do the same.

Throughout the process, these critics appeared outraged not that CTB’s server crash disrupted testing, but that the state insists on measuring student performance at all. Their comments seemed designed to derail education measurement, not ensure its accuracy. For example, when Barresi recently noted that CTB was clearly to blame for testing problems on the two days the company’s servers crashed, but said problems on other days were often related to technological challenges at the district level, some superintendents howled.

The Oklahoma State School Boards Association said Barresi was blaming schools for CTB’s failures. That was a dishonest attack clearly refuted by Barresi’s numerous public comments criticizing CTB. But that didn’t stop OSSBA. Instead, many of the same administrators who insist Oklahoma schools are being financially “starved” suddenly acted as though every school is a Mecca of cutting-edge, 21st-century technology. Well, which is it?

In reality, an Oklahoma Department of Education survey found only 33 percent of state school sites are technologically prepared for online testing. Some schools’ bandwidth is much closer to dial-up speed than not. In response, Barresi announced future assessment tests will include a pencil-and-paper option to prevent technological challenges from disrupting the process.

Oh, where to start…

Barresi herself said after the testing collapse that she herself had nothing to do with the contract. As I stated at the time, that falls somewhere between malfeasance and nonfeasance. Her job, as State Superintendent of Public Instruction is keep the ship on the right track. By avoiding responsibility for one of the most highly visible and controversial things the SDE does, she shirked her duty.

And while Barresi was critical of CTB, she did also make the comments blaming schools. The editorial staff of her campaign the Oklahoman may want to write off school districts’ concerns with a false comparison (Mecca – really?), but the fact remains that students were dumped from their tests because of failures on CTB’s end.

The fact that this made it to an editorial shows that the outrage has traction. The Oklahoman can try to defend the settlement all they want. The punishment to CTB is minimal. The damage to the integrity of the test results (and the SDE) is huge.

Advertisements

July Review/August Preview

July 30, 2013 Comments off

Best Month Ever!

Typically, when I blog about my blog, I don’t get a lot of page views. Still, I like to take the time to recap a busy month and preview the next one when I can. And this month I get to brag a little bit. Okeducationtruths is a small operation with a good local following (and a few readers from outside the state). Prior to this month, the best month for page views was April with just over 11,000. I’m on track for 17,000 in July, thanks in no small part to the efforts below. Keep in mind, if the education policy landscape in Oklahoma were better, this blog wouldn’t even exist.

Here are the top five posts from July:

  1. A well-written letter – The most viewed post all-time on this blog was actually written by someone else. Broken Arrow Public Schools superintendent Dr. Jarod Mendenhall. His letter captures the questions that we all have as Oklahoma educators about the failures of the testing cycle and the SDE response to it.
  2. Pulling Out of PARCC – Before we learned that we wanted to keep our toes in the pond, we learned that staying in the testing consortium in which we remain a governing state would be like “drinking a milkshake through a cocktail straw.” And that metaphors are strange little rhetorical devices.
  3. Of Standards and Rules – How would you, if you were Janet Barresi, deal with the Republican backlash over the Common Core while continuing to appease your Chiefs for Change overlords? You would continuing implementing the standards but rename them something else – all without legislative approval.
  4. We were on a break! – The SDE did have to clarify that while Oklahoma will not be testing with the consortium, we are remaining with PARCC. And since, as of this writing, Oklahoma is still listed on the PARCC website as a governing state, I guess that’s true. What’s not clear is why.
  5. More Responses to Barresi’s Response to the Testing Debacle – Two groups representing a large number of districts and administrators took the state superintendent to task for blaming the breakdowns in testing on school districts. And rightly so.

Going back to number three, I’ll try to have a post written in a day or two discussing the recently adopted arts standards and the upcoming cycle for science adoption. I also have been working on a glossary of eduspeak to post (and add to continuously). Finally, I’ve been thinking of posting a page on the blog linking to all candidates for state superintendent with the stipulation that I won’t be making any endorsements. Hopefully I’ll have that completed in August too.

Indiana: A Case Study in Taking Action

Since CTB/McGraw-Hill testing failed in April in both Oklahoma and Indiana, the responses have been very different. Oklahoma’s state superintendent vacillated blame between the testing company and the school districts it failed. Indiana unrelentingly blamed CTB. Oklahoma issued a penalty to the testing company that was really a sweetheart deal bringing them more revenue in the long run. Indiana continues to pursue a steep penalty that actually hurts CTB.

Oklahoma’s deal with CTB allows them to pay for their own study on the impact of the testing disruption. Indiana has already completed theirs. Indiana State Superintendent Glenda Ritz summarized the results:

“I have spent the last several months talking with Hoosiers about the impact these interruptions had in the classroom.  Although Dr. Hill’s report found that the statewide average score was not affected by the interruptions, there is no doubt that thousands of Hoosier students were affected.  As Dr. Hill stated in his report, ‘We cannot know definitively how students would have scored this spring if the interruptions had not happened.’ Because of this, I have given local schools the flexibility they need to minimize the effect these tests have on various matters, such as teacher evaluation and compensation.  I have also instructed CTB McGraw-Hill to conduct enhanced stress and load testing to ensure that their servers are fully prepared for next year’s test and ensure that this never happens again.”

Flexibility? Consideration of the students and teachers? That’s revolutionary talk. My guess is that the CTB report of the Oklahoma disruption will insist that the statewide impact lacks statistical significance, although the impact on individuals will be episodically concerning. With these findings, I expect the Oklahoma SDE to provide no such quarter to students or teachers.

Need an EOI to graduate? Sorry. You should have passed the other ones. Need that value added point in a couple of years (based on a baseline established in 2013) to ensure that you get to keep teaching? Sorry, the disruption wasn’t that big of a deal.

Going back a few months earlier, Indiana voters shocked Jeb Bush by defeating Tony Bennett, one of his Chiefs for Change cronies, in 2012. Do you think there’s a chance we’ll follow Indiana down that road?

*****

(By the way, if you haven’t read about the latest Tony Bennett scandal, you need to. This is one of Barresi’s closest reformer friends.)

If you’d like read the entire 20 page report on Indiana’s testing disruption here.

About the Testing Settlement

The State Department of Education announced Thursday that it had reached a settlement with CTB/McGraw-Hill for more than $1.2 million in damages. Below is the SDE press release:

State Department of Education Settles with CTB/McGraw-Hill for Over $1.2 Million

OKLAHOMA CITY (July 25, 2013) – State Superintendent of Public Instruction Janet Barresi announced today that the Oklahoma State Department of Education, in conjunction with the Office of Management and Enterprise Services, has reached a settlement agreement with CTB/McGraw-Hill for more than $1.2 million in damages. The settlement covers damages suffered by students and teachers during the testing disruptions and server outages experienced on April 29 and 30 during the spring testing window.

“I am very pleased with this settlement. I was outraged over the disruptions during the two day period.  I announced that we would seek damages to the full extent of the contract and we took an aggressive stance. The settlement agreement amounts to three times the value of damages defined in the contract,” said Superintendent Barresi.

Superintendent Barresi pointed out that the settlement agreement directly benefits teachers and supports classroom instruction for increased student achievement through several in-kind components. “Our teachers and students suffered the most during the testing disruptions and I wanted them to benefit the most. We’ve accomplished that. During negotiations, we were mindful of the suffering of the children and the stress this placed on teachers, testing coordinators and parents.”

The settlement agreement includes:

  • A cash settlement ($367,205)
  • Professional development for teachers to help them become more acquainted with the type of items that can be expected on new English language arts and math assessments and how to adjust instruction so students will be successful on these tests. ($13,000)
  • Formative tests for teachers that can be given on a voluntary basis twice a year to measure student learning and drive instruction for the benefit of increased student achievement in the second grade. ($678,400)
  • Formative tests for teachers that can be given on a voluntary basis twice a year to measure student learning and drive instruction for the benefit of increased student achievement for grades 3 through 11. ($6,600)
  • The commissioning of an independent study to evaluate the impact of the disruptions on student test scores. HUMRRO, Inc. has expertise in the area of analyzing testing disruptions. They will provide an independent opinion that is expected to be delivered in late August. ($48,000)
  • Prior to testing, CTB will conduct a technology readiness assessment of each Oklahoma School District to: ($125,000)
    -Capture specifications regarding bandwidth, number of workstations, server
    configuration, etc. at each school site
    -Identify a technology contact at each school district
    -Perform online stress tests at every site
    -Conduct training and deploy implementation services at all sites
    -Establish a technology forum to deliver regular communications to districts

The entire cash settlement amount will be disbursed to schools in order to help compensate for extra personnel costs incurred due to assessment issues this past spring. The State Department of Education will communicate with district superintendents as the HUMRRO report is issues, cash is distributed and in-kind components are put into place.

The Oklahoma Attorney General’s office has expressed interest in reviewing the settlement prior to OMES finalizing paperwork.

While I’ve been tardy in adding my thoughts on the settlement, others have been all over it.

From the OEA:

“About $678,000 of the “in-kind” services are not needed, nor required at this time. Second grade is not a testing grade in Oklahoma, and we should keep it that way. Our students are already over-tested and filled with anxiety because of high-stakes tests.  Our second-graders should not be put under that amount of stress.” said Oklahoma Education President Linda Hampton.

From Broken Arrow Superintendent Jared Mendenhall:

Mendenhall Settlement Tweet - Copy

And a great article from the Tulsa World with several good quotes:

“When you consider that we have over 500 school districts in the state, that equals about $700 if divided evenly,” said Janet Dunlop, assistant superintendent at Broken Arrow Public Schools. “What effect will that truly have on our overtime costs, not to mention time lost to instruction due to testing delays?”
“If ‘affected’ simply means students whose tests were invalidated and were forced to retest on another day, then yes, 9,100 may be correct,” Dunlop said. “However, if we count all the students whose tests were delayed by significant wait times between questions due to the overload of the system, or students who were bumped from the system but who were able to log back on after a wait of sometimes up to an hour, then the figure of 9,100 is significantly lower than reality.”
Added Dunlop, “Why would we want more of a product when we don’t have confidence that the testing system will even operate?”

On one hand, it was just nice to see Barresi include herself in the process. And she did acknowledge that the malfunction was CTB’s fault. If you’ll remember, first it was, then it wasn’t, and now it is again. Aside from that, each of the six bullets listed in the settlement is worthy of few sentences.

  • The cash settlement is paltry. It offsets nothing. The extra time for school employees and the disruption to students is worth a hell of a lot more than 30 percent of the total cost to CTB. Also, this is probably the only area where we are seeing the true cost to the testing company.
  • Professional development for teachers will likely be provided in the form of webinars. Do you know what people do when they’re on a webinar? They check their email and catch up on Words With Friends. They are not engaged learners. Good professional development comes with opportunities for collaboration and is ongoing over a significant period of time. For one percent of the total settlement, any professional development provided will be completely meaningless.
  • Nearly 55 percent of the settlement comes in the way of benchmark tests to be provided by CTB for second graders. This isn’t a grade in which schools currently do a large battery of tests. We’re also letting them get away with counting the retail value of the benchmark program towards the settlement rather than their actual costs. Think of the way school organizations typically receive donations. If a pizza place donates 10 large pizzas to your PTA fundraiser, and typically they would sell each pizza for $10, this could be seen as a $100 donation. On the other hand, it probably only cost them $30 to make the pizzas. The settlement gives CTB credit for a service that is being provided at a cost to them not even coming close to approaching $678,000.
  • As superintendents and testing coordinators know, districts were asked in June to commit to the benchmark testing provided by CTB. This isn’t even part of the settlement. Oklahoma schools were going to receive this service for free before the testing problems. Listed as only half of one percent of the settlement, it is still misleading.
  • The cost of the independent study seems high, even though it accounts for less than four percent of the settlement. Besides, I thought Barresi and her supporters didn’t put stock in studies paid for by a third party. This will be the ultimate in-house job. And the findings won’t be released until after schools have their test scores (which will be after school begins, but that is an entirely different post I need to find time to write).
  • The technology readiness piece should probably be built into any testing contract in the future. Online testing will eventually replace bubble testing altogether. At some point, even the writing component will be exclusively conducted online. We need to know that schools can handle the load. On the other hand, it was not Oklahoma technology that created these problems. And it is completely unclear what will happen at the last minute if the assessment determines that a school district’s technology is inadequate. It’s not like they can just upgrade with all of the extra money the legislature provided this year.

The settlement is predictably a disappointment. The true costs to the vendor are less than advertised, and the stated benefit to districts is even more misleading. While it is likely to be approved by the state attorney general, unfortunately, it does nothing to restore the confidence we have  lost in either the SDE or CTB.

Continuing Discussion of Testing

July 24, 2013 Comments off

The ongoing saga of this year’s testing debacle cannot be captured as neatly as the Oklahoman editorial page tries to do this morning. Unlike Superintendent Barresi, who has blamed the school districts for the loss of connectivity (after initially blaming the vendor), the paper acknowledges that it was CTB/McGraw-Hill who screwed up. Also clear in today’s discussion is the fact that Indiana has been more transparent and aggressive in dealing with the same problem. The important thing to remember, as the paper points out, is the stress caused to students and teachers:

No amount of money can make up for the stress that students (and their teachers) felt as they tried repeatedly to complete the required tests despite technology-related issues. This was serious because high school students must pass a certain number of tests to graduate; pressure builds for months as test day approaches.

We understand the official reason the state gave for not giving details. It’s not unusual for negotiations to remain hush-hush. But the perception is a problem, particularly given the differing approach in Indiana. It has the feel of trying to shield a poor-performing company from adverse publicity. Problem is, there was no shield for the students and educators affected by the testing company’s apparent failures.

The testing breakdown has become one more distraction for an education system that desperately needs to focus time, attention and resources on core academic issues. And let’s not forget that the issue of damages — which should be substantial — won’t answer lingering questions about the validity of the scores. The validity question should perhaps be determined before deciding on financial and other potential penalties to the company.

It was a misstep for state schools Superintendent Janet Barresi to place some of the blame on school districts. That was inconsistent with previous comments about the problems. And it was no secret that Indiana experienced similar difficulties.

The fallout from the testing problems simply has to run its course. This can’t happen soon enough. A significant debate is happening in Oklahoma and throughout the country about testing, school accountability and academic standards. Too often in recent years, these important discussions have been overshadowed by egos, political rhetoric and poor communication.

Without getting into a “you started it” loop, I think it’s important to focus the debate on what’s important: the children.

The question we are not asking is Why do we test? Is it for the kids, or for us? Does the testing we do tell us anything that we can use? Is the disruption caused to the learning process (even when things go well) worth the expense and the low-quality data yielded from the results? That has to be how we frame the discussion moving forward.

(I also have to mention that our testing problems are compounded by the fact that as a state, we literally have no plan for testing over Common Core, which we have renamed something else, and which we are supposed to begin assessing in the 2014-15 school year.)

As far as the penalties go, the details will be critical. If CTB is to be “punished” by having to provide schools with extra services (such as more testing) that districts didn’t ask for, it’s really not much of a penalty. Tomorrow, when what’s left of the State Board of Education meets, they need to question progress towards a settlement in these terms. If we are to punish a vendor who failed us by agreeing to a settlement that effectually gives them more business (Once we get a taste for free online benchmark tests, we’ll be hooked, right?), it will be the state of Oklahoma who has failed the 2013 tests.

Tomorrow’s SBE Agenda hints at an update on testing. No mention of a settlement is included. I think I’ve established on this blog that I don’t exactly know every law ever written, but I have to think that any settlement would have to be approved by the Board. The lack of this as an actionable item on the agenda tells me either (a) that no settlement has been reached, or (b) that it will not be vetted publicly.

It Depends on What Your Definition of “Up” Is

Almost exactly a year ago, I wrote a post titled Another Bad Decision discussing the SDE’s decision to withhold 3.52 percent of school funding from districts. It was my first post to go over 2,000 page views, and it remains one of my three most-viewed posts. And that was back when this blog had about one-third of the readers it does now. The fact that so many people were outraged over a finance issue that, to be honest, involves a lot of nuance and detailed formulas, shows that many Oklahomans are becoming better informed about education policy.

This year, there’s still plenty of room for outrage. School districts received their initial state aid allocations from the SDE this week, and initial reports indicated a slight increase. From the SDE press release:

OKLAHOMA CITY (July 18, 2013) – This week, the State Department of Education released to school districts initial state aid allocations for the FY14 school year. Districts will see an increase in per-pupil funding of $8.60 per student. The initial allocation for FY14 is $3,038.60 per student as opposed to the $3,030.00 received for FY13.

Contributing most to the increase in FY14 state aid allocations is the State Department of Education’s receipt of an additional $21.5 million in funding for state aid to schools during this past legislative session.

“Our state is growing in population and that is a very good thing, but with more children comes increased financial needs for our school districts across the state. Overall, education received 43 percent of the new revenue in the state’s budget. That increase in funding shows us the commitment to education by the Governor and Legislature,” said State Superintendent of Public Instruction Janet Barresi.

That $8.60 may seem huge to you, and I’ve definitely enjoyed reading some humorous suggestions on Twitter about how to best utilize those funds, but seriously, this fixes nothing. At face value, it’s an increase of 0.3%. In truth, it’s not really an increase at all. It’s higher than last year’s initial per pupil allocation, when the state withheld 3.52 percent of the formula money. It’s lower than the adjustment that was made the next month, however, and considerably lower than the mid-term adjustments that were made in December. Making things worse are the reduced ad valorem funding that districts will collect due to SQ 766, and federal budget sequestration.

Repeat after me: Schools once again will have less money to serve more students and meet more political mandates.

Nobody deserves thanks for that.

OSSBA on the Testing Problems

July 18, 2013 Comments off

In case you missed this one (which I did earlier in the week), here is the Oklahoma State School Boards Association’s take on the testing debacle, as written by Jeff Mills:

It is very disappointing that our State Superintendent, Janet Barresi, has decided to cast the blame on someone else for this year’s testing failures and for future testing issues.  Superintendent Barresi was quoted in The Oklahoman expressing outrage over the testing failures and CTB/McGraw-Hill openly acknowledged that the failure was on their end, not the districts.   CTB/McGraw-Hill said, “we regret the impact…(of) system interruption” and “have made changes to correct the situation[.]” referring to the disruption of 3,000 Oklahoma students and 30,000 Indiana students on just the first Monday and Tuesday alone of federally mandated testing.  NBC News, May 1, 2013.

The testing company stated that the outage in Indiana occurred because “our simulations did not fully anticipate the pattern of live student testing.”  Superintendent Barresi was quoted saying, “I am outraged that our school districts are not able to administer assessments in a smooth and efficient manner.”  Spokesperson at that time for the SDE, Tricia Pemberton, stated, ” CTB/McGraw-Hill said it did not have enough ‘hardware space’ for the number of students who went online.  They assured us that if we continue in the fall, they will test it properly to make sure we don’t  have this problem again.”  NBC New, May 1, 2013. CTB/McGraw-Hill clearly admitted the problem was their fault, not the schools.

So why the change of heart?  After all, someone who oversees a $22 million budget should have ownership of a $27 million dollar contract.  On the contrary,  Superintendent Barresi was quoted saying,  “I had zero involvement in the entire process [testing and contract] from start to finish personally.”  The Oklahoman, May 20, 2013.  Finally, Superintendent Barresi  said “(The testing vendor) crashed for two days because of server problems, but almost every bit of it was due to district issues.  I’m not pointing fingers, but it is the reality.”  Tulsa World, July 3, 2013.

The elected office of State Superintendent of Public Instruction is vital to the growth of Oklahoma students.  It requires accountability and should be taken seriously by a leader that accepts responsibility when things go wrong, seeks to correct mistakes, and praises others for their hard work.

Time and again educators and school board members have offered their assistance to Superintendent Barresi only to have those requests ignored.  When these same people finally get fed up with not being taken seriously,  they take matters into their own hands and commission studies such as the report done recently by the Oklahoma Education Association on the problems associated with CTB/McGraw-Hill and spring testing and the A-F Report made by researchers at OU and OSU at the request of the Oklahoma State School Boards Association and the Cooperative Council of School Administrators.  But these efforts are viewed as attacks on the State Department of Education rather than  helpful, collaborative  tools to be utilized by the SDE and are rebuffed as “whining” or outright ignored.

We will continue to advocate for Oklahoma school board members and the children they represent.

-Submitted by Jeff Mills, Ph.D., executive director, Oklahoma State School Boards Association

%d bloggers like this: