About the Testing Settlement

The State Department of Education announced Thursday that it had reached a settlement with CTB/McGraw-Hill for more than $1.2 million in damages. Below is the SDE press release:

State Department of Education Settles with CTB/McGraw-Hill for Over $1.2 Million OKLAHOMA CITY (July 25, 2013) – State Superintendent of Public Instruction Janet Barresi announced today that the Oklahoma State Department of Education, in conjunction with the Office of Management and Enterprise Services, has reached a settlement agreement with CTB/McGraw-Hill for more than $1.2 million in damages. The settlement covers damages suffered by students and teachers during the testing disruptions and server outages experienced on April 29 and 30 during the spring testing window. “I am very pleased with this settlement. I was outraged over the disruptions during the two day period.  I announced that we would seek damages to the full extent of the contract and we took an aggressive stance. The settlement agreement amounts to three times the value of damages defined in the contract,” said Superintendent Barresi. Superintendent Barresi pointed out that the settlement agreement directly benefits teachers and supports classroom instruction for increased student achievement through several in-kind components. “Our teachers and students suffered the most during the testing disruptions and I wanted them to benefit the most. We’ve accomplished that. During negotiations, we were mindful of the suffering of the children and the stress this placed on teachers, testing coordinators and parents.” The settlement agreement includes: A cash settlement ($367,205) Professional development for teachers to help them become more acquainted with the type of items that can be expected on new English language arts and math assessments and how to adjust instruction so students will be successful on these tests. ($13,000) Formative tests for teachers that can be given on a voluntary basis twice a year to measure student learning and drive instruction for the benefit of increased student achievement in the second grade. ($678,400) Formative tests for teachers that can be given on a voluntary basis twice a year to measure student learning and drive instruction for the benefit of increased student achievement for grades 3 through 11. ($6,600) The commissioning of an independent study to evaluate the impact of the disruptions on student test scores. HUMRRO, Inc. has expertise in the area of analyzing testing disruptions. They will provide an independent opinion that is expected to be delivered in late August. ($48,000) Prior to testing, CTB will conduct a technology readiness assessment of each Oklahoma School District to: ($125,000) -Capture specifications regarding bandwidth, number of workstations, server configuration, etc. at each school site -Identify a technology contact at each school district -Perform online stress tests at every site -Conduct training and deploy implementation services at all sites -Establish a technology forum to deliver regular communications to districts The entire cash settlement amount will be disbursed to schools in order to help compensate for extra personnel costs incurred due to assessment issues this past spring. The State Department of Education will communicate with district superintendents as the HUMRRO report is issues, cash is distributed and in-kind components are put into place. The Oklahoma Attorney General’s office has expressed interest in reviewing the settlement prior to OMES finalizing paperwork.

While I’ve been tardy in adding my thoughts on the settlement, others have been all over it.

From the OEA:

“About $678,000 of the “in-kind” services are not needed, nor required at this time. Second grade is not a testing grade in Oklahoma, and we should keep it that way. Our students are already over-tested and filled with anxiety because of high-stakes tests.  Our second-graders should not be put under that amount of stress.” said Oklahoma Education President Linda Hampton.

From Broken Arrow Superintendent Jared Mendenhall:

And a great article from the Tulsa World with several good quotes:

“When you consider that we have over 500 school districts in the state, that equals about $700 if divided evenly,” said Janet Dunlop, assistant superintendent at Broken Arrow Public Schools. “What effect will that truly have on our overtime costs, not to mention time lost to instruction due to testing delays?”

“If ‘affected’ simply means students whose tests were invalidated and were forced to retest on another day, then yes, 9,100 may be correct,” Dunlop said. “However, if we count all the students whose tests were delayed by significant wait times between questions due to the overload of the system, or students who were bumped from the system but who were able to log back on after a wait of sometimes up to an hour, then the figure of 9,100 is significantly lower than reality.”

Added Dunlop, “Why would we want more of a product when we don’t have confidence that the testing system will even operate?”

On one hand, it was just nice to see Barresi include herself in the process. And she did acknowledge that the malfunction was CTB’s fault. If you’ll remember, first it was, then it wasn’t, and now it is again. Aside from that, each of the six bullets listed in the settlement is worthy of few sentences.

• The cash settlement is paltry. It offsets nothing. The extra time for school employees and the disruption to students is worth a hell of a lot more than 30 percent of the total cost to CTB. Also, this is probably the only area where we are seeing the true cost to the testing company.
• Professional development for teachers will likely be provided in the form of webinars. Do you know what people do when they’re on a webinar? They check their email and catch up on Words With Friends. They are not engaged learners. Good professional development comes with opportunities for collaboration and is ongoing over a significant period of time. For one percent of the total settlement, any professional development provided will be completely meaningless.
• Nearly 55 percent of the settlement comes in the way of benchmark tests to be provided by CTB for second graders. This isn’t a grade in which schools currently do a large battery of tests. We’re also letting them get away with counting the retail value of the benchmark program towards the settlement rather than their actual costs. Think of the way school organizations typically receive donations. If a pizza place donates 10 large pizzas to your PTA fundraiser, and typically they would sell each pizza for $10, this could be seen as a $100 donation. On the other hand, it probably only cost them $30 to make the pizzas. The settlement gives CTB credit for a service that is being provided at a cost to them not even coming close to approaching $678,000.
• As superintendents and testing coordinators know, districts were asked in June to commit to the benchmark testing provided by CTB. This isn’t even part of the settlement. Oklahoma schools were going to receive this service for free before the testing problems. Listed as only half of one percent of the settlement, it is still misleading.
• The cost of the independent study seems high, even though it accounts for less than four percent of the settlement. Besides, I thought Barresi and her supporters didn’t put stock in studies paid for by a third party. This will be the ultimate in-house job. And the findings won’t be released until after schools have their test scores (which will be after school begins, but that is an entirely different post I need to find time to write).
• The technology readiness piece should probably be built into any testing contract in the future. Online testing will eventually replace bubble testing altogether. At some point, even the writing component will be exclusively conducted online. We need to know that schools can handle the load. On the other hand, it was not Oklahoma technology that created these problems. And it is completely unclear what will happen at the last minute if the assessment determines that a school district’s technology is inadequate. It’s not like they can just upgrade with all of the extra money the legislature provided this year.

The settlement is predictably a disappointment. The true costs to the vendor are less than advertised, and the stated benefit to districts is even more misleading. While it is likely to be approved by the state attorney general, unfortunately, it does nothing to restore the confidence we have  lost in either the SDE or CTB.