October Review/November Preview (2013)

If I made a habit of assigning subtitles to months, there would be several choices for this October. Most would probably include the word snafu, which apparently not everybody knows to be an acronym with a military origin. Those of us partaking in Twitter had a pretty good time with that one!

Frankly, my readers and I have a lot of concerns in public education these days. As school districts scramble to account for bad ideas such as the 3^rd Grade Retention law (with no funding), they don’t need the distraction of an A-F Report Card system that offers no value to the public. I had to laugh today when Superintendent Barresi posted to her campaign blog that superintendents need to focus on kids rather than “ridiculous political theater.”

https://twitter.com/JanetForKids/status/395565645464481792

She fails to understand (or chooses to pretend to the contrary) that the A-F Report cards are nothing but “political theater.” They are a cheap stunt intended to separate schools into groups and play into the hands of charter school operators, who are just waiting for a more permissive path into our state. They have no statistical meaning. The criteria are poorly selected, and the formula is nonsense. Making things worse, the SDE can’t even figure out how to use it.

Two things should come as no surprise then. First is that it was another record month for the blog. As of this writing, there have been more than 20,000 page views in October (eclipsing the 17,900 from August). That’s pretty good considering (a) I write anonymously and in my spare time, and (b) I don’t make a penny from this. The second is that three of the top five (and eight of the top ten) posts from this month concern the report cards:

1. Simply Outraged – This brief 141 word post more than doubled the next most-viewed entry. It was a simple, quick response of emotion to the fact that grades were changing almost as quickly as administrators could view them. Little did we know what the next few weeks would hold.
2. Stuck in the Middle with A-F Report Cards – Readers who have been with me for long enough know that I can’t resist a good classic rock, movie, or poetry reference. More so, you know how much I love it when our state politicians are whisked off to conferences to hobnob with Jeb Bush. I assume no taxpayer dollars were spent on the trip, but it did not sit well with many of the state’s school administrators that Barresi and Sen. Clark Jolley were off playing in Boston while the A-F system caused problem after problem.
3. One More Delay – While schools were verifying their testing data early this month, the SDE sent an email to superintendents letting them know that thousands of fifth and eighth grade writing tests still needed to be re-scored. There was no explanation of why this was happening six months after students took the tests. What was explained later on, after the A-F extravaganza began in earnest, was that regardless of the outcome of the re-scoring (of thousands of tests statewide), the results would not be counted in schools’ report cards. While it is equally possible that the excluded scores would be positive or negative, this points another finger at the testing company, the SDE, and the usefulness of the A-F grades.
4. In Defense of Opt Outs – You have probably heard that the SDE is deeply concerned that Jenks parents thought for themselves and opted their children out of field tests. How dare they want to limit the wasted time their children have to endure?! The problem, from the SDE’s perspective, is that they don’t believe the movement was parent-driven. I have two thoughts: yuh-huh, and so what? Jenks Middle School parents claim the opt-out movement was their doing, not principal (and fellow blogger) Rob Miller. I’ve looked over a number of statutes since this witch hunt was revealed. I don’t see what laws they think he or the parents broke. As I said at the time, the SDE should carry on as if this never happened. No good can come of it.
5. The Pot We Watch – We’ve made a big deal out of the delays in test results and the release of A-F Report Cards. These are the distractions. There are many other pieces of data or funding that the SDE hasn’t made available to districts either. The SDE holds all the cards and apparently wants to lord them over schools. This is what we’ve come to expect from the agency that promised to be “less regulatory and more service-oriented.”

What does November hold? For starters, there will be a special State Board of Education meeting to approve the final A-F Report Card grades and award a new testing contract on November 6^th. I also expect Barresi to persist with the lie that school districts can give teachers a $2,000 raise without additional funding. As always, expect a surprise or two as well.

About the FY15 Budget Request

Today, since the State Department of Education has postponed the release of the A-F Report Cards (special board meeting Nov. 6), the most interesting thing about the monthly State Board of Education meeting was the release of the Fiscal Year 2015 (July 1, 2014 to June 30, 2015) budget request. Every year, the SDE’s finance office prepares a proposed budget around this time and presents it to the SBE, which can approve or disapprove it. It then goes to the Legislature, which will chop it up and give them part of what they’re asking for. In reality, it’s a wish list.

That won’t stop us from analyzing the wishes of the SDE, though, will it? You can look at their original budget here. I took their numbers and created my own spreadsheet (because you’d be disappointed if I didn’t).

Activity	FY14	FY15	$ Increase	% Increase
Financial Support of Schools	$1,837,570,779	$1,919,019,250	$81,448,471	4.43%
School Performance Incentive	–	$16,289,694	$16,289,694	New Activity
State Aid Formula	$1,837,570,779	$1,902,729,556	$65,158,777	3.55%
Instructional Materials	$33,000,000	$35,452,248	$2,452,248	7.43%
Activities Budget – Total	$507,022,781	$596,500,385	$89,477,604	17.65%
Ag in the Classroom	$38,675	$39,000	$325	0.84%
Oklahoma Technical Assistance Center	$300,000	$300,000	–	0.00%
Statewide School Programs (Alt. Ed.)	$13,727,366	$17,377,366	$3,650,000	26.59%
Early Childhood Initiative	$10,500,000	$10,500,000	–	0.00%
Early Intervention Sooner Start	$14,417,922	$14,417,922	–	0.00%
NBPTS Teacher Bonus	$11,695,000	$11,695,000	–	0.00%
Speech Pathologists and Audiologists	$3,247,350	$3,247,350	–	0.00%
Flex Benefit Allowance (Certified)	$244,347,203	$275,423,849	$31,076,646	12.72%
Flex Benefit Allowance (Support)	$123,433,659	$151,519,168	$28,085,509	22.75%
Oklahoma Arts Institute	$350,000	$350,000	–	0.00%
Oklahoma Parents as Teachers	$1,000,000	$1,000,000	–	0.00%
Personal Financial Literacy	$150,000	$150,000	–	0.00%
Reform Implementation	$43,014,000	$69,494,990	$26,480,990	61.56%
ACE Remediation	$8,000,000	$21,765,840	$13,765,840	172.07%
AP Training, Test Fees, AVID, NMSI	$4,150,000	$5,743,600	$1,593,600	38.40%
Charter Schools Incentive Fund	$50,000	$200,000	$150,000	300.00%
Oklahoma Academic Standards Implementation	$564,000	$564,000	–	0.00%
STEM Ready Elementary Schools	$300,000	$300,000	–	0.00%
Think Through Math	$ 1,800,000	$2,400,000	$600,000	33.33%
Reading Sufficiency	$6,500,000	$16,091,550	$9,591,550	147.56%
TLE – VAM Development	$2,000,000	$2,000,000	–	0.00%
3rd Grade Reading Readiness Support Teams	$500,000	$500,000	–	0.00%
Staff Development for Schools	$4,250,000	$4,250,000	–	0.00%
Oklahoma Student Information System	$2,000,000	$2,000,000	–	0.00%
REAC3H Coaches	$4,250,000	$5,000,000	$750,000	17.65%
Testing	$3,000,000	$3,000,000	–	0.00%
Teach for America	$2,500,000	$2,500,000	–	0.00%
School Reforms Competitive Grants Pool	$3,150,000	$3,150,000	–	0.00%
Rural Infant Stimulation Environment (RISE)	$529,943	$600,000	$70,057	13.22%
School Lunch Matching	$4,960,288	$5,074,365	$114,077	2.30%
Teachers Retirement System	$35,311,375	$35,311,375	–	0.00%
Administrative and Support Functions	$22,426,642	$24,000,000	$1,573,358	7.02%
Agency Operations	$11,426,642	$13,000,000	$1,573,358	13.77%
Assessments/Testing contracts	$11,000,000	$11,000,000	–	0.00%
TOTAL SDE BUDGET	$2,400,020,202	$2,574,971,883	$174,951,681	7.29%

Categorical expenses are in bold. For example, Financial Support of Schools is a categorical expense, while School Performance Incentive and State Aid Formula are the component expenses. Under the Activities Budget category, I further designated the Reform Implementation expenses using red text.

Starting first with the overall numbers, the SDE is asking for an increase of nearly $175 million to the education budget. This includes an increase of just over $65 million to the funding formula. If the Legislature were to surprise us and give the SDE what they’re asking for, this would be the only money that school districts could use for basic functions, such as salaries, utilities, transportation, and the like. By comparison, this would still be less money in the State Aid Formula than there was in FY08. Yes, seven years later, we still aren’t catching up.

If it were me rubbing the lamp, the first thing I would ask of the genie would be adequate funding for schools. The SDE may be asking for an increase in the money going into the State Aid Formula, but they asked for a much bigger one last year. In fact, they justified the request in the margins by explaining that it was based upon 2010 funding levels and the increase to the number of students enrolled in public schools. Apparently, the need this year seems less severe. Last year, they asked for $234.7 million to be added to the funding formula. The legislature gave $21 million, or about nine percent of what they asked for.

When asked by a board member if teachers could expect a raise based upon anything listed in the budget, Barresi simply responded that she has asked districts to do that on a voluntary basis. Remember, her people continue to try to make this the number one talking point of the entire campaign. She seems to think if she complains enough about administrators, people will forget everything she has done during the last three years.

It would certainly appear that voters with a short attention span are her best bet at winning a second term.

The proposed budget also includes something new labeled School Performance Incentives, which ring in at more than $16 million. Details were vague for this line item, but the SDE did present a slide showing potential high-poverty, high-achieving schools that might qualify for the incentives. I assume this would require new legislation, or at the least, new administrative rules. While I wouldn’t describe this as an expense that supports schools, it does give us a hint about at least one thing we’ll be discussing throughout the Spring.

Overall, the listed increase for Financial Support of Schools is 4.43%. The next category – Instructional Materials – shows an increase of 7.43%. While purchasing textbooks is certainly expensive, and the money allocated to districts hardly covers this expense, is this a more critical need than that of restoring funding to the State Aid Formula (which actually only increases by 3.55%)?

The next category is the Activities Budget, which includes pretty much everything else besides testing and the cost of running the SDE. The SDE proposes an overall increase of $89.5 million for this function (17.65%). Within this area are two major increases – FBA and Reform Implementation.

The Flex Benefit Allowance (FBA) is the cost of providing health insurance for employees. Yesterday, while speaking to a group of first year superintendents from around the state, SDE Chief of Staff Joel Robison made the claim that Obamacare is costing the state more than $59 million in FBA increases. Superintendent Barresi doubled down on that position today:

As is proving to be the case throughout the nation, the consequences of Obamacare are severe and painful. Millions of dollars that could have gone to the classroom instead must be eaten up in insurance costs.

I’m certainly not going to argue that health care is inexpensive. Since we often look at 2008 as a baseline year for funding comparisons, I thought it might be instructive to look at the average monthly FBA cost for employers (school districts) since then.

Year	Monthly Premium  (FBA)	Dollar Increase	Percent Increase
2008	$364.24
2009	$409.12	$44.88	12.32%
2010	$442.80	$33.68	8.23%
2011	$449.48	$6.68	1.51%
2012	$449.48	–	0.00%
2013	$463.99	$14.51	3.23%
2014	$484.87	$20.88	4.50%

 

The FBA increase for next year is the third-highest in the last six years. Notably, the two highest increases were at a time when school district budgets were being hit hardest. So yes, it’s fair to say that next year’s increase places a burden on schools. I’m just not sure if blaming the increase on the Affordable Care Act is entirely accurate.

In case voters have a longer attention span than she had hoped, she could always resort to clamoring over President Obama specifically and liberals in general.

The FBA discussion is a nice distraction. The red meat in this budget is Reform Implementation, for which the SDE asks for an increase of $26.5 million (61.56%). Within this total are six activities that I will discuss separately.

• ACE Remediation (172.07% increase) – One of two things is happening here. Either the SDE realizes that schools need more money to support remediation programs for high-stakes high school testing or the cut scores for those tests are about to get a whole lot scarier. While I hope it’s the former, what districts really need is access to those funds sooner. It will be at least mid-November at this point before schools get their funding notices.
• Advanced Placement Training, etc. (38.40% increase) – Restoring support for AP programs would be welcome. Teachers miss the opportunities that used to be more abundant for intensive two-day trainings during the school year, in addition to the summer institutes. Also, as more schools increase the number of AP classes they offer, more students will need help with the test fees.
• Charter Schools Incentive Fund (300% increase) – I don’t know what charter schools (or prospective ones) have to do to access this money, but the request shows that the SDE wants to see more of them in Oklahoma.
• Think Through Math (33.33% increase) – Some teachers have found this to be a useful tool for helping students reinforce (or remediate) math skills. Many schools, however, have purchased their own support programs. Time after time, we see that migrating between supplemental programs (or using more than one simultaneously) diminishes the effectiveness of both. This increase in the budget may or may not be something schools want.
• Reading Sufficiency (147.56% increase) – As with ACE Remediation, we really need to fully fund the programs that have the highest stakes for students. Anyone who visits this blog frequently knows that I’m no fan of the 3^rd grade retention law. That said, as long as it is something for which schools have to plan, I’m all for giving them the support structure they will need to be successful. The law is bad for kids. More funding for staff, resources, summer programs, and interventions is the best way to mitigate the damage.
• REAC3H Coaches (17.65% increase) – Among the stories told today at the SBE meeting was the one about how districts want more and more of the coaches in their schools. I would argue that it depends on whom you ask. I would also point your eyes up a couple of lines to the staff development funding for districts. Last year, these two lines were on even footing. Once again, the SDE is trying to dictate the needs of districts.

I should probably add a few words about some other programs with flat funding requests. Once again, the SDE has missed an opportunity to increase collaboration among some of the most effective teachers in the state by choosing not to increase funding for NBPTS stipends, or restoring scholarships for more teachers to go through that program. Conversely, the money being flushed down the drain for programs such as TLE-VAM development, Teach For America, and Testing could be used to help students, in an ideal world.

To close this extra-long post, I’ll just add what I always say when we’re discussing the disappointing state of school funding in Oklahoma. Schools are being asked to meet more mandates for more students with less money and what can be best described as superficial support from state leaders.

The Pot We Watch

I predict that tomorrow, the sun will come up. It will probably be hidden behind clouds, but it will be there nonetheless. I have faith (or a basic grasp of science).

I also predict the State Board of Education will hold its monthly meeting as scheduled. Since schools have been informed that the release of A-F Report Cards has been postponed, I will go out on a limb and say that they’re going to talk about that.

I doubt, however, the SBE will have a discussion about other key resources and pieces of information that remain unavailable to schools. Allow me…

State Averages – In addition to finally having report card grades finalized, schools would like to know the state averages for the tests they had students take six months ago. With so many of the cut scores moving as they did, having that information would be helpful in giving data reports to school boards. Numbers always need context. If our scores went down in Biology, but we’re still above the state average, then that can be a reference point. Not only do we want to see improvement in our test scores each year; we want to see improvement relative to state averages.

Waiver Designations – The A-F Report Cards are but the first hammer to drop. To be honest, they are little more than a marketing tool. If you have a good grade, you promote that to your community. You put a banner up at the local Chamber of Commerce. If you have a bad grade, the SDE calls you names and tells the world you have failed the children, which probably isn’t accurate.

Oklahoma’s waiver to No Child Left Behind, however, comes with some action items for low-performing schools. Not only can schools be placed on the Targeted Intervention Schools list with a D or the Priority Schools list with an F, they can be placed on the Focus School list with a higher grade. All it takes is for one of the three designated subgroups to score lower than 90 percent of the state. At this point, schools literally have no way of knowing whether they are on these lists (although to be fair, schools remaining in the D or F range during all the flip-flopping of the last couple of weeks pretty much know). Halfway through the year is a little late to have schools start jumping through the hoops that come with these labels.

On a related note, I suppose schools might like to know if they have landed on either of the reward lists so they can receive “flexibility incentives.” Last year, though, only 14 of 229 eligible schools applied to receive their reward.

RSA Money – In case you somehow missed it, this is the year that Oklahoma’s third grade retention law (the Reading Sufficiency Act) kicks in. It sure would be nice if the funding allocated by the legislature were available to schools now. They’ve done their part, completing beginning of the year reports. They even have plans for using those funds to help their struggling readers. Many, in the absence of an allocation, have put those plans into motion anyway, hoping the funding catches up. The problem – as always – is planning with so much remaining uncertain. Without knowing how much money will be available, a school or district has a hard time knowing how far their scant resources will go.

ACE Remediation Funds – As with the RSA money, schools have not received one dime of support this school year to help remediate students needing to pass End of Instruction tests to graduate, as required by the Achieving Classroom Excellence law. This can be attributed, in part, to the delay in finalizing test scores. Allocations are based on the number of students scoring below proficient on the EOIs or on the 7^th or 8^th grade reading and math tests. Since the pot of money is only so big, the SDE has to have a final number to work with in order to make the allocations.

I get that. Life is hard up at the Hodge Building. It wouldn’t be that much of a challenge to do an estimate at the beginning of the school year, allocate half to schools at that point, and then make the rest available after scores are final, would it? Educators know that the longer we wait to begin remediating students, the less effective it will be. Plus, we are quickly approaching the winter re-testing window. It would be nice to get some re-teaching and preparation done before that.

Federal Program Reimbursements – We all know that the federal government has had its money problems this year. First there was Sequestration, a budgeting process through which districts lost about 10 percent of their federal funding. Then there was the Shutdown, in which millionaire and billionaire politicians played a riveting game of chicken (not enough brains for chess, I guess) while workaday bureaucrats took unplanned (and unpaid) vacation time.

While Sequestration impacts school budgets, the Shutdown should not. Districts have known the amount of their federal aid (Title I and other Title programs, special education, child nutrition, certain career tech programs) for a few months. They diligently planned and submitted their budgets. Now they wait patiently for those to be approved. This matters because schools do not receive federal funds in advance. They receive reimbursements for approved expenses in those programs.

Last year, schools did not begin receiving payments on their federal claims until after New Year’s Day. On the hook for huge personnel, training, and materials expenses, districts began to worry about cash flow. This year, understandably, many finance and federal programs managers around the state are worried it will happen again.

By the way, if you ever wonder again why districts like to keep carryover funds, maybe you should re-read the last six paragraphs.

It’s fair to say that the recent delay in the release of A-F Report Cards has Oklahoma school districts playing the waiting game. Unfortunately, that’s not the only thing slowing them down.

Looks like a Monday

Even though A-F Report Card grades changed again over the weekend for many schools, the SDE is holding firm to the insistence that all data verification requests be completed by 10:00 a.m. … wait, that was this morning? !@#%@#$:

***SDE***Data Verification Deadline

OK State Dept of Ed sent this bulletin at 10/25/2013 04:01 PM CDT

OK State Dept of Ed sent this bulletin at 10/25/2013 04:01 PM CDT

Superintendents, Principals and District Test Coordinators,

Please note that 10:00 am Monday, October 28, is the due date for all Data Verification Requests to be submitted via the Single Sign-on page.  The Accountability staff will continue to work diligently to process all of the Data Verification Requests submitted.  You will receive a response to any outstanding requests by early next week.

Thank you for submitting your Data Verifications so that we have accurate information for our accountability measures.

Maridyth McBee, PhD

Assistant State Superintendent

Accountability and Assessment

405-522-6250

To be fair, they did send out notice on Friday, arguably before the close of business. With more unexplained changed over the weekend though, it really caused some scrambling today.

About the Growth Points

I’ve lost count of how many times grades have changed. I think it depends on how frequently superintendents were hitting refresh their browsers. Over the weekend, schools logging on to the SDE secure website were likely to see their A-F Report Card grades jump around. Again. Maybe even multiple times.

We received the explanation Friday that the grades would be released late due to “an abundance of caution.” It’s an abundance of something alright. That caution turned to the wagging finger of scorn, quickly blaming schools for continuing to find mistakes with the grades.

In the last two weeks, countless people have asked SDE staff for an explanation about their grades resembling the Celebrated Jumping Frog of Calaveras County. To their credit, SDE staff have tried to keep up with the volume of success. Gratefully, many of those responses have been forwarded to me. Again, I credit the SDE staff for patiently trying to answer the questions they’re getting. Those charged with directly facing external constituents did not create this mess. The policy-makers above them, the legislature, and Jeb Bush did.

The central idea of most responses is that the growth points are the root of most recalculations. To understand the growth points, it is probably a good idea to review the report card formula first.

The 2013 formula for the A-F Report Cards has three components:

1. Student Performance (50%)
2. Overall Student Growth (25%)
3. Bottom Quartile Student Growth (25%)

There are also bonus points available for certain other criteria. But those are an add-on and not included in the formula per se. And discussing them here is probably not critical to help with understanding the growth issues.

The first part of the formula is simple. Half of a school’s grade is based on the percentage of valid tests that students passed, as long as the students were in school for the full year. Special education students count (whether they took the regular test, took a modified test, or completed a portfolio). English language learners count. All kids count. All subjects count (except for social studies: see here). If students at a school took a total of 100 tests and passed 80, then the school would get an 80 for that section.

For the next section, the formula counts all students who have a valid score for a test that can be matched to a prior test. For example, if a student took a 3^rd grade reading test in 2012 and a 4^th grade reading test in 2013, those scores would be matched and compared. For growth consideration, only reading and math count. If a student took a modified test in 3^rd grade but a regular test in 4^th (or vice-versa), there wouldn’t be a match. The process intends to show linkage between compatible exams. For the formula, any student scoring Proficient or Advanced in 2013 automatically gets a point. Additionally, any student scoring Limited Knowledge in 2013 who scored Unsatisfactory in 2012 also gets a point. From there, the remaining students count for zero points, unless their growth on the test from 2012 to 2013 is greater than the state average growth.

This is where interpretations matter. The SDE only counts students who actually showed growth into this average. The metric they use is the scaled score, or the Oklahoma Performance Index (OPI). There are two huge statistical problems with this. First is that OPI was never intended to be used for comparisons across grades or tests. A scaled score of 650 on the 3^rd grade reading test may not reflect the same relative deficit to a proficient score as a 650 on the 4^th grade reading test. The second problem is that the method excludes a significant number of students from the calculation of growth. In some cases, we were told last year, the average change would actually be a decline of up to a few points.

If we are to blindly accept the idea that OPI change from year to year is meaningful, then we should insist that all students’ OPI change be considered. Excluding students for whom the change was negative introduces all kinds of selection bias to the process. Then again, we wouldn’t want anything about the report cards to resemble an academic study. We all know how the SDE and their newspaper feel about things being too researchy.

That brings me to the third section – calculating the bottom quartile growth. This sounds like it should be simple. Take all the matched scores and rank them based on last year’s performance. Remove the top 75 percent. For all the students who remain, count their growth point (or lack thereof) a second time. Well, it’s almost that simple. From page 16 of the technical manual:

The bottom 25% is determined by rank ordering the previous year’s OPI scores for all students with both pre- and post-scores at a specific school. Students who scored at or below the 25% percentile at that site will be included in the bottom 25% growth calculation. The bottom 25% group is calculated separately for Math and Reading. Because OCCT, EOI, OMAAP, and OAAP exams are on different scales, a bottom 25% will be identified separately for each exam type. In other words, for a school that administers both OCCT and OMAAP exams, the bottom 25% will consist of the bottom 25% of OCCT Math scores, the bottom 25% of OCCT Reading scores, the bottom 25% of OMAAP Math scores, and the bottom 25% of OMAAP Reading scores. A school must have at least four (4) exams of the same type (e.g., OMAAP Math, OAAP Reading, etc.) in order to identify a bottom 25% for that specific type.

The bottom 25 percent of regular tests (OCCT and EOI) are calculated separately from the modified tests (OMAAP) and portfolios (OAAP).Last year, the growth index for the bottom quartile stopped short of 25 percent if a school had few enough low performing students. This year we are supposedly counting up to 25 percent, even if that takes us into students who scored in the advanced range last year.

I say supposedly because most changes to schools’ grades have come from the bottom 25 percent growth. The first mistake was that the SDE actually applied the formula to the top 25 percent. Schools loved that. Since then, there have been too many tweaks to count. You would think by this point, whatever changes remained to be made would be small. They must not be. Many schools have seen their grades move by three points or more over the weekend. While there may not be a big difference between an 82 and an 85, there is a huge difference between a 78 and an 81.

Even if every school in the state sends out a letter like Keith Ballard’s explanation to Tulsa parents, there is going to be a perception problem that schools not receiving a good grade will have to manage. It won’t be based on anything we can trust, but the problem will exist just the same.

The point here is simple. We’re too far into the process now to see scores still acting with this much volatility. As hard as the hard-working SDE staff charged with managing this travesty work, nothing can explain that.

***

Additional resources from the SDE:

AtoF_Guide_2013

2013_State_Average_Positive_OPI_Change_By_Grade-Subject_1

Another one for the SNAFUBAR

No, that’s not another educational acronym. It’s the most succinct way I can summarize today’s announcement from the SDE:

Release of state A-F Report Cards delayed until early November OKLAHOMA CITY (Oct. 25, 2013) –To ensure complete accuracy with the A-F Report Cards for Oklahoma schools, the Oklahoma State Department of Education is delaying public release of the grades. The letter grades, which communicate a school’s academic performance in an easily understood snapshot, had been scheduled to go before the State Board of Education at its Oct. 29 meeting. Instead, the grades will be presented to board members during a special meeting to be held within the next two weeks. Education Department staff has worked to remedy all concerns with accuracy of the grades, which were calculated using a new formula established earlier this year in legislation, House Bill 1658. “In an abundance of caution, the State Department of Education is going to take additional time to guarantee absolute, 100-percent accuracy of the grades,” said state Superintendent of Public Instruction Janet Barresi. “Despite two periods in which school districts were to certify assessment results as well as the current review underway, our assessment office is still receiving a number of corrections and changes. To say this has been frustrating is putting it mildly. The A-F Report Cards are too critical a tool for parents and communities to accept anything less than quality.”

We’re nearing the end of October, and the SDE still can’t finish the testing process they screwed up in April. Just as when the testing company failed us completely, Barresi is blaming school districts. This has nothing to do with the fact that grades changed at least five times in a 24 hour period last week. This has nothing to do with the different OPI growth scales (explaining that one is another post altogether) that have been disseminated this week. No, it’s the school districts.

It’s not just the districts though; it’s also the legislature. They changed the formula, dontchaknow (that’s my best Sarah Palin voice – deal with it). They changed it months ago, however, and the SDE still can’t get this right.

In case you’re scoring at home, this is two years in a row that the report cards have been delayed. In spite of what the news release says, these grades are not a “critical tool for parents and communities.” They are propaganda.

Earlier today, I had started to write a response to the Tulsa World article discussing Dr. Keith Ballard’s letter to Tulsa Public Schools parents. Ballard wrote, in part:

“Never in my career have I seen this level of dysfunction and ineptitude coming out of the Oklahoma State Department of Education. Last week, school grades began to trickle out, but they were not ready for prime time. As school principals logged into the OSDE website, many of them watched in horror as grades that were initially posted as A’s, B’s and C’s began to morph into C’s, D’s and F’s. A feeble apology was issued by the OSDE for serious miscalculations that should have been caught before any grades were posted. At last count, grades were changed five or six times by the State Department of Education. To say we have no trust in this system is an understatement. The incompetence in the roll-out of these grades is rivaled only by the testing debacle that took place in April when computer servers of the state’s new testing vendor crashed, interrupting test-taking throughout the state.”

Neither the testing collapse nor the inconsistencies of the grading process point to problems caused by schools. The testing company did not anticipate the surge of students signing on even though they had bid on the contracts knowing how many students each state had. And the SDE simply has not had a clear protocol for applying the formula that they ostensibly want to use.

The World article quotes new SDE Director of Communications, Phil Bacharach, on the agency’s position about Ballard’s letter:

“First, it is important to note that changes in the new law on the A-F Report Card have resulted in a grading formula that is more reflective of academic realities,” he wrote. “With all due respect, given Dr. Ballard’s stated political preference, this letter amounts to campaign material on school district letterhead.It’s unfortunate that the head of a school district would undermine an initiative that gives parents information they need about how their child’s school is doing. It’s worse than unfortunate – and perhaps unconscionable – that he would do so before the grades even come out. Given the academic challenges that exist in Tulsa Public Schools, this letter is disappointing. Tulsa schoolchildren deserve better.”

All of Oklahoma’s schoolchildren deserve better, but not for the reasons stated by Bacharach. Ballard wasn’t campaigning. He was speaking honestly to parents (on school letterhead). Most Oklahoma superintendents share his frustration. So do principals, teachers, and a growing number of parents – who are becoming increasingly aware of the problems their schools have in dealing with the state educational agency.

Of course, the SDE continues to defend these simplistic and statistically unreliable report cards. They (and their defenders) also continue to cast aspersions at the OU/OSU study criticizing our A-F Report Cards. Here is the opening paragraph of the study’s executive summary:

“We are among those who favor examining schools to determine how effective they are in their mission to maximize learning for all children. We are passionate about making the evaluation of schools a truthful and credible process. Oklahoma is one of many states that has chosen to report school performance using a single letter grade generated primarily from standardized test results. In a white paper published earlier this year, we examined Oklahoma’s school evaluation system and discovered fundamental flaws that make letter grades virtually meaningless and certainly ineffective for judging school performance. Our analysis and conclusions were reviewed by two nationally renowned testing and evaluation experts who concurred with our claims. Subsequently, the State made some changes to the system, but the changes do not address the flaws; in fact, the likelihood is that they made them worse.”

What the researchers could not have foreseen is that the revised methodology, in addition to being worse, confuses the people tasked with applying it. The list of reasons for a letter like Ballard’s to parents is huge. It grows daily. The dissatisfaction with the people making decisions at the Capitol and the Hodge Building is not political. It stems from the frustration that our work in school districts suffers from confused interference that threatens the effectiveness of the most competent educators among us. 

Those Pesky Academics

Predictably, today’s editorial in the Oklahoman completely misses the point of the OU/OSU study critical of our state’s A-F Report Cards. Here is the first paragraph:

With the release of Oklahoma school sites’ A through F grades looming, opponents of accountability are predictably ramping up attacks. School officials should think twice before embracing one such tirade issued by a small group of college academics. To discredit A-F school grades, those researchers effectively argue that there is little correlation between a public school education and actual student learning.

The second sentence is the kicker for me. Policy makers and editorial writers who eschew the work of the highly educated are quick to reveal their own limitations. This was no tirade. This was a long, methodical study of student data, including test scores and key socioeconomic variables. And their finding was quite different than the representation given here by the paper.

Read the report for yourself. The findings are compelling and spot on.

Of course, it wouldn’t be an editorial from the Oklahoman if they didn’t mention money. Here’s the last paragraph:

Even as some school administrators demand a $200 million education funding increase, it’s ironic that they may also embrace a report that suggests public school expenditures are as beneficial to raising student achievement as dumping money from a helicopter.

Yes, those darn school people want enough money to do the things that state and federal laws require them to do. The premise conveniently forgets that $200 million would not even restore per pupil funding to 2008 levels.

Teacher quality matters. School culture matters. Communication with parents matters. Money matters. Nothing matters as much as the home life of children, however. Some children come to school with every advantage in the world. It isn’t hard to get that group to be successful. Some children come to school from a foundation of distress. While it is possible to get individuals in this group to experience success in school, it is much harder.

It is worth the effort and yet incredibly draining. Teachers burn out faster teaching kids from poverty. And the gains they make with students are harder to maintain. That said, even Oklahoma’s most challenging schools have teachers show up every day, ready to teach, bruised from being tossed around among reform movements, and committed to the children.

Editorials, by their nature spin the truth. They take shreds of facts and swirl them together with a larger agenda. Fortunately for us, the Oklahoman does not even attempt to hide theirs.

Conjuring Enemies

Since it’s Fall Break across much of the state, I assume many of you have been at home, responsibly playing the new SDE A-F Report Cards Drinking Game for the last couple of days. Scores changed again… shot! Phone call to the SDE that doesn’t get past the switchboard…shot! Actual response to a question from a live person at the SDE…double! Superintendent Barresi responded today to the frustration of those who have been watching school report card grades flip back and forth like a metronome over the last 48 hours:

***SDE*** A-F Report Card Update

OK State Dept of Ed sent this bulletin at 10/18/2013 09:42 AM CDT

School districts and schools now have access to review their A-F report card and double check their data and calculations. I understand the experience of the past few days has been frustrating for school and district administrators. I am deeply sorry for the resulting delay and confusion, as are all of us at the Oklahoma State Department of Education.

A commitment to transparency can have an embarrassing downside. That was certainly the case when the grades were posted Oct. 16 even as they continued to undergo several versions. A last-minute correction in the calculation resulted in errors that subsequently had to be fixed. To ensure transparency in this process, the decision was made to leave the grades up as they were modified.

Fortunately, the problems were addressed and corrected during the designated 10-day window for schools, districts and the OSDE to check for errors. That doesn’t excuse the snafu, but only explains it — and we thank you for your patience.

Because of the delay, districts and schools have an extended period, until 10 a.m. Oct 28, to review and seek corrections on their prescribed grades. After that point, the proposed grades will be brought before the State Board of Education at the board’s Oct. 29 meeting for final approval.

Some opponents of school accountability will no doubt seize on the recent delay as yet another reason to postpone, reconfigure or simply trash the A-F report cards. Oklahoma parents, students and all interested parties can rest assured that will not happen. The annual grades are critical to heightening accountability, arming parents with important information and furthering the simple proposition that all children can learn.

Janet Barresi

State Superintendent of Public Instruction

The first thing I want to address is the word snafu – but I don’t have to. Rob Miller has that one for me. As Rob does, I think the acronym fits, but maybe not as much as another one.

Her flippant remark about the embarrassing downside notwithstanding, what I really want to address is the last paragraph, and I’m going to do so sentence-by-sentence.

Some opponents of school accountability will no doubt seize on the recent delay as yet another reason to postpone, reconfigure or simply trash the A-F report cards.

The frustration that has been expressed over A-F Report Cards, going back at least 18 months, is not a product of opponents of school accountability. As I’ve said before, politicians such as Barresi like to pretend they invented accountability. Nothing could be farther from the truth. Test scores have been made available online since the late 1990s. The 1500 point API scale was in place prior to No Child Left Behind and had an 11-year run before being replaced by the A-F Report Cards. Only the current occupants of the SDE feel the need to keep those reports offline now, and only they could justify such action in the name of transparency.

Remember, after last year’s catastrophe, the report cards were completely reconfigured by legislators. One of these was Clark Jolley, who went to Jeb Bush’s conference (at which Barresi also presented) this week to praise their impact on accountability. Nor was it the Education Establishment or rogue liberals that she often invokes as enemies authoring studies blasting both last year’s version of the report cards and now this year’s. It was researchers at Oklahoma’s flagship universities – OU and OSU.

Schools welcome accountability. They want a light to shine on how they are performing, knowing full well that what will be seen won’t always be flattering. The people who lead and work in those schools just want this to be done in a way that is accurate.

Oklahoma parents, students and all interested parties can rest assured that will not happen.

I don’t think that is hers to promise. Last year, the State Board of Education surprised everybody by postponing the release of the A-F Report Cards. That was followed by pointed words from Barresi for pretty much everybody. There could be more changes by the legislature. There could be an unsuccessful primary next summer. Who knows what will happen? I don’t know about you, but I feel neither rested nor assured.

The annual grades are critical to heightening accountability, arming parents with important information and furthering the simple proposition that all children can learn.

The system arms (not a fan of the choice of words here, but we’ll go with it) people with misinformation. Schools are trying to figure out how to get the highest possible grades rather than focusing on children. Even worse, the rules have been different each of the last three years, and the rules for 2012-13 weren’t completed until students had gone home for the summer. The discrepancy between student performance and the overall grades is even more telling.

Reformers such as Barresi didn’t concoct all by themselves the ideal that every child can learn. This has been the focus of teacher prep programs for decades. It means something coming from career educators. It means little coming from politicians such as Barresi, Jolley, and Bush.

Is this a snafu? Yes. To be more accurate, I’d call it fubar.

Quit Picking at It!

You’ll just make it worse!

That’s what my parents used to tell me when I would fall off my bike and scrape my knee.

We’re up to version 5 of our A-F Report Cards.* There’s no end in sight. Around lunchtime, the SDE sent out the following notice to districts:

***SDE*** Report Cards UpdateOK State Dept of Ed sent this bulletin at 10/17/2013 12:42 PM CDT Dear Superintendents, Principals and DTC’s, It came to our attention yesterday that the bottom 25 percent growth on the A to F report card was calculated incorrectly.  A last-minute correction was made immediately before posting that inadvertently caused the errors. We are working to remedy this problem as swiftly as possible, and we will notify all districts once this has been corrected. The date for submitting Data Verification Forms for calculation errors is extended until 10:00 am, Oct 28th. I deeply regret the challenges you experienced yesterday afternoon.  If additional calculation changes are needed, please submit the Data Verification Form and we will be happy to process it. Maridyth McBee, PhD Assistant State Superintendent Accountability and Assessment

The problems didn’t end yesterday. That needs to be clear. They are ongoing. School districts continue to have deadlines, like the one that was extended earlier this week (due to system problems, not school district procrastination). They meet them. They get their data entered correctly.

Their frustration is evident on the secure site, which includes a discussion forum. Here are some quotes from system users:

Just like Obamacare, this site was not ready for rollout.

When will the scores quit changing? We have seen at least 4 different scores in less than 24 hours. OSSBA and CCOSA were right. This is worthless.

My school has 2 different scores simultaneously.

If I go in as District User I see one score. If I go in as Site Principal I see a different score. Of course this is aside from the fact that each score has changes 4 times at least. The discrepant score are occurring simultaneously.

Please take the keyboard away from the monkey!!

Why have a forum for Q & A when there is no one home to answer our questions? Why release something that obvious was not ready for release. Don’t send me junk I want valid information not this stuff. Our scores have changed several times!

And eventually, this system response:

Announcement from the SDE

I would like to apologize for the lack of SDE responses on the forum. All of our resources are currently focused correcting and verifying the A-F report card and fulfilling all data verification requests. We received the final testing data from the vendor on Oct. 15, and have been working non-stop since then to perform and verify the calculations. Please refer to the homepage for updates regarding the progress of this process, but we expect all known issues to be resolved in the afternoon of Oct.17th.

In light of these challenges, the deadline for submitting data verification requests for calculations has been extended until 10:00 A.M. Oct. 28th.

We completely understand and empathize the frustration that many have experienceed within the last several days, and we please ask for your patience and assistance as we work together to ensure the calculations are accurate when the report cards are released on Oct. 29.Sincerely,
Michael Tamborski

This is a complete embarrassment, and one that the SDE can hardly afford. Their PR is bad enough without the added stress of self-inflicted wounds.

Accountability is the stated commitment of Superintendent Barresi. She needs to hold her staff to a higher standard than this.

*As of 4:00 p.m. 10/17/13

Stuck in the Middle with A-F Report Cards

Let’s all sit around this Stealer’s Wheel (link embedded if you’re not of a certain age) and discuss the “clowns to the left” of us and “jokers to the right,” that have come to us in the form of the people responsible for our letter grades. It has taken quite a process for us to reach this point. Let’s review.

In 2012, the first A-F grades were set to be released by the State Department of Education, then tabled by the State Board. An OCU researcher showed that the majority of difference in grades could be attributed to socioeconomic factors. Then and OU/OSU study found that the grades lacked, among other things, something that researchers like to call statistical validity.

The legislature then decided to re-write the rules for the report cards. In fact, Oklahoma’s own Clark Jolley is set to present on our A-F Report Cards in all their glory today at Jeb Bush’s education summit. It’s such a big deal that he can be seen here making last minute preparations at Fenway Park last night.

Yeah. I'm in the press box at Fenway. #EIE13 pic.twitter.com/vv5mmRr5lE

— Sen. Clark Jolley (@ClarkJolley) October 16, 2013

Yesterday, when the grades were finally released, they looked a little high. Thirty minutes later, the SDE adjusted them, and they looked really low. Several schools received the explanation that the people plugging numbers into the formula had inadvertently mistaken the top quartile of last year’s test-takers for the bottom quartile. (Ironically, it was the math scores the y had miscalculated.) And they had fixed the scores. And the grades were final.

The emails actually told administrators that the grades were final.

Then last night, I started receiving messages from people telling me that the grades had changed again. Now, they were somewhere in the middle of the first two iterations.

It takes months to calculate the grades, but only minutes to re-calculate them? And then a few hours to do it again – this time with no new explanation?

If anyone has ever doubted the idea that the biggest problem with A-F Report Cards is how easy they are to manipulate, this should be the day they stop. As Rob Miller says, the report cards are DOA.

I don’t know what we’ll find today. It’s possible that by the time I post this to my blog, the grades will have changed again. I doubt anybody will ask Senator Jolley about this at his presentation today. It doesn’t fit the narrative that Jeb and his merry band of reformers have brought accountability to a place where none previously existed.

What I do know is that superintendents, principals, and teachers have their heads on a swivel. The lack of usefulness in these grades should do nothing to detract from the fact that hard working educators all over the state do their best for kids every day.

That’s my narrative. Now. In 30 minutes. And into the night.