Reason #7 to Pick a New State Superintendent: PASSing Around Our Standards

Ten days from now, Oklahoma voters will go to the Oklahoma polls to utilize Oklahoma voting technology and choose the Oklahoma candidate who best represents their Oklahoma values. If that seems to be a little bit over the top, it’s because I want to make it clear that this blog is not the part of some out-of-state entity, lurking in the shadows, trying to usurp our schools. I am, as the About page of my blog states, “a long time Oklahoma educator who thinks the false narrative about failing public schools needs to be refuted.”

The meandering path we have taken these last four years has left our schools in chaos this summer. Barresi’s leadership is a big part of why that road is anything but a straight line. Cumulatively, it is worthy of a spot in the Top 10 in this countdown.

#10 – Ignoring Researchers

#9 – The A-F Rollout

#8 – The 2014 Writing Test Debacle

#7 – PASSing Around Our Standards

Before discussing where we are now, let’s look at how we got here. As a reference, I present a timeline straight from the public relations campaign the SDE began last summer, The Road Ahead.

A Timeline of Academic Standards in Oklahoma1983 – President Ronald Reagan’s National Commission on Excellence in Education releases report “A Nation at Risk”, documenting the need for education reform in the United States. The development of new standards begins. 1996 – A coalition of Nation’s governors and corporate leaders form Achieve, Inc., a bi-partisan organization to raise academic standards and graduation requirements. 2005 – Achieve, Inc. launches the American Diploma Project Network to align standards and graduation requirements to college and career readiness. Concept of the Common Core begins. 2005 – 2006 – Oklahoma joins the American Diploma Project Network. 2009 – Oklahoma joins other states in the Common Core State Standards Initiative, a state-led process to develop more rigorous, higher, and clearer academic standards. 2010 – Three Oklahomans selected for writing committees to draft PreK-12 standards in English Language Arts and Mathematics. June 24, 2010 – State Board of Education adopts Common Core State Standards in English Language Arts (ELA) and Mathematics for the Oklahoma Academic Standards. July 6, 2010 – Governor approves adoption. 2010 – Transition to new standards begins. OSDE leads teacher development, local curriculum revision, and test development. 2011 – 2014 – OSDE provides ongoing assistance to districts for implementation of the Oklahoma Academic Standards. 2012 – State Board of Education adopts revised Oklahoma Academic Standards forSocial Studies and History, written by Oklahoma educators and content experts. 2012 – 2013 – OSDE leads revision process for Oklahoma Science Standards, written by Oklahoma educators and content experts. 2013 – OSDE launches For the Road Ahead family and community engagement initiative. Spring 2014 – For the final year, state assessments reflect the Priority Academic Student Skills (PASS). June 2014 – Transition to new Oklahoma Academic Standards complete. Spring 2015 – New state assessment administered, reflects the Oklahoma Academic Standards.

I don’t know how much they paid for the PR effort, but it came with its own logo. Maybe it’s one of those crazy-high expenses Rob Miller discovered for us last week.*

That’s a pretty selective summary of how we got where we are (minus the 2013 developments). Yes, three whole Oklahomans were on the committee that wrote the Common Core. Three! But prior to that – hey, wait. Something’s missing. Apparently, nothing happened between 1983 and 1995 regarding standards in Oklahoma.

Nothing except for PASS, that is. Yes, the state’s promotional materials left off the standards that arose out of HB 1017 in 1990 – standards that were written by hundreds of Oklahomans! For more than two decades, these were the state standards. Each subject area under PASS has seen multiple revisions, but the title of the overall document has remained the same.

Under Barresi, all academic standards were rebranded as C³ standards in 2011. You can still see it in the logo above. Then in 2013 came OAS – the Oklahoma Academic Standards. It got even more amusing when OAS for science, or OASS as we’ve come to know it, came into being.

The SDE under Janet Barresi is serious about the business of rebranding. For 20 years, Sandy Garrett had a summer conference called Leadership. In 2011, it was rebranded as Innovation. In 2012 it became Vision 2020. It has grown from a two day conference with chocolate fountains into a four-day extravaganza with an assortment of expensive keynote speakers. It is vendor-palooza, which is fairly important now that public education is in constant chaos.

We’ve also rebranded our tests. In 2013, the SDE changed the name of the Oklahoma Core Curriculum Tests (OCCT) to the Oklahoma College and Career Readiness Assessments (OCCRA) – which led to the realization that nobody copy-checks acronyms up there. Also, it spawned this image (have I mentioned how excited I am to have @FakeOKSDE back in the conversation here with us?.

In the meantime, we also dabbled in the PARCC consortium, until abruptly pulling out last summer. I digress – back to the standards.

The biggest problem we’ve seen regarding the standards (and similarly, to testing) is that we’re more interested in image and substance. Are the Common Core State Standards any good? That’s not the relevant question. What do people think of them? That’s what really matters. Last summer, when momentum was building across the country to dump them before full implementation, the SDE pushed us to accept OAS, but here was how they sold it to us.

What are the Oklahoma Academic Standards? OAS… are custom-built for PreK-12 students in Oklahoma prepare students with skills and knowledge needed to succeed in a rapidly changing world provide a route for partners in education to readily follow with fewer, higher, and clearer standards include assessment stops along the way so students get additional help needed to achieve peak performance put teachers back in the driver’s seat to make instructional decisions that set graduates on the road to being college, career, and citizen-ready

I do think the SDE and I have different operational definitions for custom-built. Aside from that, who can argue with these bullet points? We absolutely want to prepare our students for a rapidly changing world.  We want to allow more room for our partners to be … wait, partners? Who are these partners? Is this like how every vendor comes into your school and swears they want to partner with you on your school improvement efforts? Is that why so many companies flock to Vision 2020? Are they looking for fresh meat? The fourth bullet highlights the SDE’s mindset on testing (in spite of what Barresi said in her debate against Hofmeister Thursday night). The fifth is a farce. Nothing about the way the SDE has operated since 2011 indicates that the top leadership there trusts teachers to do anything.

The passage of HB 3399, which overturned the Common Core State Standards, has set off a frenzy of summer activity around Oklahoma. Right now (well, hopefully not at 8:00 p.m. on a Saturday night), teachers and administrators are working to retrofit the work they’ve done over the last four years into PASS. They can’t simply back out. Whereas under PASS prior to 2010, a specific math skill might have been located in one grade, and under CCSS, it is in another, simply switching back would leave gaps in the curriculum. No, this switch back will take considerably more finesse than what Janet Barresi and Mary Failin think.

And why rush? In 2016, we will have yet another set of standards. Every candidate for state superintendent guarantees that they will not in any way under any circumstances resemble the Common Core. They are all going to load up a room with an assortment of people from all over the state and not emerge until new standards are written. It will be interesting to see if the phrase Oklahoma values means the same thing everywhere. Or rigor. Or even a phrase like critical thinking.

In the meantime, we have PASS. Barresi says now that these standards are fine. That’s definitely not how she felt in October.

That’s why I’m excited about the new Oklahoma College and Career Ready Assessments being planned for students for the 2014-15 school year. They move students away from the fill-in-the-bubble, rote memorization tests that now exist. Instead, these performance-based exams include strategies to promote critical thinking and problem solving as well as practical application of securely held foundational knowledge.

I know a lot of people who supported the Common Core. I also know a lot who fought against it. Most of the people I know in both camps are angry at the double-speak we’ve seen from Barresi. When it comes to education, it’s all about the façade. Nothing about her or the reforms she pushes helps children. The fight now is to get people who only marginally follow educational issues to see it.

We have 10 days, Oklahoma. Get it done.

 

*Actually, The Road Ahead was funded by the GE Foundation – yet another out-of-state entity.

On Science, Executive Orders, and Plagarism

We in the blogosphere kid Janet Barresi and Mary Fallin about their wild declarations sometimes. They get really excited when they’re letting us know that Oklahoma is Oklahoma and nobody is going to tell us what to do.

The problem is that there’s not an original idea between them. Somebody is always telling us what to do, and they’re letting it happen.

I posted late last week that the Oklahoma State Department of Education is asking for comments from Oklahomans like you on the new OASS (Oklahoma Academic Standards for Science). They want you to know believe that the standards were written by Oklahoma educators.

They weren’t. They were lifted, practically verbatim, from the Next Generation Science Standards. As Jenni White and Rob Miller point out in their analyses (which are eerily similar), reading the OASS side by side with the NGSS shows very little difference. Essentially, Oklahoma has removed references to evolution and climate change. That’s it. While both White and Miller (and I for that matter) like the structure and organization of the standards, we all deserve to be told the truth.

Of course, for six months, the SDE has been passing off the Common Core State Standards for English/Language Arts and Mathematics as the Oklahoma Academic Standards. As with the NGSS, the CCSS were developed by Achieve, Inc. That is the group driving standards, curriculum, and assessment in Oklahoma.

Naturally, because leaders of conservative states like ours pretend to believe in local control, they want to assert the state’s supremacy. That’s why Fallin issued an executive order declaring that those interloping Feds better keep their interloping hands off of our schools – except for special ed and Title I money of course. We need to be mad at the Feds for cutting that!

There are two reasons why her executive order makes me laugh. First is that the US Department of Education may have incentives for adopting CCSS and other poorly researched reforms (VAM, anyone?), but they are not the author, merchant, and carnival barker for them. That responsibility has fallen to the Chief State School Officers – especially the nine who are members of Jeb Bush’s Chiefs for Change. The second is that Fallin didn’t even write the executive order. It’s nearly identical to the one released in October by Iowa’s governor, and it’s nearly identical to the one released this week by Mississippi’s governor. The anti-Fed position is a red herring intended to disguise the fact that these governors are actually embracing the Common Core.

We shouldn’t be fooled into thinking that Oklahoma writes our own standards or executive orders when we don’t even write our own laws. We also shouldn’t believe that the tests that we’re going to pay Measured Progress to write for us are anything terribly different than what PARCC would have written before we pulled out of their tests (while trying to remain on their governing board). As the Request for Proposals issued by the SDE made clear, the tests will be written to PARCC specifications. And they shall be called OCCRA.

Now for the punchline: last week, the SDE released sample responses to the current fifth and eighth grade writing tests. Of note is how the instructions indicate scorers should deal with responses that do not fully cite their sources. Fifth graders will not have to use quotation marks or reference the title and author of sources. Eighth graders will not have to reference the title and author (which I suppose means that they will have to use quotation marks).

Using this as a reference point, I think we can say that our entire state government is performing pretty well when held to a fifth-grade standard. Another way to say this is that Oklahoma fifth graders who pass the state writing test are pretty much ready to be in charge of this state. Maybe this is why I tend to have so many citations in my posts.

For further reading, please see the following:

Next Generation Science Standards

Oklahoma Academic Standards for Science

Iowa Governor Terry Branstad’s Executive Order

Oklahoma Governor Mary Fallin’s Executive Order

Mississippi Governor Phil Bryant’s Executive Order

SDE Memo on Citing Evidence on State Tests

But will it fit through the straw this time?

The state that loves change for its own sake and the profession that loves acronyms has a new treat for you. Because our SDE has decided to vacate the time, effort, and money sunk into the development of the PARCC exam, we had to come up with our own new thing.

First, let’s flash back about 10 weeks. This was Superintendent Barresi’s message about why we were pulling out of PARCC:

If we move ahead with this, we are going to be asking the state to drink a milkshake using a cocktail straw,” Barresi said. “If you look at what happened with testing this year — kids getting screen frozen, knocked off the test — those were technical issues that were from the districts’ end of things. (The testing vendor) crashed for two days because of server problems, but almost every bit of the rest of it was due to district issues. I’m not pointing fingers, but it is the reality.”

The SDE let the world know that Oklahoma would not administer the PARCC tests, but rather would pay a vendor to develop and administer a test with PARCC-like items. Now, you can go to the state’s Central Purchasing website and view the Request for Proposals (RFP) for Elementary & Secondary School Student Testing.

The first 16+ pages of the RFP are standard legal requirements. The meat of the document begins at the bottom of page 17:

C.1  The purpose of this RFP is to solicit proposals for activities related to the implementation of a statewide summative assessment. The assessment will be aligned to the Grades 3-8 Oklahoma Academic Standards for administrations in the spring of 2015 with additional one year optional renewals for the administrations in the spring of 2016, 2017, and 2018. Henceforth in this document, this test shall be called the Oklahoma College and Career Readiness Assessment (OCCRA). Development and field testing of custom-created items will occur during the 2013-14 school year.

There’s your new acronym, folks. OCCRA. Get ready immediately for field-testing. Keep in mind this is only the proposal to award a testing contract for 3^rd through 8^th grades. The End-of-Instruction (EOI) exams are separate.

Other parts of the proposal got my attention as well:

C. 2  This RFP will specify a complete package of products and services so that the OCCRA will be administered, scored, and reported separately from other state assessments. The OCCRA will be administered via paper and pencil to students in Grade 3-5. The OCCRA will be administered online with paper and pencil options to students in Grades 6-8.

This is the same as what we’re doing now.

C.4.3  Although the state testing dates for spring 2015 have not yet been officially determined, the projected dates are shown in Table 1 below. The final determination of the test administration dates for the OCCRA will be based on statutory requirements for test administration and reporting. Currently, no summative assessment may take place earlier than April 10 of each year. Immediate raw scores are required for all assessments administered online. Complete preliminary reports including the raw score, performance level and scale score must be provided no later than two weeks following the close of the testing window. When standard setting is needed the two week reports do not include performance levels.

Proposed Test Administration Dates
ACTIVITY	2015
Oklahoma Core Curriculum Test Administration (Grades 3-8)	April 13-May 3
OCCRA Administration	April 13-May 10

This section strikes me on a couple of levels. First is the expectation that schools will receive scores no later than two weeks after the end of testing. That’s optimistic, based on what we’ve seen the past couple of years (from two different vendors). As nice as it would be to have raw score reports in a timely manner, without the performance levels, they aren’t that meaningful. Schools like to run remediation programs in the summer and populate those with students who scored poorly the year before. Lacking this information complicates the process of communicating the need to parents. Second is the realization that OCCRA is just a testing program for reading and math. The SDE apparently plans to continue administering the OCCTs for other subjects. The following section of the RFP backs this conclusion:

C.6.1  The supplier(s) will provide the State of Oklahoma with a criterion-referenced achievement test measuring mastery of the Common Core State Standards in English Language Arts and Mathematics proficiency for administration to students in Grades 3–8 in all Oklahoma public schools. The student achievement scores must be interpretable in terms of the criterion and must be valid and reliable for use in student promotion, retention, and graduation decisions. The scores must also be valid and reliable for use in teacher, school, and district accountability measures. This RFP does not require nor expect the Supplier to create an entirely custom test but, instead, assumes the purchase of products and services derived from existing products published by the supplier(s). The proposed OCCRA must meet the design shown in Tables 3 and 4.

If you’ll look closely at the proposal, it doesn’t seem to contain anything labeled Table 3 or Table 4. Never fear – here they are in a separate document. They show that the selected vendor will report student data in terms of the strands of the Common Core. For anyone thinking that the SDE can take cover from the national criticism of the Common Core by hiding it under the newly branded Oklahoma Academic Standards, this is a rather prominent inclusion.

C.8.4  Throughout the development and implementation of the OCCRA, the supplier is required to protect student privacy.

It is important to note that no such requirement exists for the SDE, apparently.

C.8.5  Reading passages should expose students to grade-level text of appropriate complexity. Texts should reflect a balance of 50 percent authentic literature (stories, drama, and poetry) and 50 percent informational text/literary nonfiction which would include passages in grades 6-8 that address the literacy standards requirement for history/social studies, science, and technical subjects that students are expected to read. Selections should include a combination of single texts and paired passages of sufficient complexity for close, analytical reading which would allow students to discern and locate evidence from the text to support assertions in multiple-choice items or writing responses.

This is a bone of contention in the Common Core with many teachers. The 50/50 literature/informational text split is proscribed for all grades. If we were talking about high school students, there might be less frustration over this. With third grade students, who will have to pass this new test to move on to fourth grade, this is a huge transition. While schools have had three years to start the transition to Common Core, they haven’t had this split in mind.

C.8.6  English/language arts items aligning to each passage should include single and/or two part multiple choice items as well as short, constructed responses to text-dependent questions. One extended response item should be drawn from a paired or multiple-text passage set. Extended writing prompts should allow students to produce opinion/ arguments, informative/explanatory responses, and narratives. Grade 3 students should be able to generate a summary as well.

While the writing tests for fifth and eighth grade will now be separate, all grades will have to show proficiency by writing. And who will score all of these student-written responses? We’ll get to that in a bit. I’m going in order.

C.12.3  The proposal must include details as specified below about its computer-based testing delivery system. The proposal must list a minimum of three customers, within the preceding twenty-four months, utilizing the base platform proposed, or list a minimum of three customers utilizing fully customized platforms within the preceding twenty-four months. Their names, titles, email addresses, and phone numbers must be included in the proposal. The supplier must be prepared to demonstrate significant portions of the proposed system during the proposal review meeting.

This is pretty self-explanatory. The proposal continues talking about technology specifications and requirements of the vendor, including:

C.13.1  Support all eligible devices that meet the specifications in the Table 6. The supplier must also commit to continued support for future updates made available on the latest version of Partnership for Assessment of Readiness for College and Careers’ (PARCC) Technology Guidelines that can be found at (http://www.PARCConline.org/technology).Other assistive technologies may be needed for students requiring accommodations.

So we’re still not entirely pulling away from PARCC. We just love them for their technology specs.

C.30.1  The supplier shall describe the online accommodations that are available for administration of the test to students with disabilities and to students with limited English proficiency (LEP). Suppliers should refer to the existing Oklahoma policies in this regard which can be found in Appendix B of this RFP. The proposal shall confirm that the test can be administered according to these policies. If there are variations of accommodations to be proposed by the supplier, this should be made clear in the proposal.

C.30.5  Describe how the needs of students with disabilities and limited English proficiency were included in the early development of the proposed test items and the extent to which such students were included in field tests and other related studies.

Together, these two sections give us our only glimpse into the treatment of special education students by OCCRA. Accommodations will be made. I’m glad they were specific.

C.36.2 The SDE requires that written responses be read and scored by two sources with at least one of those sources being a human reader. This may include a machine score and a human score. If using a machine score the supplier shall address appropriate procedures to ensure validity and reliability.

E.18.1.3  All multiple-choice items will be machine-scored/computer-scored, and short constructed response items can be hand scored by humans or by Artificial Intelligence (AI). Reporting of scores on these items will be based on the procedures specified by the SDE and will include results at the domain level. The supplier will hand score any documents containing student responses upon SDE request to verify student scores.

Now we’re there. We casually enter the world of machine-scored essays and AI-scored constructed response items. It comes not as a thump at your door, but in a sub-clause of a sub-clause of a business document.

E.20.1  The supplier will conduct standard settings for each content and all grade levels following the Spring 2015 test administration. The SDE will approve the method for setting standards. The supplier will provide psychometricians and content area specialists to facilitate the standard setting meetings. The procedures shall include creating  descriptors of what students can do at each of five performance levels. The performance level descriptors are to be unique to each content area. The supplier shall supply evidence of successful standard settings for large-scale assessment programs. The supplier will provide psychometricians and to assist with the information needed for the presentation of cut scores at state board meetings.

We all know how much input educators had this year in standard setting. Their recommendations largely went ignored.

That’s the heft of the RFP. I do want to turn back to page 10 for a moment – back in the legal mumbo jumbo. I did a keyword search for penalty, and I found this:

B.3. Penalty Clause

B.3.1. Once completion dates are mutually agreed upon, and the supplier fails to perform any of the services and/or make deliveries within the time Specified in the contract, or any extension period, the supplier shall in place of actual damages, pay the State of Oklahoma the sum of $15,000 per Calendar day for delayed services/deliveries. The supplier shall NOT be charged when delay in performance and or delivery arises out of causes beyond the control and without fault or negligence of the supplier.

B.3.2. Failure to correct any errors in materials or prevent disruptions that negatively alter the ability to administer the test, to use the score reports, or that denigrates confidence in the testing program will be viewed as a violation of the contract and the will pay liquidated damages to the SDE in the amount of 3% of the total annual contract amount for each day during which the online, electronic, paper, print or other document is incorrect until a corrected online, electronic, paper, print or other document approved by the SDE is distributed by the supplier. SDE agrees that when liquidated damages become a possibility, it will expedite its responses and requirements in this regard in an effort to limit the amount of liquidated damages.

I guess we’ve learned at least one thing.

***

By the way, if you haven’t read Rob Miller’s analysis of how OCCRA will impact his middle school, I encourage you to give it a look.