MDE: Testing company misgraded nearly 1,000 exams

[Disir]

A testing giant could be out millions of dollars and an unknown number of Mississippi seniors may have been awarded or denied diplomas in error, after nearly 1,000 exams were misgraded by NCS Pearson.

The state Board of Education moved to end the contract, issuing a one-year emergency procurement with Questar to administer the U.S. history, biology I and fifth- and eighth-grade science assessments for the 2017-18 school year.

Since 2009, Mississippi has spent more than $28 million with NCS Pearson.

More: Miss. accountability plan targets achievement gaps in state schools

State education officials said Friday that the error by the test-making vendor affected 951 U.S. history assessments.

Paula Vanderford, MDE director of research and development, said the test-making vendor used an incorrect conversion table when grading the assessments.

“By them using the incorrect table, it gave some students a slightly higher score than they should have received, others a slightly lower score than they should have received,” Vanderford said.

She said MDE is working to determine the scope of how many students might have been given an incorrect failing or passing grade.

At issue is the fact the incorrectly scored exams were all taken by seniors.

Approximately 27,000 students in the state took the exam during the 2016-17 school year, which is typically administered in the spring semester of one’s junior year.

Vanderford explained that students who failed the test during their junior year have an opportunity to retest as seniors. Students electing for an early exit from high school might also take the exam outside their junior year. Because of this, MDE requested scores be released by May 12 for students falling in either category in order to verify whether they met graduation requirements.

MDE: Testing company misgraded nearly 1,000 exams

It's just money, amiright?

 

[usmbguest5318]

RE:/From "Testing company misgraded nearly 1,000 exams":

What I would conclude about the Mississippi (MS) School Board terminating it's contract with the testing company depends on materiality and what be the marginal impact of the mistakes.

Sure, ~1000 affected students/tests seems like a "a lot" but we're talking about a whole state. The article notes that the error affected 981 individuals' history assessment -- that's not the same thing as affecting the students' scores-- and that's out of some 27K students to took the test. That means that 3% of the tested students may have earned a higher or lower score, but it remains unclear is for how many of those students their "corrected" performance score will alter their assessment. That's germane because apparently the test was scored on a "pass-fail" basis rather than a "specific grade" basis....

[Paula Vanderford, MDE director of research and development,] said MDE is working to determine the scope of how many students might have been given an incorrect failing or passing grade.

...Thus it doesn't matter how highly one scores on the test. Passing with the lowest possible passing score is just as good as earning a perfect score.​

To illustrate what I mean by the above.....In school I took a test or two whereby the teacher incorrectly marked correct or incorrect one of my answers. I even had teachers deduct on papers because they felt an idea I'd presented was in some regard amiss. In none of those instances (none were "pass-fail" assignments; I never until grad school took a class or had assignments whereupon I was graded that way) did the adjustment for which I argued and obtained alter my grade, mainly because I wasn't a student whose performance was marginal. (I.e., my course completions grades were "solid" -- 97+ -- rather than being 93 -- the lowest possible "A" grade one could earn in my school -- or 94.)

That said, absolutely, the students whose performance was on the margin and who thus should have received a passing or failing (as the case may be) "scale score," even if "just barely passing/failing," should have their scores adjusted so they receive the grade they truly earned. Grade inflation and deflation is of no real value to anyone.

As for the testing company, well, if this is a first or isolated instance of negligence in executing the grading process, I don't see the point of having terminated their contract. If, on the other hand, they have a pattern perfunctoriness, then, yes, they deserve to have lost their contract.

A “scale score is very important for use to a student in a concordance table or composite table,” Vanderford said.

[If you clicked on the link for "scale score," or you already know what that term means, you'll understand the following.]

I may be mistaken, but it seems to me that MS is ticked off more because of the impact the mistake has on the "scale scores," than because of its impact on the students' raw scores. Although I understand the mistake may lead to misrepresentation in the state's analysis of its students' performance, I think it kind of shameful that Vanderford explicitly cited scale scores' role rather than that of the raw scores, which Vanderford does not indicate were at all in error. After all, the kids care about their raw score as that is what determines whether they meet the minimum topical learning expectations set by the state. Scale scores are merely a tool that primarily facilitates analytical comparability among multiple years' test results. [1]

Vanderford's express mention of scale rather than raw scores alludes to her and the MS Board of Education's focus being less on how students perform, and more on how the state's education system (i.e, how she and her colleagues at or near the top of the school system) appears to parents, the governor, the state superintendent of schools and MS legislators. The Board's action re: the testing company smacks of one's taking drastic action in an effort to obfuscate the fact that, for the most part, errant 2017 scale scores or not, the schools in MS do a subpar job of educating students. Unless Vanderford is seriously going to try to make the case that the states low educational achievement levels are due to testing companies' having made myriad gross errors one year after the next and literally tens of thousands of students instead earned higher scores than they were given, she should never have mentioned scale scores.

Note:

1. If someone here is a member of a school board or something materially similar and can show me that I've misunderstood/misapplied what scale scores are and how they are used, by all means please do so. I've based the above remarks on what I understand scales scores to be based on the content at the link I provided and on my conceptual correlation of them to what in my profession we call "indexing." The CPI, which is a tool that allows one to compare dollars earned/spent among various periods in time, is one example of indexing.

From the article titled "Miss. accountability plan targets achievement gaps in state schools":

If the draft proposal outlined Friday by the state Department of Education goes according to plan, 70 percent of the state’s students will have reached proficiency in English and math by 2025.
With just over a third of students reaching proficiency in those subjects on state assessments, Mississippi will have to beat the odds — the number of students scoring above grade level in math and English would have to more than double in less than 10 years — to make that goal happen. But the state Board of Education is willing to take that shot.

Willing to take the shot? Well, of course it is. What is the alternative? Not try to increase the math and English acumen and proficiency rate among Mississippi's public school students?

​

 

[Disir]

'Cept this is not Pearson's first mistake. It's not even Pearson's first mistake in Mississippi.
Pearson's History of Testing Problems | FairTest