Are the SATs Biased Against Blacks?

[IanC]

Now, in the latest issue of the Harvard Educational Review, the two scholars who took on that project have published a paper saying Freedle was right about a flaw in the SAT, even in its current form. They say "the SAT, a high-stakes test with significant consequences for the educational opportunities available to young people in the United States, favors one ethnic group over another."

"The confirmation of unfair test results throws into question the validity of the test and, consequently, all decisions based on its results," said Maria Veronica Santelices, now at the Pontificia Universidad Catolica de Chile in Santiago, and Mark Wilson of UC Berkeley. "All admissions decisions based exclusively or predominantly on SAT performance--and therefore access to higher education institutions and subsequent job placement and professional success--appear to be biased against the African American minority group and could be exposed to legal challenge."

Class Struggle - New evidence that SAT hurts blacks

this type of study and the media articles that it spawned are the reason why so many people believe the nonsense about cognitive tests being biased against blacks. as usual, the weakest bit of evidence is given unwarranted weight while the massive amounts of evidence pointing in the other direction are easily dismissed by those who are only interested in promoting a political agenda rather than finding the truth.

Why am I mocking this evidence? because it only involves 2 questions! not only that but the two questions are counted twice because the two years studied both had these two questions. not only that but those same two questions were ignored in different years when they did not show the desired effect! "All admissions decisions based exclusively or predominantly on SAT performance--and therefore access to higher education institutions and subsequent job placement and professional success--appear to be biased against the African American minority group and could be exposed to legal challenge." just because two questions sometimes showed a DIF and sometimes didn't.

the easy explanation to this difference is that the two questions were quite difficult and answered incorrectly by most students and the DIF was showing mostly in weaker students that were likely guessing. random guessing should produce the right answer 20% of the time. if there is a particularly attractive distractor(plausible wrong answer) the actual correct response rate can be less than 20%.

of course we could easily wipe out the racial gap. either make the questions so easy that everyone scores 100% or conversely, so hard that eveyone scores zero.

 

[Tank]

Asians score highest on the SATs.

Let the African Americans develop a SAT test, I bet they would still fail it.

 

[topspin]

yes

 

[uscitizen]

Did the pope support child abuse?

 

[manu1959]

yes...intelligence is racist.....

 

[IanC]

yes

That is a typical response with the usual amount of evidence to back it up. At least you were concise.

 

[IanC]

Asians score highest on the SATs.

Let the African Americans develop a SAT test, I bet they would still fail it.

I believe there are many af-ams involved with designing and testing the SATs. Freedle from the OP is one of them. His disingenuous study is all the more appalling because he had the evidence available to him to debunk his theory and chose not to use it.

 

[manifold]

I don't believe that the SATs are racially biased.

But clearly they are biased against the unintelligent.

 

[IanC]

yes...intelligence is racist.....

Mother Nature does appear to be racist but I am sure she has her reasons.

 

[IanC]

I don't believe that the SATs are racially biased.

But clearly they are biased against the unintelligent.

Indeed. Not unlike the way a stopwatch is biased against the slow.

 

[Ravi]

because it only involves 2 questions!

I didn't see anything in the article that said the study was based on two questions. Please quote it...

 

[kwc57]

I don't believe that the SATs are racially biased.

But clearly they are biased against the unintelligent.

Whaaaaaaaaa..................

View attachment 10725

 

[IanC]

Legal challenges to college admissions decisions! Could anything be more dire? As a matter of fact, things could be much more dire because this report is nothing more than junk science masquerading as scholarship.
A quick read of the Santelices/Wilson report shows quite a number of things that somehow escaped Mathews. It will cost you ten bucks to read the thing but if you’re really serious about looking for a case of badly done research, it’s money well spent.

For starters, the research examined four separate SAT tests, two of which were administered in 1994 and two of which were administered in 1999.

Those dates, by the way, are not typos. The tests really do date back to the Clinton Administration.



An examination of the performance of different groups of students on individual questions – something researchers refer to as differential item functioning or DIF – showed that in three of the four tests included in the study, there was no significant DIF between white students and minority students. But on one of the tests from 1999, researchers found a significant DIF on two questions.

Again, not a typo. Two questions.

Another salient fact that did not occur to Mr. Mathews is that the 1999 test with a significant DIF on two questions included the exact same questions as the second 1999 test that had no significant DIF – the questions were just presented in a different order. Rather than an indictment of the SAT, we have a statistical quirk!

But wait, there’s more.

Millions of kids across the country take the SAT every year as they begin their own personal Paper Chase in earnest. But this study didn’t look at that. It looked only at students from California and even then, it included only kids who were accepted and admitted to the University of California system. Wouldn’t a larger sample be needed to draw any meaningful conclusions?

We’re still not finished here –

The four tests included in the Santelices/Wilson study don’t exist anymore. That’s because the SAT went through a major overhaul in 2005 when it added a writing section and excised analogy questions.

Junk Reporting – the Hidden Truth Behind Claims Of Racial Bias In the SATs - Big Journalism

good enough?

 

[Ravi]

No. The original link said nothing about only two questions. It said that blacks scored slightly higher on hard questions and lower on easy questions.

You're going to have to do better than link to some idiot on an idiotic blog.

 

[Ravi]

And for the record I don't think the test is biased. But I do think the education system favors the Majority...which causes the minority to struggle with these STANDARDIZED tests.

 

[IanC]

No. The original link said nothing about only two questions. It said that blacks scored slightly higher on hard questions and lower on easy questions.

You're going to have to do better than link to some idiot on an idiotic blog.

pay the ten dollars and you can read the idiots who performed the idiotic study.

I looked for the College Board rebuttal of the original 2003 study but I could only find a ghost from #2 Pencil--

Roy O. Freedle's recent article in Harvard Educational Review, entitled "Correcting the SAT's Ethnic and Social-Class Bias: A Method for Reestimating SAT Scores," is based on small differences between white students' responses and the responses of students from other ethnic groups to test items that were discussed by a number of researchers...Although any study that purports to reduce group differences must be looked at seriously, Freedle's study is so flawed that its conclusions are misleading.

There are myriad technical problems with the report, including misuse of regression and differential item functioning (DIF), and even a misunderstanding of how scores on the SAT are calculated. But one need not be a psychometrician to understand the fundamental problem with the study. The reduction in group differences is not the result of more sensitive or appropriate measurement, but rather, it is because the proposed measure relies mostly on students' guessing the answers to test questions.

To probe a little deeper, let us examine more closely Freedle's argument around DIF. Researchers have found that, on average, African-American, Hispanic, and Asian-American students tend to choose the correct response on easy test questions slightly less often than white students with an equal total test score. In contrast, they choose the correct response on difficult test questions slightly more often than white students with an equal total test score. Noting that this phenomenon occurs with SAT vocabulary questions but not with critical reading questions, Freedle suggests that the College Board should dispense with SAT critical reading questions, as well as the easier half of all vocabulary questions to improve the scores of ethnic minority test-takers.

Te suggestion that critical reading be dropped or de-emphasized on the SAT, given its importance for success in college, would not be educationally or psychometrically sound even if it were based on a credible analysis..Freedle himself notes that the critical reading items lack what he calls "the familiar pattern of bias."

To summarize so far - Mr. Freedle is suggesting dropping items that show no bias, according to his own results. The College Board alleges that he doesn't even correctly grasp the scoring method of the SAT, much less calculate DIF in the proper fashion. Doesn't look good for Mr. Freedle, does it?
Let us look briefly at the data for the so-called SAT-R Section that Freedle recommends. On the difficult items that are included in the SAT-R, African-American candidates receive an average score of 22 percent out of a perfect score of 100 percent. Since there are five answer options for each question, 22 percent is only slightly above what would be expected from random guessing, namely 20 percent. White candidates do somewhat better, achieving an average score of 31 percent. [I'm assuming this gap is smaller than for the SAT overall.] The results indicate that this test is too hard for either group and would be a frustrating experience for most students. There are simply too many questions that are geared to those with a much higher level of knowledge and skill than is required of college freshmen. Extending Freedle's argument, we could substantially reduce all group differences if the test were made significantly more difficult so that all examinees would have to guess the answers to nearly all of the questions. We could then predict that each subgroup would have an average of 20 percent of their answers correct, based on chance...

In brief, Freedle's suggestions boils down to capitalizing on chance performance. This kind of performance may represent either random guesses, or unconnected bits of knowledge that are not sufficiently organized to be of any use in college studies.

Very interesting. I hadn't even considered the guessing argument, but then, I wasn't aware of just how difficult the difficult items were. The College Board is claiming that the proposed revised SAT would not be a true measure of anyone's ability, because it would be so difficult a test that most test takers would be guessing the answers. If black students at high ability levels guess better than white students, that is most certainly not a valid measure of ability.

As the College Board puts it, "Freedle's suggestions boils down to capitalizing on chance performance." For those of you not in the field of psychometric research, the statement that one is "capitalizing on chance" is synonymous with saying, "You started with the end result in mind, and now you're trying to prove that the data show more than they actually do, and if you collect another set of data, you'll get a different answer, because your results aren't going to generalize." It's an important and fundamental criticism to make against a research study.

The rebuttal also emphatically denies that the mathematics questions measure any sort of secondary vocabulary dimension, which removes any justification whatsoever for creating a revised SAT for difficult math items. Overall, the rebuttal feels pretty definitive to me, but it won't surprise me if reporters pick up on Mr. Freedle's article without mentioning the rebuttal. The buzzwords of "racial bias" and "SAT" will be just too tempting for some to ignore, and chances are they won't look further to assess the validity of Mr. Freedle's claims. (emphasis added)

 

[Ravi]

I might pay $10 but the ball is in your court to produce the evidence to back up your claim that the study was based on two questions.

 

[eagleseven]

Maybe the SAT is biased against those who attended horrible schools?

Perhaps if we stopped asking racist questions, we would stop being a racist society?

 

[Mad Scientist]

If:
White 2+2=4 and Asian 2+2=4,
Then:
Black 2+2=5?

Amidoinitrite?

 

[Tank]

Perhaps if we stopped asking racist questions, we would stop being a racist society?

Do you not like the question, or the answer?