Gallup once again in Unicornland

[Statistikhengst]

Yepp, Gallup is at it again:

While Rasmussen, hardly a left-leaning pollster, is showing our President's approval at -2/-3, Gallup is showing it at -16.

It is statistically impossible that both of these values can be correct at the same time. Even if you stretch the traditional MoE +/-3.0 out to +/-6.0, accounting for both possible directions, that still cannot account for a 13-14 point difference.


Gallup, once a reputable national polling firm, is nothing more than a mathematical disgrace, and I can prove it beyond doubt.

In 2010, on the eve of mid-term elections, Gallup predicted this:​

Gallup.Com - 2010 Congressional Midterm Elections

(That was a generic vote R-D for the mid-terms)

Here's a screenshot, just for clarity:

That's right: Gallup predicted an R+15 win on election night. BTW, on the eve of the election Gallup only published the "lower voter turnout figures".

Actual result from 2010:

R +6


In the House:

United States House of Representatives elections, 2010 - Wikipedia, the free encyclopedia

R +6.8


In the Senate:

http://uselectionatlas.org/RESULTS/data.php?year=2010&datatype=national&def=1&f=1&off=3&elect=0

R +5.53


Average: R +6

15 - 6 = 9

Gallup was off by NINE FULL POINTS in 2010.

Fast forward to 2012:​

https://docs.google.com/spreadsheet...Q0U3aFBuT09zQ2xXQ29fTjlJRlE&usp=sharing#gid=1

Romney 49%, Obama 48% in Gallup's Final Election Survey


Gallup's final prediction:

Romney 50 / Obama 49 (Romney +1)

Actual result:

Obama 51.01 / Romney 47.15 (Obama +4)

Gallup was off by FIVE FULL POINTS in 2012.

But it gets much, much worse.

In the weeks leading up to election day, Gallup had Romney up as high as +7 over Obama, while all other pollsters were showing a +1 for either Romney of Obama. I recorded every single one of those numbers for posterity, they are all in the above excel googledocs linke to see. Plus, virtually every outlet was talking about this outlier results at the time:

(BTW, there were 749 national polls taken for 2012, I recorded all of them)

The Gallup from 10/29 was Romney +5
From 10/28, it was Romney +4
From 10/26 and 27, it was Romney +5, and so on.

When Hurricane Sandy hit, Gallup stopped it's 3-day-rollling-tracking polls. When it then produced it's final poll, the margin shifted 4 points to the left, but was still 5 points away from reality. That's just plain old sad.

On 10/29, Gallup was NINE FULL POINTS off the actual results about to happen. Does that sound familiar? Yepp, go look at 2010 again.

So, what happened?

Well, this happened: Gallup put out a "mea culpa" in June of 2013, 8 months after the election.

Gallup Explains How It Blew the Presidential Election - Businessweek

"This morning, the Gallup organization tried to explain why its polls of the 2012 presidential election turned out to be so wrong—Gallup’s final tracking poll indicated a 49 percent to 48 percent win for Mitt Romney, when in fact Barack Obama won by about 52-48. But the problem wasn’t just the final poll. Throughout the race, Gallup’s polls tended to be outliers that favored the Republican candidate."

The bolded: this is not correct. It was much closer to 51-47 than to 52-48, but the point spread would be the same.

More:

"Gallup examined 20 issues and concluded that four were chiefly responsible for its having overestimated Romney’s response. They were:

1. Its “likely voter” model was flawed. It relied more heavily than other polls on someone’s past voting behavior and “thought given to the election,” a variable meant to predict who would and who wouldn’t vote that turned out to be not so accurate.

2. It blew the geography. By not interviewing enough people in certain regions of the country, specifically “the Eastern time zone within the Midwestern and Southern regions” and the “Pacific time zone within the Western region” (according to the report), it got a distorted picture of the race.

3. It messed up race and ethnicity. Gallup forced people to answer a series of yes/no questions about their race and ethnicity that didn’t yield an accurate picture, and then assumed minorities would make up a smaller share of the vote than they actually did.

4. It used a shortcut that didn’t work. For years, pollsters have relied on “random-digit dialing” (RDD) to reach both listed and unlisted numbers. This is expensive but, many pollsters believe, necessary since 45 percent of numbers are unlisted. Some recent academic research has indicated that it might be possible to get an accurate view of the electorate by scrapping RDD and combining (much cheaper) published phone numbers and cell phone users. (Huffington Post’s polling editor, Mark Blumenthal, has a thorough explainer here.) Gallup attempted this, and it didn’t work..."

-------------------------------------------------------------------------------------


More than anything, Gallup used a model based on a projected 78% white voter in the mix. But that was bullshit to begin with. Already in 2008, it was only 74% white and the tendency has been shrinking in every single presidential election since such measurements began. We had an electorate that was 72% white on election day 2012, so Gallup, on this measurement alone, was off by 6 points. Disgraceful.

A couple of days after the election of 2012, I sent Gallup the following letter:

Statistikhengst's ELECTORAL POLITICS - 2013 and beyond: An Open Letter to Gallup:

----------------------------------------------------------------------------------------------

Now, shortly after the time that Gallup issued a belated apology for having fucked-up so badly, it was caught in a financial scandal:


Gallup to Pay $10.5 Million to Settle U.S. Contract Claim - Bloomberg

"Gallup Organization agreed to pay $10.5 million to settle U.S. claims that it inflated contract prices and engaged in prohibited employment negotiations with a government official, the Justice Department said.

Gallup knowingly overstated costs in contract proposals to the U.S. Mint and the State Department that were to be awarded without competition, according to U.S. allegations in a False Claims Act suit against the polling and market-research firm.

The company also improperly discussed employment with a then-Federal Emergency Management Agency official, Timothy Cannon, to obtain a FEMA subcontract at an inflated price, the government alleged.

“This case exposed a cozy arrangement between a contractor and government employee where nobody was looking out for the American taxpayer,” Ronald Machen Jr., U.S. Attorney for the District of Columbia, said today in a statement.

Alena Naff, a spokeswoman for Gallup, said in an e-mail, “By ending this civil action with no admission of wrongdoing, Gallup can avoid further distraction and focus on serving its customers.”"

More was also reported in POLITICO at that time:

Department of Justice slaps Gallup with $10.5M fine - Tal Kopan - POLITICO.com

"According to the original complaint filed in 2009, Gallup allegedly inflated estimates of labor hours required for market research contracts with the U.S. Mint and U.S. Passport Agency to drive up costs.The complaint alleges that Gallup had a $2 million-per-year contract with the Mint to “to conduct market research to identify likely purchasers of newly issued coins and how best to reach them.”

Gallup had a $2.7 million contract with the Passport Agency to conduct surveys to predict how a change in border laws would increase passport applications, according to the complaint."

As a result, GALLUP has lost all of it's Government contracts for polling. It's services will no longer be used.


---------------------------------------------------------------------------------------------


So, when you see Gallup trumpeting a presidential approval rating of -16, while Rasmussen is reporting -3, your far better bet is with Rasmussen, who himself was a very-right oriented pollster. Gallup is but a shadow of what it once was. What a shame.

 

[CrusaderFrank]

If you like your poll numbers you can keep them. Period

 

[Statistikhengst]

You have every right to your opinion. Tell me, do you every contribute anything factual in USMB, or is whining more your thing?

 

[AceRothstein]

The crazies here clung to Gallup in the run up to the election. I believe Gallup was +5 or +6 for Romney before Sandy and then stopped polling for a few days. Then they came back with the +1 Romney number right before the election. They obviously did something to their model to swing it that much in a week as none of the other polls showed a swing like that.

 

[Stephanie]

everyone should put this nasty piece of work on ignore...

You have every right to your opinion. Tell me, do you every contribute anything factual in USMB, or is whining more your thing?

Now they're whining about Gallup over their DEAR LEADER

cry us all river

 

[AceRothstein]

everyone should put this nasty piece of work on ignore...

You have every right to your opinion. Tell me, do you every contribute anything factual in USMB, or is whining more your thing?

Now they're whining about Gallup over their DEAR LEADER

cry us all river

You have every right to your opinion. Tell me, do you every contribute anything factual in USMB, or is whining more your thing?

 

[HelenaHandbag]

The first 12 seconds are the pertinent pull quote.

[ame=http://www.youtube.com/watch?v=DffJxH2j5ns]Back to School (11/12) Movie CLIP - Oral Exam (1986) HD - YouTube[/ame]

 

[Amelia]

All the recent polls at RCP except Rasmussen have greater disapproval than approval by double digits.

And Rasmussen's approval index is -16.

In other words, all the polls show that America is finally realizing what sludge Obama is, but you single out Gallup.

 

[Stephanie]

everyone should put this nasty piece of work on ignore...

You have every right to your opinion. Tell me, do you every contribute anything factual in USMB, or is whining more your thing?

Now they're whining about Gallup over their DEAR LEADER

cry us all river

You have every right to your opinion. Tell me, do you every contribute anything factual in USMB, or is whining more your thing?

More grown up talk?

 

[Stephanie]

What are you suppose to debate about this in the election forum?

it's so, zzzzzzzzzzzzzzzzz and whiny

 

[jillian]

[MENTION=46168]Statistikhengst[/MENTION] gallup wasn't very reliable during the last election cycle either. but you have to forgive the people who still think the polls are skewed

 

[Stephanie]

oh really, please forgive them for they don't know as much as all you snobs

 

[Zander]

I seem to recall a lot of liberals getting very upset with Rasmussen a while back. Now, Gallup is no good? Who's next? Marist? or gasp, PPP?

 

[Dont Taz Me Bro]

I seem to recall a lot of liberals getting very upset with Rasmussen a while back.

Yep, the left was constantly trashing Rasmussen prior to the last election. Now suddenly they are credible.

 

[Dont Taz Me Bro]

I did notice the sample size for Gallup was 1500, whereas it was 500 for the other two. Maybe that is accounting for the difference.

 

[Amelia]

I seem to recall a lot of liberals getting very upset with Rasmussen a while back.

Yep, the left was constantly trashing Rasmussen prior to the last election. Now suddenly they are credible.

Rasmussen's superficial number has been way more favorable to Obama than all the rest for months.

I've been watching this closely for a long time now -- posted a couple of threads about it a short while back.

When you click through and look at Rasmussen's approval index it lines up with the other polls, but basically it's been Rasmussen which has kept Obama treading water. If Rasmussen weren't in the mix, Obama would have drowned by now.

 

[Toro]

I'm part of the Gallup sample.

I fib to them all the time.

So you can blame me.

 

[AceRothstein]

I seem to recall a lot of liberals getting very upset with Rasmussen a while back. Now, Gallup is no good? Who's next? Marist? or gasp, PPP?

Both Gallup and Rasmussen did a terrible job over the last few elections. I'm not sure what to think about approval ratings as you have nothing to measure them against like there are for electoral polls.

 

[Statistikhengst]

The crazies here clung to Gallup in the run up to the election. I believe Gallup was +5 or +6 for Romney before Sandy and then stopped polling for a few days. Then they came back with the +1 Romney number right before the election. They obviously did something to their model to swing it that much in a week as none of the other polls showed a swing like that.

Yes, they stopped polling because of Hurricane Sandy, so did PPP (D) and a number of others, but the others started back to polling earlier than Gallup once the storm was over.

 

[Statistikhengst]

everyone should put this nasty piece of work on ignore...

You have every right to your opinion. Tell me, do you every contribute anything factual in USMB, or is whining more your thing?

Now they're whining about Gallup over their DEAR LEADER

cry us all river

I love it when Righties do this kind of stuff. Class bespeaks itself, and you just told the world a whole lot about Righties. And it doesn't look pretty, to be sure.

That said, it's not whining, it's mathematical fact.

Or do you dispute even a single, verified proven fact from the OP?

Do you dispute that Gallup was off by 9 in 2010?

Do you dispute that Gallup - but it's own admission - used a faulty voter model in 2012, and more importantly, grossly, and I mean, grossly overestimated the white voter percentage of the population?

Do you dispute that Gallup came to the press and admitted these things one month before it was also forced to admit that it settled for 10 million in court to avoid prosecution for having deliberately cooked the books?

This is all public record. Those are all facts. I know it's hard for your dinosaur-like brain to wrap itself around facts, but they are not going away.

The only one here whining - is you.

Thank you for playing, "nasty piece of work".