Rating the Pollsters

[Annie]

http://www.fordham.edu/images/acade...ccuracy in the 2008 presidential election.pdf

Poll Accuracy in the 2008 Presidential Election

—Initial Report, November 5, 2008—

Costas Panagopoulos, Ph.D.

Department of Political Science
Fordham University
For inquiries: [email protected] or (917) 405-9069

For all the derision directed toward pre-election polling, the final poll estimates were not
far off from the actual nationwide voteshares for the two candidates. On average, pre-
election polls from 23 public polling organizations projected a Democratic advantage of
7.52 percentage points on Election Day, which is only about 1.37 percentage points away
from the current estimate of a 6.15-point Obama margin in the national popular vote.

Following the procedures proposed by Martin, Traugott and Kennedy (see Public Opinion
Quarterly, Fall 2006, pp. 342-369) to assess poll accuracy, I analyze poll estimates from
these 23 polling organizations. Four of these polls appear to have overestimated McCain
support (indicated with a * below), while most polls (17) overestimated Obama strength.
Pre-election projections for two organizations’ final polls—Rasmussen and Pew—were
perfectly in agreement with the actual election result (**).

The following list ranks the 23 organizations by the accuracy of their final, national pre-
election polls (as reported on pollster.com).

1. Rasmussen (11/1-3)**
1. Pew (10/29-11/1)**
2. YouGov/Polimetrix (10/18-11/1)
3. Harris Interactive (10/20-27)
4. GWU (Lake/Tarrance) (11/2-3)*
5. Diageo/Hotline (10/31-11/2)*
5. ARG (10/25-27)*
6. CNN (10/30-11/1)
6. Ipsos/McClatchy (10/30-11/1)
7. DailyKos.com (D)/Research 2000 (11/1-3)
8. AP/Yahoo/KN (10/17-27)
9. Democracy Corps (D) (10/30-11/2)
10. FOX (11/1-2)
11. Economist/YouGov (10/25-27)
12. IBD/TIPP (11/1-3)
13. NBC/WSJ (11/1-2)
14. ABC/Post (10/30-11/2)
15. Marist College (11/3)
16. CBS (10/31-11/2)
17. Gallup (10/31-11/2)
18. Reuters/ C-SPAN/ Zogby (10/31-11/3)
19. CBS/Times (10/25-29)
20. Newsweek (10/22-23)

 

[Xenophon]

Gallup is near the bottom, that is surprising.

 

[Toro]

They did this in Canada, and found that there was no correlation between pollsters and accuracy, meaning the pollsters that were closest in one election were not closest the next.

There is a fair amount of randomness in the statistical process. The best way is to look at a sample of all polls. If you average the all the polls in the final few days before the 2008 election, they were off by about 1%, which is pretty good.