Poll Reading 101

[Dick Tuck]

Phantom I just posted the most recent fox poll. What's do you take away from it?

It's an outlier. The party identification flies in the face of other larger surveys. The result also files in the face of other FOX polls. To me it looks like FOX chose not to weigh the results in terms with known party identification.

Thanks idiot, but we wanted an educated response.

Whatever ass monkey. The point is the party self-identification in that survey flies in the face of much more reliable, large scale studies.

Show us your education and explain how FOX took that into account.

 

[BluePhantom]

Phantom I just posted the most recent fox poll. What's do you take away from it?

Well the first thing I see is that it's a RV poll, which is good...not great but good. However, Independents are underrepresented. Party identification varies in the USA but as a general rule of thumb it's pretty much split into thirds between the Republicans, Democrats, and Independents. So on this poll the breakdown is:

Democrats (n = 361) ± 5%
Republicans (n = 354) ± 5%
Independents (n = 173) ± 7.5%

Now because Democrats and Republicans are reasonably even it doesn't create as much of a problem, but it is something to file away in the back of my mind. We can have a look at the crosstabs and extrapolate a bit of information.

For example in question #1 we have:

"1. Do you approve or disapprove of the job Barack Obama is doing as president?"​

Well the total is 42% approval vs 51% disapproval...BUT among Independents it's 37% - 53% respectively. So if we consider that Independents are underrepresented by about 50%, in reality Obama's approval rating is probably a couple points lower than the 42% the poll suggests.

At question #8 we see a similar thing:

"8. If the presidential election were held today, how would you vote if the candidates were:"​

Now again Obama is losing in this poll 44% - 46%, BUT again the Independents are underrepresented and among that group Obama is down 37% - 43% so again the real spread is probably a couple points larger.

It's good that they asked that question early. The later you ask the question on who you would vote for the more likely it is that previous questions will bias the results. Looking at the previous questions I don't see anything from the wording that would necessarily create a bias and they are randomizing and rotating the order in which they list the candidates which is a good sign.

Question #24 for example is a real shitty question. The wording of the bolded part paints Obama in a negative light and biases the response.

"Before Barack Obama became president, the national debt was $10.6 trillion dollars. Another five trillion dollars has been added since Obama took office – or nearly a third of the country’s fifteen trillion dollar debt. If John McCain had been elected instead of Obama, do you think the national debt would be higher today, lower today or about the same?"​

Had they asked this question prior to question #8 then the results would be too biased to pay much attention to. Thankfully, they didn't do that, but anything after question #24 is now tainted goods. Fortunately, there were only two more questions anyhow, but it's a great example of a question that could have been very problematic had it been asked earlier.

I would like to see more information on the demographics of race, gender, education, etc that make up the survey sample. If 65% of the survey sample was men, for example, or 45% of the survey sample were black, then we have an issue because those are not accurate proportional representations of the demographics of the United States. Unfortunately they do not provide that information so we have to be a little careful here about the reliability of the data.

So for the poll itself I would say that spread is probably around 4% instead of 2% in favor of Romney, but because of the lack of some critical demographics information I am lukewarm on how much I trust it.

Now one of the things that I did look at is that according to this poll Obama has a 49% - 41% advantage among women. Very interesting because in other polls that advantage can be as high as 19%. Did Hilary Rosen's comments have something to do with it? No. Her comment came on April 11th which was the date that the poll was completed. It's possible that he received a bounce from Santorum suspending his election which happened on the 10th. The poll was conducted from the 9th - 11th but if he did it was not a massive one since that event happened in the middle of the poll date. However, the latest unemployment figures came out prior to the poll and they were disappointing. Obama only holds a 2% lead over Romney among women on issues related to the economy (question #10), and 94% of women said the economy was "extremely important" or "very important" (question #15). It's probable that Obama's decline among women had more to do with that than anything else.

Now if we look at other polls we can deduce some things. The Fox poll seems to be supported by the Rasmussen poll from the 11th - 13th. The ABC/WaPo poll from the 5th - 8th is problematic for reasons that Trajan and I pointed out in earlier posts. The TIPPonline poll should be completely disregarded. It's a terrible poll with very little information and online polling agencies are notoriously unreliable. The USAToday/Gallup poll on March 25th-26th is a little old at this point and unfortunately their crosstabs link is broken so there's no way to look at their demographic breakdown.

So what I would say now is a few things:

1) the Fox poll is "interesting" but I would not bet the family jewels on it until I see more polls that support its results.

2) With two polls (Fox and Rasmussen) suddenly showing Romney ahead and the trend on the Rasmussen polls showing Romney closing slowly in the past, it's an initial indication that the momentum of the race is shifting toward Romney...but again we need more polls to see if the trend is supported or not.

3) We must keep in mind that Romney will get an initial bounce after Santorum's withdrawal. In the Fox poll that probably wasn't much of an issue but it probably was in the Rasmussen poll because of the dates the poll was taken. It remains to be seen whether Romney can hold that

4) The gap among women appears to be closing. We need to keep an eye on that in the crosstabs for the next few weeks to see if a) it's actually a trend instead of just a freak accident on this particular poll, b) if it is a trend will it hold, and c) keep in mind Rosen's comments recently have ignited a large portion of women against Democrats that may only be temporary.

In a word I would say it's "interesting"...but I worry about the source (media poll) and it requires more supporting data before I get too excited about it.

 

[California Girl]

Phantom I just posted the most recent fox poll. What's do you take away from it?

Well the first thing I see is that it's a RV poll, which is good...not great but good. However, Independents are underrepresented. Party identification varies in the USA but as a general rule of thumb it's pretty much split into thirds between the Republicans, Democrats, and Independents. So on this poll the breakdown is:

Democrats (n = 361) ± 5%
Republicans (n = 354) ± 5%
Independents (n = 173) ± 7.5%

Now because Democrats and Republicans are reasonably even it doesn't create as much of a problem, but it is something to file away in the back of my mind. We can have a look at the crosstabs and extrapolate a bit of information.

For example in question #1 we have:

"1. Do you approve or disapprove of the job Barack Obama is doing as president?"​

Well the total is 42% approval vs 51% disapproval...BUT among Independents it's 37% - 53% respectively. So if we consider that Independents are underrepresented by about 50%, in reality Obama's approval rating is probably a couple points lower than the 42% the poll suggests.

At question #8 we see a similar thing:

"8. If the presidential election were held today, how would you vote if the candidates were:"​

Now again Obama is losing in this poll 44% - 46%, BUT again the Independents are underrepresented and among that group Obama is down 37% - 43% so again the real spread is probably a couple points larger.

It's good that they asked that question early. The later you ask the question on who you would vote for the more likely it is that previous questions will bias the results. Looking at the previous questions I don't see anything from the wording that would necessarily create a bias and they are randomizing and rotating the order in which they list the candidates which is a good sign.

Question #24 for example is a real shitty question. The wording of the bolded part paints Obama in a negative light and biases the response.

"Before Barack Obama became president, the national debt was $10.6 trillion dollars. Another five trillion dollars has been added since Obama took office – or nearly a third of the country’s fifteen trillion dollar debt. If John McCain had been elected instead of Obama, do you think the national debt would be higher today, lower today or about the same?"​

Had they asked this question prior to question #8 then the results would be too biased to pay much attention to. Thankfully, they didn't do that, but anything after question #24 is now tainted goods. Fortunately, there were only two more questions anyhow, but it's a great example of a question that could have been very problematic had it been asked earlier.

I would like to see more information on the demographics of race, gender, education, etc that make up the survey sample. If 65% of the survey sample was men, for example, or 45% of the survey sample were black, then we have an issue because those are not accurate proportional representations of the demographics of the United States. Unfortunately they do not provide that information so we have to be a little careful here about the reliability of the data.

So for the poll itself I would say that spread is probably around 4% instead of 2% in favor of Romney, but because of the lack of some critical demographics information I am lukewarm on how much I trust it.

Now one of the things that I did look at is that according to this poll Obama has a 49% - 41% advantage among women. Very interesting because in other polls that advantage can be as high as 19%. Did Hilary Rosen's comments have something to do with it? No. Her comment came on April 11th which was the date that her comment was made. It's possible that he received a bounce from Santorum suspending his election which happened on the 10th. The poll was conducted from the 9th - 11th but if he did it was not a massive one since that event happened in the middle of the poll date. However, the latest unemployment figures came out prior to the poll and they were disappointing. Obama only holds a 2% lead over Romney among women on issues related to the economy (question #10), and 95% of women said the economy was "extremely important" or "very important" (question #15). It's probable that Obama's decline among women had more to do with that than anything else.

Now if we look at other polls we can deduce some things. The Fox poll seems to be supported by the Rasmussen poll from the 11th - 13th. The ABC/WaPo poll from the 5th - 8th is problematic for reasons that Trajan and I pointed out in earlier posts. The TIPPonline poll should be completely disregarded. It's a terrible poll with very little information and online polling agencies are notoriously unreliable. The USAToday/Gallup poll on March 25th-26th is a little old at this point and unfortunately their crosstabs link is broken so there's no way to look at their demographic breakdown.

So what I would say now is a few things:

1) the Fox poll is "interesting" but I would not bet the family jewels on it until I see more polls that support its results.

2) With two polls (Fox and Rasmussen) suddenly showing Romney ahead and the trend on the Rasmussen polls showing Romney closing slowly in the past, it's an initial indication that the momentum of the race is shifting toward Romney...but again we need more polls to see if the trend is supported or not.

3) We must keep in mind that Romney will get an initial bounce after Santorum's withdrawal. In the Fox poll that probably wasn't much of an issue but it probably was in the Rasmussen poll because of the dates the poll was taken. It remains to be seen whether Romney can hold that

4) The gap among women appears to be closing. We need to keep an eye on that in the crosstabs for the next few weeks to see if a) it's actually a trend instead of just a freak accident on this particular poll, b) if it is a trend will it hold, and c) keep in mind Rosen's comments recently have ignited a large portion of women against Democrats that may only be temporary.

In a word I would say it's "interesting"...but it requires more supporting data before I get too excited about it.

Purely from an academic standpoint, I prefer polls that have a decent size poll. I tend to dismiss 910 as not a large enough sample to really tell me anything.

 

[The T]

Phantom I just posted the most recent fox poll. What's do you take away from it?

Well the first thing I see is that it's a RV poll, which is good...not great but good. However, Independents are underrepresented. Party identification varies in the USA but as a general rule of thumb it's pretty much split into thirds between the Republicans, Democrats, and Independents. So on this poll the breakdown is:

Democrats (n = 361) ± 5%
Republicans (n = 354) ± 5%
Independents (n = 173) ± 7.5%

Now because Democrats and Republicans are reasonably even it doesn't create as much of a problem, but it is something to file away in the back of my mind. We can have a look at the crosstabs and extrapolate a bit of information.

For example in question #1 we have:

"1. Do you approve or disapprove of the job Barack Obama is doing as president?"
​

Well the total is 42% approval vs 51% disapproval...BUT among Independents it's 37% - 53% respectively. So if we consider that Independents are underrepresented by about 50%, in reality Obama's approval rating is probably a couple points lower than the 42% the poll suggests.

At question #8 we see a similar thing:

"8. If the presidential election were held today, how would you vote if the candidates were:"
​

Now again Obama is losing in this poll 44% - 46%, BUT again the Independents are underrepresented and among that group Obama is down 37% - 43% so again the real spread is probably a couple points larger.

It's good that they asked that question early. The later you ask the question on who you would vote for the more likely it is that previous questions will bias the results. Looking at the previous questions I don't see anything from the wording that would necessarily create a bias and they are randomizing and rotating the order in which they list the candidates which is a good sign.

Question #24 for example is a real shitty question. The wording of the bolded part paints Obama in a negative light and biases the response.

"Before Barack Obama became president, the national debt was $10.6 trillion dollars. Another five trillion dollars has been added since Obama took office – or nearly a third of the country’s fifteen trillion dollar debt. If John McCain had been elected instead of Obama, do you think the national debt would be higher today, lower today or about the same?"
​

Had they asked this question prior to question #8 then the results would be too biased to pay much attention to. Thankfully, they didn't do that, but anything after question #24 is now tainted goods. Fortunately, there were only two more questions anyhow, but it's a great example of a question that could have been very problematic had it been asked earlier.

I would like to see more information on the demographics of race, gender, education, etc that make up the survey sample. If 65% of the survey sample was men, for example, or 45% of the survey sample were black, then we have an issue because those are not accurate proportional representations of the demographics of the United States. Unfortunately they do not provide that information so we have to be a little careful here about the reliability of the data.

So for the poll itself I would say that spread is probably around 4% instead of 2% in favor of Romney, but because of the lack of some critical demographics information I am lukewarm on how much I trust it.

Now one of the things that I did look at is that according to this poll Obama has a 49% - 41% advantage among women. Very interesting because in other polls that advantage can be as high as 19%. Did Hilary Rosen's comments have something to do with it? No. Her comment came on April 11th which was the date that the poll was completed. It's possible that he received a bounce from Santorum suspending his election which happened on the 10th. The poll was conducted from the 9th - 11th but if he did it was not a massive one since that event happened in the middle of the poll date. However, the latest unemployment figures came out prior to the poll and they were disappointing. Obama only holds a 2% lead over Romney among women on issues related to the economy (question #10), and 94% of women said the economy was "extremely important" or "very important" (question #15). It's probable that Obama's decline among women had more to do with that than anything else.

Now if we look at other polls we can deduce some things. The Fox poll seems to be supported by the Rasmussen poll from the 11th - 13th. The ABC/WaPo poll from the 5th - 8th is problematic for reasons that Trajan and I pointed out in earlier posts. The TIPPonline poll should be completely disregarded. It's a terrible poll with very little information and online polling agencies are notoriously unreliable. The USAToday/Gallup poll on March 25th-26th is a little old at this point and unfortunately their crosstabs link is broken so there's no way to look at their demographic breakdown.

So what I would say now is a few things:

1) the Fox poll is "interesting" but I would not bet the family jewels on it until I see more polls that support its results.

2) With two polls (Fox and Rasmussen) suddenly showing Romney ahead and the trend on the Rasmussen polls showing Romney closing slowly in the past, it's an initial indication that the momentum of the race is shifting toward Romney...but again we need more polls to see if the trend is supported or not.

3) We must keep in mind that Romney will get an initial bounce after Santorum's withdrawal. In the Fox poll that probably wasn't much of an issue but it probably was in the Rasmussen poll because of the dates the poll was taken. It remains to be seen whether Romney can hold that

4) The gap among women appears to be closing. We need to keep an eye on that in the crosstabs for the next few weeks to see if a) it's actually a trend instead of just a freak accident on this particular poll, b) if it is a trend will it hold, and c) keep in mind Rosen's comments recently have ignited a large portion of women against Democrats that may only be temporary.

In a word I would say it's "interesting"...but it requires more supporting data before I get too excited about it.

So in effect? Polls are geared towrd partisan ends?

 

[Remodeling Maidiac]

Phantom I just posted the most recent fox poll. What's do you take away from it?

Well the first thing I see is that it's a RV poll, which is good...not great but good. However, Independents are underrepresented. Party identification varies in the USA but as a general rule of thumb it's pretty much split into thirds between the Republicans, Democrats, and Independents. So on this poll the breakdown is:

Democrats (n = 361) ± 5%
Republicans (n = 354) ± 5%
Independents (n = 173) ± 7.5%

Now because Democrats and Republicans are reasonably even it doesn't create as much of a problem, but it is something to file away in the back of my mind. We can have a look at the crosstabs and extrapolate a bit of information.

For example in question #1 we have:

"1. Do you approve or disapprove of the job Barack Obama is doing as president?"​

Well the total is 42% approval vs 51% disapproval...BUT among Independents it's 37% - 53% respectively. So if we consider that Independents are underrepresented by about 50%, in reality Obama's approval rating is probably a couple points lower than the 42% the poll suggests.

At question #8 we see a similar thing:

"8. If the presidential election were held today, how would you vote if the candidates were:"​

Now again Obama is losing in this poll 44% - 46%, BUT again the Independents are underrepresented and among that group Obama is down 37% - 43% so again the real spread is probably a couple points larger.

It's good that they asked that question early. The later you ask the question on who you would vote for the more likely it is that previous questions will bias the results. Looking at the previous questions I don't see anything from the wording that would necessarily create a bias and they are randomizing and rotating the order in which they list the candidates which is a good sign.

Question #24 for example is a real shitty question. The wording of the bolded part paints Obama in a negative light and biases the response.

"Before Barack Obama became president, the national debt was $10.6 trillion dollars. Another five trillion dollars has been added since Obama took office – or nearly a third of the country’s fifteen trillion dollar debt. If John McCain had been elected instead of Obama, do you think the national debt would be higher today, lower today or about the same?"​

Had they asked this question prior to question #8 then the results would be too biased to pay much attention to. Thankfully, they didn't do that, but anything after question #24 is now tainted goods. Fortunately, there were only two more questions anyhow, but it's a great example of a question that could have been very problematic had it been asked earlier.

I would like to see more information on the demographics of race, gender, education, etc that make up the survey sample. If 65% of the survey sample was men, for example, or 45% of the survey sample were black, then we have an issue because those are not accurate proportional representations of the demographics of the United States. Unfortunately they do not provide that information so we have to be a little careful here about the reliability of the data.

So for the poll itself I would say that spread is probably around 4% instead of 2% in favor of Romney, but because of the lack of some critical demographics information I am lukewarm on how much I trust it.

Now one of the things that I did look at is that according to this poll Obama has a 49% - 41% advantage among women. Very interesting because in other polls that advantage can be as high as 19%. Did Hilary Rosen's comments have something to do with it? No. Her comment came on April 11th which was the date that the poll was completed. It's possible that he received a bounce from Santorum suspending his election which happened on the 10th. The poll was conducted from the 9th - 11th but if he did it was not a massive one since that event happened in the middle of the poll date. However, the latest unemployment figures came out prior to the poll and they were disappointing. Obama only holds a 2% lead over Romney among women on issues related to the economy (question #10), and 94% of women said the economy was "extremely important" or "very important" (question #15). It's probable that Obama's decline among women had more to do with that than anything else.

Now if we look at other polls we can deduce some things. The Fox poll seems to be supported by the Rasmussen poll from the 11th - 13th. The ABC/WaPo poll from the 5th - 8th is problematic for reasons that Trajan and I pointed out in earlier posts. The TIPPonline poll should be completely disregarded. It's a terrible poll with very little information and online polling agencies are notoriously unreliable. The USAToday/Gallup poll on March 25th-26th is a little old at this point and unfortunately their crosstabs link is broken so there's no way to look at their demographic breakdown.

So what I would say now is a few things:

1) the Fox poll is "interesting" but I would not bet the family jewels on it until I see more polls that support its results.

2) With two polls (Fox and Rasmussen) suddenly showing Romney ahead and the trend on the Rasmussen polls showing Romney closing slowly in the past, it's an initial indication that the momentum of the race is shifting toward Romney...but again we need more polls to see if the trend is supported or not.

3) We must keep in mind that Romney will get an initial bounce after Santorum's withdrawal. In the Fox poll that probably wasn't much of an issue but it probably was in the Rasmussen poll because of the dates the poll was taken. It remains to be seen whether Romney can hold that

4) The gap among women appears to be closing. We need to keep an eye on that in the crosstabs for the next few weeks to see if a) it's actually a trend instead of just a freak accident on this particular poll, b) if it is a trend will it hold, and c) keep in mind Rosen's comments recently have ignited a large portion of women against Democrats that may only be temporary.

In a word I would say it's "interesting"...but it requires more supporting data before I get too excited about it.

Impressive analysis. Would you mind posting it in the other thread? Cut and paste maybe?

Anyhow, nicely done. Thanks

 

[Remodeling Maidiac]

Good point also CG.

 

[BluePhantom]

It's an outlier. The party identification flies in the face of other larger surveys. The result also files in the face of other FOX polls. To me it looks like FOX chose not to weigh the results in terms with known party identification.

Thanks idiot, but we wanted an educated response.

Whatever ass monkey. The point is the party self-identification in that survey flies in the face of much more reliable, large scale studies.

Show us your education and explain how FOX took that into account.

Well a polling agency doesn't exactly "take it into account". They release the data collected based upon the sample they reached and if they are a good company they will release the sample demographics in the crosstabs so we can analyze it and take it into account ourselves. You are correct that the party-identification breakdown in the Fox poll is problematic as Independents are underrepresented and that's an important thing to keep in mind. But the balance between Democrats and Republicans is proportional to each other as far as what they should be....which is roughly even (give or take a percentage point or two depending on what is going on in the nation and the world).

 

[BluePhantom]

Purely from an academic standpoint, I prefer polls that have a decent size poll. I tend to dismiss 910 as not a large enough sample to really tell me anything.

I tend to agree. The larger the sample size the more comfortable I am with the results.

 

[The T]

It's an outlier. The party identification flies in the face of other larger surveys. The result also files in the face of other FOX polls. To me it looks like FOX chose not to weigh the results in terms with known party identification.

Thanks idiot, but we wanted an educated response.

Whatever ass monkey. The point is the party self-identification in that survey flies in the face of much more reliable, large scale studies.

Show us your education and explain how FOX took that into account.

Why you fail. Party over country.

 

[Full-Auto]

It's an outlier. The party identification flies in the face of other larger surveys. The result also files in the face of other FOX polls. To me it looks like FOX chose not to weigh the results in terms with known party identification.

Thanks idiot, but we wanted an educated response.

Whatever ass monkey. The point is the party self-identification in that survey flies in the face of much more reliable, large scale studies.

Show us your education and explain how FOX took that into account.

So you have nothing but hot air.........

Which was my point...

Post up wuss.................

 

[BluePhantom]

So in effect? Polls are geared towrd partisan ends?

Some are sure. Most respectable polling agencies are not. They may on occasion ask a badly phrased question that unintentionally biases the results but usually that's error instead of intent. Trajan and I have been discussing media polls and how they are problematic and I would definitely say that media polls tend to be geared toward reaching a given partisan result far more than professional polling firms.

 

[The T]

So in effect? Polls are geared towrd partisan ends?

Some are sure. Most respectable polling agencies are not. They may on occasion ask a badly phrased question that unintentionally biases the results but usually that's error instead of intent. Trajan and I have been discussing media polls and how they are problematic and I would definitely say that media polls tend to be geared toward reaching a given partisan result far more than professional polling firms.

So I've been reading. Very interesting exercise here. Very informative. Nice insight.

 

[California Girl]

It's an outlier. The party identification flies in the face of other larger surveys. The result also files in the face of other FOX polls. To me it looks like FOX chose not to weigh the results in terms with known party identification.

Thanks idiot, but we wanted an educated response.

Whatever ass monkey. The point is the party self-identification in that survey flies in the face of much more reliable, large scale studies.

Show us your education and explain how FOX took that into account.

It appears that you don't know the difference between a 'study' and a 'poll'. Idiot.

 

[The T]

It's an outlier. The party identification flies in the face of other larger surveys. The result also files in the face of other FOX polls. To me it looks like FOX chose not to weigh the results in terms with known party identification.

Thanks idiot, but we wanted an educated response.

Whatever ass monkey. The point is the party self-identification in that survey flies in the face of much more reliable, large scale studies.

Show us your education and explain how FOX took that into account.

You first.

 

[California Girl]

So in effect? Polls are geared towrd partisan ends?

Some are sure. Most respectable polling agencies are not. They may on occasion ask a badly phrased question that unintentionally biases the results but usually that's error instead of intent. Trajan and I have been discussing media polls and how they are problematic and I would definitely say that media polls tend to be geared toward reaching a given partisan result far more than professional polling firms.

I agree.... and I have raised that more than once on this forum. People tend to take the findings as legitimate without interrogating the source for validity. Often, particularly with media polls, the questions are very clearly leading or biased in their wording... which means the results are skewed... but try explaining that to the ******* morons on either side.

 

[NYcarbineer]

As the general election heats up between Romney and Obama I am seeing a lot of posts pointing to this poll and that poll and I am noticing an equal amount of misunderstanding from all sides. As I am an admitted "poll geek" (to the point that I have a spreadsheet I wrote to analyze polling data - I know "get a life") I want to take some time to explain how to read polls and get the most from the information they offer. While I imagine there are threads on this topic from 2008 or 2010 it appears it's time for a refresher course at the very least.

Rule #1: Consider the Sample

You will generally see polls sampled in three ways. The first is "adults" (A). Polls that sample "adults" is the least reliable because only about 50%-55% of the eligible population actually turn out to vote. Polls of A might give you an idea of public perception but they don't tell you a lot about who is in the best position to win an election.

The second (and most common) is "registered voters" (RV). This is better than A polls because they disregard anyone who is not eligible and in a position to vote. Still they are not the best because only about 70% of registered voters actually go and cast a ballot. So it's better, but still slightly problematic. Quinnipiac and Gallup are examples of firms that use RV sampling.

The third (and best) is "likely voters" (LV). This considers only people who are registered to vote and meet a statistical criteria that indicates they actually will go vote. LV polls, with only a few rare exceptions (like Quinnipiac, for example) are the ones to pay the most attention to. Rasmussen and SurveyUSA are examples of firms that use a LV sampling method.

Rule #2. Understand Margin of Error

I see people all the time get so excited about a poll that shows their candidate up by 3%. In reality, from a statistical perspective that's a tie. Every poll will have a slightly different margin of error but a good rule of thumb is 4%. If a poll shows a lead of 4% or less, it's a statistical tie and could go either way.

Rule #3. Pay Attention to Timing

A poll in April about an election in November doesn't mean a whole lot. Too much can happen. The economy could dramatically recover or totally tank between those times. A scandal may break. We could get attacked and forced into a military confrontation. A candidate may get a temporary bounce from their party's convention, the selection of a running mate, or even a human interest story that captures the nation's attention. All of these things will influence the polls and voter preference. The closer to the election, the more valuable a poll becomes. This is why we experience the "October Surprise" (the dirty secret that a candidate exposes about their opponent a week or two before the election). Knowing what is happening and when can help you identify the difference between a trend that is likely to stick and a temporary bounce.

Rule #4. Know the Polling Agencies Affiliations and History

Any agency can luck out in a given year. It's important to know which firms show a history of accuracy over multiple election cycles. For example, the Washington Post was great on a few selected state polls in 2010. In 2008 however they were absolutely dreadful. Gallup has a great reputation but over the last several years they have been getting less and less accurate. Rasmussen had a surprisingly weak 2010 but in 2008 and for years prior they were absolutely deadly accurate. What changes? Sometimes their methodology, sometimes nothing....they had a bad year or a good year.

Also keep in mind that some firms are affiliated with a given party. Public Polling Policy (PPP), for example, is funded and affiliated exclusively with the DNC. Magellan Strategies, the RNC. Usually, on RealClearPolitics, thoss agencies are noted (D) or (R) for Democratic affiliated firms or Republican affiliated firms respectively. It's wise to keep in mind who is paying their bills when you consider the validity of their data. That's not to say these firms should be completely disregarded...just that it should be kept in mind.

Rule #5. READ THE ******* CROSSTABS

The crosstabs are information about the specifics of the polling demographics in that sample. They are usually at the very beginning or the very end of a polling report. Many liberals might be excited as hell with a poll that shows Obama with an 11% lead until they look more closely and notice that (simply by sheer chance) the polling agency reached a sample where 47% of them identified as Democrats compared to only 23% that identified themselves as Republicans. This creates what is known as the dreaded "outlier". Simply by sheer random chance the agency reached a given demographic that is out of proportion with the United States as a whole and it skewed the results to the point where the data is unreliable.

Rule #6. Trends and Averages are More Important Than Snapshots

A poll is basically a snap shot: "at this precise moment in time and according to the sample we reached, this is what the feeling is". The best way to read polls is to look at a collection of reliable polls and average the results. RealClearPolitics does this with the "RCP average" but that average does not consider all the points I have discussed. If it's a recent poll it gets counted whether it's a good poll or a bad poll.

Tracking the trends associated with the averages shows more than just what the snapshot is but where there is momentum toward one side or the other. It's the trends that matter more when election day is distant. Those snapshots only have real relevance a week or so away from election day because things can happen so fast that even a historically accurate poll can show a dramatic change in their data within a very short period of time depending on what happens to be going on at one point in time compared to the other.

So with all that said let me list in order the common agencies that, through my research and tracking on my spreadsheet, are the most valuable and the most accurate.

The Deadly Accurate Duo (First Tier)
1. SurveyUSA
2. Quinnipiac

Damned Accurate (Second Tier)
3. Rasmussen (slipped from First Tier after a shaky 2010)

Pay Close Attention To (Third Tier)
4. Mason-Dixon
5. PPP

Worth Consideration (Fourth Tier)
6. Gallup
7. Magellan Strategies
8. Strategic Vision

Consider With Care
9. ABC/Washington Post (one good year in a history of disaster does not establish confidence)
10. Fox News (historically getting more and more accurate but not there quite yet)

Best to Ignore
Pretty much everything else

By keeping these above points in mind a true "student of the polls" will be able to get a much more solid understanding of who is winning and losing, how they are winning, why they are winning, and will be able to distinguish between what is important and what is irrelevant. The ability to effectively analyze the polling data can also mean the difference between making a strong argument on a thread or being exposed as a complete tool. These concepts are vital to understand whether you are simply looking for ammo in a debate or you really want a true understanding of the political landscape and your candidate's chances for victory.

How do you know a poll is accurate?

 

[BluePhantom]

I agree.... and I have raised that more than once on this forum. People tend to take the findings as legitimate without interrogating the source for validity. Often, particularly with media polls, the questions are very clearly leading or biased in their wording... which means the results are skewed... but try explaining that to the ******* morons on either side.

Yes. I remember an experiment that I read back in 2008 about wording of questions. I will try and find the link but basically what they did was ask who the person supported between Obama and McCain with the very first question and again at the very last question. In between they asked a series of 25 or so questions that were intentionally biased toward one candidate. They then identified how many people changed their mind during the course of the poll and at the end supported a different candidate than the one they had said at the beginning. I don't recall the exact statistics but I remember being surprised that the number that changed their minds was so high. It was like 20% if I recall correctly.

Now firms with an agenda know this and they can structure a poll specifically in order to achieve a given result. Then they can publish that poll to RCP (who takes just about anyone) and influence the RCP average or release it to friendly media outlets who can use it, they hope, to bias the public and create a bandwagon effect for their candidate.

Now that's uncommon of course, but you just have to be really careful with the data and consider the source. If it's not a respected independent polling agency, I always recommend that you approach with extreme caution.

 

[NYcarbineer]

Again.

How do you know a poll is accurate?

 

[RightWingFerret]

does someone here have the stasitcs of what % of registered voters are R/Conserv/Ind/and Dems? what we really need to know is how many registered voters are dems, then compare it to the rest.they say 40% of us are conservatives. so how are the other 60% caculated? is it true that millions of dems have switched red over the last 2 years?

 

[BluePhantom]

Again.

How do you know a poll is accurate?

in what sense? How do you know they are releasing the actual data they collected? How do you know if it's an accurate representation of what they are trying to measure? Be more specific as to what you are referring to