Poll Reading 101

[NYcarbineer]

Ah, so the guy who puts 'educator (of liberals)' under his username wants to shoot the messenger Nate Silver

on an unsubstantiated charge of partisan bias?

Did you REALLY think you could slip that one through? Don't insult me.

If all Nate did was "deliver the message" I wouldn't shoot him. Unfortunately, that's not what he does. Consider:

In his rankings on June 6, 2010 Silver lists his rankings based on "PIE" (pollster introduced error) instead of "rawscore" which is historically how accurate they have been. He argues that regardless of past results the best way to predict their future dependability is through PIE. Problem....he never really explains how he distinguished between avoidable errors and unavoidable errors and he regresses the statistics toward different means. So if a firm is part of those organizations, they will be regressed toward one mean. if they are not, they will be regressed toward a completely different mean.

Now that is a decision not based upon results, historical accuracy, or anything related to the integrity of the data. It's based completely on Silver's personal determination of what is avoidable vs. unavoidable and when he cannot see their methodology (like with Rasmussen) he assumes the error is avoidable and regresses them toward a dramatically more unfavorable mean.

That my friend is not statistical analysis. That's politics and that's bias.

I encourage you to read Mark Blumenthal's excellent article regarding Nate's ratings for a more detailed explanation.

Rasmussen makes you pay for crosstabs. You say crosstabs are very important. You're essentially an advertisement to sell Rasmussen subscriptions.

 

[NYcarbineer]

Your reasoning is fatally flawed because you're assuming that the Independents sampled in the poll accurately represent Independents.

Independents can be anything from far left to far right. In a poll that oversampled conservative Independents, as this one may well have done, then approval of the president is going to skew against him,

which it did in this poll, and which is a good bet why this poll was an outlier.

Secondly, that R vs D ratio is laughably skewed in favor of Republicans.

Not really.

According to Gallup and Rasmussen the split between party identification is as follows

Party.......Gallup........Rasmussen
Dem.........30%.............33.4%
Rep..........27%.............36.4%
Ind...........42%.............30.2%

Now historically Rasmussen is a bit more accurate than Gallup but just for simplicity let's average the two out with equal weight and get a more likely breakdown of:

Dem: 31.7%
Rep: 31.7%
Ind: 36.1%

This would be more in line with historical trends. Now look at the Fox Poll sample

Dem: 361 or 40.7%
Rep: 354 or 39.9%
Ind: 173 or 19.5%

The balance between Democrats and Republicans is less than 1% which is right about where it should be according to the averages of the Gallup and Rasmussen reports. But the number of Independents is over 16 points lower than it should be according to the same data. So it's not that Republicans are over-represented any more than Democrats are and frankly the Democrats are slightly more over-represented than the Republicans. It's that Independents are underrepresented.

The best indicator of political balance in a poll is liberal/conservative/moderate self-identification, not Republican/Democrat/Independent.

Yeah I would tend to agree with you on that point. Like I said, there are a lot of things about this particular poll I find a little concerning and the sample was one of them. As far as liberal/conservative/moderate self-identification vs Republican/Democrat/Independent, I think that's a fair point.

If Rasmussen is a more accurate less biased pollster, how did he manage to give Bush higher than average approval ratings over the course of his presidency,

and now manages to give Obama lower than average approval ratings throughout his presidency?

Explain that.

 

[BluePhantom]

Prove that. Show me that you know the real numbers of total party breakdown, historically, and then show me that Rasmussen was/is closer than any other pollster.

I meant taken as a whole Rasmussen has been more accurate the last several years than Gallup. I didn't mean specifically in regard to party identification. Unfortunately, very few agencies poll very aggressively on the topic. This is probably for a couple reasons: 1) pollsters gather data according to what they are paid to gather data on. Not a lot of organizations are particularly interested in this particular topic. They are more interested in who is leading who, where, and why. 2) Party identification can change dramatically in very short periods of time. The American people are a fickle bunch and one minute they will identify one way, then something will happen in the world and it will swing the other way. So unless you track it on a consistent basis and use rolling averages of the collected data (as Rasmussen and Gallup do) it becomes a basically pointless exercise, because a) there's no money in it, and b) it's a lot resources for an agency to commit to a very volatile question.

However; Gallup releases their full results for free here and Rasmussen releases a monthly breakdown here. Rasmussen makes you pay for older data so if you want to avoid paying then start a spreadsheet and check with them monthly to record their data.

 

[BluePhantom]

Rasmussen makes you pay for crosstabs. You say crosstabs are very important. You're essentially an advertisement to sell Rasmussen subscriptions.

Jesus you are really grasping at straws here. Yes the crosstabs are important. If it's that important to you to get Rasmussen's full breakdown then shell out the $20 a month to do it. If it's not that important to you then keep your money. If you wish to downgrade the weight you place on Rasmussen or disregard them completely because you are unwilling to buy their service that's entirely up to you. I really couldn't care less.

I would argue that your intentions are probably more because you have fallen victim to the liberal propaganda about their extreme bias, but it really makes no difference to me. Personally I choose to include in my calculations a polling agency that has shown a history of accuracy but if you wish to ignore them...knock yourself out.

 

[BluePhantom]

If Rasmussen is a more accurate less biased pollster, how did he manage to give Bush higher than average approval ratings over the course of his presidency,
and now manages to give Obama lower than average approval ratings throughout his presidency?

Explain that.

probably because Rasmussen uses a LV model where almost all others use a RV or A model. Go back and re-read Rule #1

 

[BluePhantom]

If Rasmussen is a more accurate less biased pollster, how did he manage to give Bush higher than average approval ratings over the course of his presidency,
and now manages to give Obama lower than average approval ratings throughout his presidency?

Explain that.

The big ***** with Rasmussen is their methodology which is quite different than most. It's a controversial method that uses recordings to ask questions rather than live interviews and random dialing programs among other things. That's not the "classical method" and it unnerves traditional pollsters and seriously pisses people like Nate Silver off. However, Silver's attacks on Rasmussen are really more rooted in their methodology rather than their results. When you look simply at results Silver is forced to concede.

"Consider also the analysis of Nate Silver. On his website Fivethirtyeight.com last year, he approached the issue of survey quality from the perspective of the accuracy of the results rather than their underlying methodology. He gathered past polling data from 171 contests for President, Governor and Senate fielded since 2000 and calculated accuracy scores for each pollster. His study rated Rasmussen as the third most accurate of 32 pollsters, just behind SurveyUSA, another automated pollster. When he compared eight daily tracking polls last fall, Rasmussen ranked first in Silver's accuracy ratings. He concluded that Rasmussen, "with its large sample size and high pollster rating -- would probably be the one I'd want with me on a desert island."​

On his website Silver states:

"In summation, none of these tracking polls are perfect, although Rasmussen -- with its large sample size and high pollster rating -- would probably be the one I'd want with me on a desert island. Conversely, the only one of the trackers that I consider obviously dubious is Zogby."​

So give it a rest about "Rasmussen's evil empire". Even Silver admits that while he hates their methods their results are historically accurate.

 

[asaratis]

Nice effort TC

+rep

Who is TC? How do I +rep him? Is he a real phantom? Is he really blue?

 

[BluePhantom]

It appears I am going to have to get into a little more detail about why the methodology between A polls, RV polls, and LV polls is so important and why it creates the illusion of bias and so much liberal bellyaching.

Again. A polls are polls of "adults". Anyone 18 or older. But because only 50%-55% of the eligible population actually votes, these polls give you a good idea of what the public thinks, but it doesn't tell you very much about who is going to win an election. The majority of the public is Democrat so polls of A tend to show a far better performance by liberal candidates.

RV polls are polls of "registered voters". Gallup uses the RV method as as a result Democrats tend to perform better in comparison to other polls. Since RV is the most common method, Democrats will tend to perform better in most polls. BUT only about 70% of registered voter actually vote. Sometimes it can be as low as 50%. It just depends on what is going on in the world. So you get a better idea of who is likely to win the election than using A polls but it's still iffy. So some pollsters use LV instead.

LV polls mean "likely voters". This is the most accurate measure of "who is likely to win an election", not "what do the people think" because it only considers those who through a formula are determined the be an active voter in the coming election.

The likelihood that someone will vote is based on a pretty basic calculation. Each polling agency will vary it a bit but it comes down to (P*B)+M > C; where P is the probability that an individual vote will change the outcome of an election, B is the tangible benefit someone stands to gain by their candidate winning, M is the non-material motivation factor and C is the cost in time, effort, and currency it takes to vote.

Now since P is basically zero then B is also basically zero and it comes down to whether M is greater than C. So, for example, if a person is not currently registered to vote their C value will be higher than a person who is currently registered to vote. Why? Because for the former it will take time and effort to register and that will raise the value of C over someone who has already registered. other things that weigh into C are things like how far the polling location is from their home, do they get time off work to vote, do they have a car, do they plan to be out of town on election day forcing the use of an absentee ballot, etc

M is determined by a lot of things; how important a critical issue is to you, whether you feel that voting is a "duty", whether you pay attention to politics in general, etc. It also considers your past voting history. How often have you voted in the last 4 elections for example.

So what they will do is after you have taken the poll, they ask you all these questions designed to calculate your personal M and C values. If M is greater than C they consider you a "likely voter" and your poll answers are considered. If M is less than C they consider you "unlikely to vote" and they ignore your responses.

Now because Republican candidates generally perform best in LV polls liberals like to argue that they are biased. No. Republicans perform better in those polls not because the poll is biased but because as it turns out Republicans are more likely to vote than Democrats and historically by a wide margin. This is precisely why large voter turnout tends to favor a Democratic candidate. While there are more Democrats in the nation, Republicans vote at a far higher percentage of their given population.

So are LV polls, like Rasmussen and SurveyUSA, "biased"? Well liberals cry that they are, but in reality if you consider an LV poll biased toward Republicans you must also consider an RV poll biased toward Democrats. It's also worth noting that according to the 2008 election results, our liberal friend Nate Silver (who the liberals love to cite), was forced to concede that out of the top three most accurate agencies, SurveyUSA and Rasmussen (both LV pollsters) were #2 and #3 respectively. So is it that LV polls are biased or simply more accurate? The data suggests the latter. Is it that RV polls are biased or simply less accurate? The data suggests that the former may be the reason for the latter.

This is also why when it comes to things like approval rating and voter identification, Rasmussen's numbers will show more favorable data in both categories for Republicans than Gallup's will. Rasmussen's data is based on LV while Gallup's is based on RV.

Now does this finally clear it all up for everyone?

 

[NYcarbineer]

If Rasmussen is a more accurate less biased pollster, how did he manage to give Bush higher than average approval ratings over the course of his presidency,
and now manages to give Obama lower than average approval ratings throughout his presidency?

Explain that.

The big ***** with Rasmussen is their methodology which is quite different than most. It's a controversial method that uses recordings to ask questions rather than live interviews and random dialing programs among other things. That's not the "classical method" and it unnerves traditional pollsters and seriously pisses people like Nate Silver off. However, Silver's attacks on Rasmussen are really more rooted in their methodology rather than their results. When you look simply at results Silver is forced to concede.

"Consider also the analysis of Nate Silver. On his website Fivethirtyeight.com last year, he approached the issue of survey quality from the perspective of the accuracy of the results rather than their underlying methodology. He gathered past polling data from 171 contests for President, Governor and Senate fielded since 2000 and calculated accuracy scores for each pollster. His study rated Rasmussen as the third most accurate of 32 pollsters, just behind SurveyUSA, another automated pollster. When he compared eight daily tracking polls last fall, Rasmussen ranked first in Silver's accuracy ratings. He concluded that Rasmussen, "with its large sample size and high pollster rating -- would probably be the one I'd want with me on a desert island."​

On his website Silver states:

"In summation, none of these tracking polls are perfect, although Rasmussen -- with its large sample size and high pollster rating -- would probably be the one I'd want with me on a desert island. Conversely, the only one of the trackers that I consider obviously dubious is Zogby."​

So give it a rest about "Rasmussen's evil empire". Even Silver admits that while he hates their methods their results are historically accurate.

Shall I repeat the question?

If Rasmussen is a more accurate less biased pollster, how did he manage to give Bush higher than average approval ratings over the course of his presidency,
and now manages to give Obama lower than average approval ratings throughout his presidency?

Explain that

 

[NYcarbineer]

[

So give it a rest about "Rasmussen's evil empire". Even Silver admits that while he hates their methods their results are historically accurate.

We've already established that no one knows the real numbers in an approval poll, so how can you claim Rasmussen's polls are historically accurate?

 

[NYcarbineer]

This is also why when it comes to things like approval rating and voter identification, Rasmussen's numbers will show more favorable data in both categories for Republicans than Gallup's will. Rasmussen's data is based on LV while Gallup's is based on RV.

Now does this finally clear it all up for everyone?

So you admit that Rasmussen intentionally uses a peculiar method that he knows will skew the numbers in favor of Republicans.

That is pure bias by definition.

 

[BluePhantom]

Shall I repeat the question?

If Rasmussen is a more accurate less biased pollster, how did he manage to give Bush higher than average approval ratings over the course of his presidency,
and now manages to give Obama lower than average approval ratings throughout his presidency?

Explain that

I already did in post #65, however, I anticipated your inability to grasp the connection and posted a more thorough explanation in post #68

 

[NYcarbineer]

Prove that. Show me that you know the real numbers of total party breakdown, historically, and then show me that Rasmussen was/is closer than any other pollster.

I meant taken as a whole Rasmussen has been more accurate the last several years than Gallup. I didn't mean specifically in regard to party identification. Unfortunately, very few agencies poll very aggressively on the topic. This is probably for a couple reasons: 1) pollsters gather data according to what they are paid to gather data on. Not a lot of organizations are particularly interested in this particular topic. They are more interested in who is leading who, where, and why. 2) Party identification can change dramatically in very short periods of time. The American people are a fickle bunch and one minute they will identify one way, then something will happen in the world and it will swing the other way. So unless you track it on a consistent basis and use rolling averages of the collected data (as Rasmussen and Gallup do) it becomes a basically pointless exercise, because a) there's no money in it, and b) it's a lot resources for an agency to commit to a very volatile question.

However; Gallup releases their full results for free here and Rasmussen releases a monthly breakdown here. Rasmussen makes you pay for older data so if you want to avoid paying then start a spreadsheet and check with them monthly to record their data.

So we're talking about party identification, you claim that Rasmussen is more accurate, then suddenly when you're asked to actually substantiate,

then we weren't talking about party identification?

lol

 

[BluePhantom]

This is also why when it comes to things like approval rating and voter identification, Rasmussen's numbers will show more favorable data in both categories for Republicans than Gallup's will. Rasmussen's data is based on LV while Gallup's is based on RV.

Now does this finally clear it all up for everyone?

So you admit that Rasmussen intentionally uses a peculiar method that he knows will skew the numbers in favor of Republicans.

That is pure bias by definition.

Good God, talk about completely missing the point and twisting the argument. LV is hardly "peculiar". Go read post #68 again, and this time take off your blue glasses.

 

[California Girl]

It appears I am going to have to get into a little more detail about why the methodology between A polls, RV polls, and LV polls is so important and why it creates the illusion of bias and so much liberal bellyaching.

Again. A polls are polls of "adults". Anyone 18 or older. But because only 50%-55% of the eligible population actually votes, these polls give you a good idea of what the public thinks, but it doesn't tell you very much about who is going to win an election. The majority of the public is Democrat so polls of A tend to show a far better performance by liberal candidates.

RV polls are polls of "registered voters". Gallup uses the RV method as as a result Democrats tend to perform better in comparison to other polls. Since RV is the most common method, Democrats will tend to perform better in most polls. BUT only about 70% of registered voter actually vote. Sometimes it can be as low as 50%. It just depends on what is going on in the world. So you get a better idea of who is likely to win the election than using A polls but it's still iffy. So some pollsters use LV instead.

LV polls mean "likely voters". This is the most accurate measure of "who is likely to win an election", not "what do the people think" because it only considers those who through a formula are determined the be an active voter in the coming election.

The likelihood that someone will vote is based on a pretty basic calculation. Each polling agency will vary it a bit but it comes down to (P*B)+M > C; where P is the probability that an individual vote will change the outcome of an election, B is the tangible benefit someone stands to gain by their candidate winning, M is the non-material motivation factor and C is the cost in time, effort, and currency it takes to vote.

Now since P is basically zero then B is also basically zero and it comes down to whether M is greater than C. So, for example, if a person is not currently registered to vote their C value will be higher than a person who is currently registered to vote. Why? Because for the former it will take time and effort to register and that will raise the value of C over someone who has already registered. other things that weigh into C are things like how far the polling location is from their home, do they get time off work to vote, do they have a car, do they plan to be out of town on election day forcing the use of an absentee ballot, etc

M is determined by a lot of things; how important a critical issue is to you, whether you feel that voting is a "duty", whether you pay attention to politics in general, etc. It also considers your past voting history. How often have you voted in the last 4 elections for example.

So what they will do is after you have taken the poll, they ask you all these questions designed to calculate your personal M and C values. If M is greater than C they consider you a "likely voter" and your poll answers are considered. If M is less than C they consider you "unlikely to vote" and they ignore your responses.

Now because Republican candidates generally perform best in LV polls liberals like to argue that they are biased. No. Republicans perform better in those polls not because the poll is biased but because as it turns out Republicans are more likely to vote than Democrats and historically by a wide margin. This is precisely why large voter turnout tends to favor a Democratic candidate. While there are more Democrats in the nation, Republicans vote at a far higher percentage of their given population.

So are LV polls, like Rasmussen and SurveyUSA, "biased"? Well liberals cry that they are, but in reality if you consider an LV poll biased toward Republicans you must also consider an RV poll biased toward Democrats. It's also worth noting that according to the 2008 election results, our liberal friend Nate Silver (who the liberals love to cite), was forced to concede that out of the top three most accurate agencies, SurveyUSA and Rasmussen (both LV pollsters) were #2 and #3 respectively. So is it that LV polls are biased or simply more accurate? The data suggests the latter. Is it that RV polls are biased or simply less accurate? The data suggests that the former may be the reason for the latter.

This is also why when it comes to things like approval rating and voter identification, Rasmussen's numbers will show more favorable data in both categories for Republicans than Gallup's will. Rasmussen's data is based on LV while Gallup's is based on RV.

Now does this finally clear it all up for everyone?

Apparently, not.

You're gonna need to dumb it down for the slow kids.

 

[BluePhantom]

So we're talking about party identification, you claim that Rasmussen is more accurate, then suddenly when you're asked to actually substantiate,

then we weren't talking about party identification?

lol

Sweet Jesus. Let me slow this down to accommodate the logically challenged. While there is no way to prove conclusively what the true breakdown of party identification is in the nation, it's logical to conclude that since Rasmussen's data is weighted according to their party identification data (and don't even try...that's common procedure. Gallup does that too and you never hear any whining about them), and since Rasmussen's data in general elections has been notoriously accurate, that their party identification data must be reasonably accurate as well. Do I need to connect the dots between taking a shit and wiping your ass for you too?

 

[BluePhantom]

Apparently, not.

You're gonna need to dumb it down for the slow kids.

I guess so. I mean good Christ, my 18 year old daughter understands this perfectly. I don't see why it's so difficult for NYC

 

[NYcarbineer]

Shall I repeat the question?

If Rasmussen is a more accurate less biased pollster, how did he manage to give Bush higher than average approval ratings over the course of his presidency,
and now manages to give Obama lower than average approval ratings throughout his presidency?

Explain that

I already did in post #65, however, I anticipated your inability to grasp the connection and posted a more thorough explanation in post #68

Rasmussen was an outlier to Bush's advantage and he's an outlier to Obama's disadvantage.
That gives Rasmussen no credibility.

 

[NYcarbineer]

So we're talking about party identification, you claim that Rasmussen is more accurate, then suddenly when you're asked to actually substantiate,

then we weren't talking about party identification?

lol

Sweet Jesus. Let me slow this down to accommodate the logically challenged. While there is no way to prove conclusively what the true breakdown of party identification is in the nation, it's logical to conclude that since Rasmussen's data is weighted according to their party identification data (and don't even try...that's common procedure. Gallup does that too and you never hear any whining about them), and since Rasmussen's data in general elections has been notoriously accurate, that their party identification data must be reasonably accurate as well. Do I need to connect the dots between taking a shit and wiping your ass for you too?

Since Rasmussen's polls are producing outlying results, then you have to assume that either Rasmussen is wrong,

or the sum total of every other pollster is wrong.

Which is the more logical conclusion?

 

[BluePhantom]

Apparently, not.

You're gonna need to dumb it down for the slow kids.

I guess so. I mean good Christ, my 18 year old daughter understands this perfectly. I don't see why it's so difficult for NYC

Except let's be honest here. He understands perfectly. He is just engaging in typical debater tactics in an attempt to discredit Rasmussen because he has the typical liberal hard on about them.