The Self Made Pundit
Tuesday, November 05, 2002
THE POLLS’ DIRTY LITTLE SECRET: The polls have a dirty little secret. Put aside for a moment all those precise numbers listed as the polls’ margins of sampling error and confidence levels. The secret is that the polls’ election day forecasts are based on a large amount of guesswork.
While polls trumpet that they have 95 percent confidence that their margin of sampling error is 4 percent or less, what they don’t emphasize is the amount of guesswork that goes into determining the group of voters to poll as “likely voters.” At late stages in campaigns, polls typically focus on “likely voters,” not registered voters, since large numbers of registered voters do not vote in any given election. The guesswork comes in deciding what questions to ask to identify those “likely voters.”
Different polls use different criteria to select these groups of “likely voters.” For example, the Ipsos-Reid/Cook Political Report Poll identifies “likely voters” as those voters “who say they are extremely likely to vote.” By contrast, Gallup labels respondents as “likely voters” through “a series of questions measuring current voting intentions and past voting behavior.”
This explains why polls of “likely voters” can be widely divergent. The most recent polls of “likely voters” show anything from a Democratic lead in the generic Congressional vote of two points to a Republican lead of seven points. The reason these polls are getting different results is that they use different criteria to select these so-called “likely voters.” They are polling different groups of people.
Thus, when Gallup says that it can say with 95 percent confidence that the margin of sampling error for its poll of “likely voters” is 4 percentage points, it is only expressing a degree of confidence in the sampling the views of people who have those “current voting intentions and past voting behavior.” Gallup and the other pollsters express absolutely no confidence level that the respondents they have labeled as “likely voters” are actually representative of those who will actually go to the polls. Due to the subjective criteria used to identify “likely voters” there is no measurable “margin of error” for whether actual voters are being accurately predicted.
Pollsters’ selections of “likely voters” can be particularly unreliable since they fail to measure the effects of Get Out The Vote (“GOTV”) drives by the political parties. If the parties had roughly similar GOTV drives, this failure might not make a big difference. But the parties are not equal in this regard. The Democrats have been far more successful at GOTV drives in recent elections. While Republicans have claimed to made advances, the Democratic operation still appears to be far superior.
The failure to measure the effects of GOTV drives is one reason that most polls failed to predict in 2000 that Gore would win the popular vote and that the Democrats would gain five Senate seats. Interestingly, Zogby, one of the few pollsters to correctly predict that Gore would win the popular vote, now has the Democrats winning the generic Congressional vote by 51 to 49 percent, while Gallup, which wrongly forecast Bush as the popular vote winner, now has the Republicans ahead in the Congressional vote by a margin of 51 to 45 percent.
I think Gallup is seriously underestimating turnout this year. Gallup forecasts that voter turnout will be 35 percent, which is less than in each of the last four midterm elections, which had turnouts that ranged from 36 to 39 percent. Since Gallup and other pollsters agree that the Democrats lead among all registered voters, any underestimation of turnout tends to underestimate the Democratic share of the vote.
Voter turnout today will probably be better than the polls are assuming, in large part due to the Democrats’ GOTV. States that have early voting (with voting starting in late October) have been reporting that turnout (at least of early voters) is up significantly this year. I suspect that tonight will be a far brighter night for the Democrats than many pundits are predicting.