Political polls are way better than you think. Here’s how to improve them more.
There’s an old cartoon on the website of software company Zoho where two pollsters discuss their survey results.
“This is interesting,” one says. “70% of the respondents to our survey said they don’t respond to surveys.”
The joke is more relevant than ever.
A week ago, poll results from The New York Times, The Philadelphia Inquirer and Siena College showed Donald Trump with significant leads in several swing states: Nevada, Georgia, Michigan, and Arizona. Two more swing states — Pennsylvania and Wisconsin — were within the polls’ margins of error.
Chicken Littles across the nation began clucking wildly, with one USA Today political science contributor calling for Biden to drop out so Democrats can nominate someone else in an effort to beat Trump.
But how much should one trust political election polls? After all, Morning Consult polls show Biden faring better against Trump than last weeks’ numbers, but in a very close race with no clear advantage.
2016 election polls: Were they really so wrong?
Some Americans have bad memories of the 2016 election, particularly those who believed — in part because of polling — that Democrat Hillary Clinton would rout Trump on Election Night.
Of course, that result most definitely did not come to pass.
But were the polls really so inaccurate then?
ALSO READ: What Trump's weird WWE Hall of Fame speech tells us about his presidential debate strategy
Scan the Internet for articles on polling for the 2016 election, and you’ll see claims that polling organizations really screwed it up.
“Pollsters have made several major miscalculations over the past several election cycles, often underestimating support for Donald Trump,” claims Rebecca Picciotto with CNBC.
I even attended a panel at the Southern Political Science Association conference in early 2017 which focused on how wrong the polls were. Those sentiments were echoed at a similar conference at St. Anselm College in New Hampshire.
In reality, the polls eight years ago were pretty good. They were better than the historical average from 1936-2020.
The RealClearPolitics average of presidential polls, for example, showed Clinton winning the popular vote with 46.8 percent of the vote to Trump’s 43.6 percent. In the end, Clinton prevailed, 48.2 percent to 46.2 percent, in the national popular vote.
But the success of the 2016 election polling was undone by one obvious flaw: The national vote doesn’t pick a president. If it did, we’d have Presidents Samuel Tilden, Al Gore and, yes, Hillary Clinton in our history books.
At these conferences, I kept remarking that we didn’t have enough polling at the state level — particularly the swing-state level — where the mechanics of the much-derided, but ever-important Electoral College are at work. I remember looking at three polls for Wisconsin in 2016 — one from September, one from October, and one from November. At best, their results were inconclusive and hardly fodder for predicting who might win.
ALSO READ: Why Trump’s strength is illusory
Most critiques of presidential polls are focused on any of several factors: sampling error, overconfidence in results, even undercounting “shy” Trump voters — those who plan to cast their ballots for the 45th president but don’t want to acknowledge it publicly.
These critiques, however, are themselves imperfect.Trump voters are many things, but “shy” is rarely one of those characteristics, as the nation has witnessed for the past nine years, particularly on Jan. 6, 2021. And if inaccurate polling is simply undersampling Trump supporters, where was the “red wave” everyone was convinced would destroy the Democrats in 2022?
Another critical factor: An individual poll is a snapshot in time. Its utility doesn’t so much come from what it says about who’s up and who’s down in a given moment. The value of a poll is what it helps explain about a candidate’s waxing or waning over the course of an election, including prospective voters’ views on issues and events of importance to the results of a given election.
How polling could be better
At the presidential level, America needs better and more frequent scientific polls that reflect state-level results in key swing states. Unfortunate as it may be, how voters vote in Idaho or Massachusetts or Mississippi or Hawaii won’t matter — thank you, Electoral College — to whether Biden wins a second term.
And that means finding large enough samples in states that will determine the presidential race outcome, such as Wisconsin, Michigan, Arizona, Georgia and Pennsylvania. You often see some polls of statewide residents with fewer than 750 people in the sample. Other polls have fewer than 500 respondents on a given survey, with large enough margins of error to render the results all but meaningless, unless one candidate is furlongs ahead of another.
Part of that problem is the “horse race” dimension, where news organizations are more interested in the “score,” or how close a contest is in the polls. That comes at the expense of providing voters with a sense of how their fellow citizens view candidates’ positions on a myriad of issues, ranging from health care to immigration to economics.
ALSO READ: Democratic congressman commits massive stock law violation
We also need better statewide polling in states that are close enough to possibly flip in 2024, or that could genuinely be in play another four or eight years from now — states such as Minnesota, Virginia, North Carolina and even Texas.
Those polls should have bigger samples and smaller margins of error, not just to be more representative, but to tell us what a smaller subset of state residents are thinking or preferring. And the questionnaires shouldn’t be so long as to turn off those asked to provide their opinion. Pick your best questions, and don’t include ones that are meaningless or redundant.
There is hope, even for today’s surveys
Finally, never rely on the results of a single poll. There’s strength — and knowledge — in numbers.
It helps to study the average of a group of polls, rather than rely on the findings of a single polling firm.
Using data from the University of California Santa Barbara’s American Presidency Project (and a RealClearPolitics (RCP) update from the 2020 election), I determined that the RCP average for a combined variety of polls came within 2.9 percentage points of the real total for both candidates combined (above or below their final total). That’s often within the margin of error of a poll.
That’s better when compared to the average error rate when using only the Gallup polling averages from 1936 to 2012, no matter how respected that company is. Gallup was off an average of 5.16 percentage points for both parties’ final totals, though the company did get better over time.
So no, polls aren’t “fake” news, and they are getting better. But with an increased focus on states, bigger sample sizes and employing a variety of polling organizations tackling the problem, polls will get better yet. We can boost polling accuracy, better understand the issues driving elections and more completely understand what to expect on Election Day.
Year | Absolute value of poll deviations from the final election result for RealClearPolitics average |
2020 | 3.1% |
2016 | 4% |
2012 | 2.9% |
2008 | 1.6% |
Year | Absolute value of poll deviations from the final election result for Gallup polling |
2012 | 4.9% |
2008 | 4.4% |
2004 | 3.7% |
2000 | 2.5% |
1996 | 3.1% |
1992 | 6.4% |
1988 | 4.2% |
1984 | 0.6% |
1980 | 6.7% |
1976 | 3.1% |
1972 | 1.8% |
1968 | 1.1% |
1964 | 5.4% |
1960 | 1.8% |
1956 | 3.6% |
1952 | 8.7% |
1948 | 9.4% |
1944 | 4.5% |
1940 | 5.9% |
1936 | 12.9% |
John A. Tures is a professor of political science at LaGrange College in LaGrange, Ga. His views are his own. He can be reached at jtures@lagrange.edu. His X account is @JohnTures2.
The grave of the author’s paternal grandmother rests at Osborne Memorial Cemetery in Joplin, Missouri. (Max McCoy/Kansas Reflector)