Dr. Brian Arbour, 2nd from left
For many, the 2016 presidential election results came as a total shock, but for those well-versed in how national polling works, Donald Trump’s win wasn’t that unexpected. Although the surprising 2016 results and a weak down-ballot performance by Democrats in 2020 have prompted soul-searching among commercial polling firms and a spreading mistrust for polls among the public, election scholar and John Jay Associate Professor of Political Science Dr. Brian Arbour says that national polling isn’t as broken as we might think.
Instead, Arbour says, “polling in both 2020 and 2016 wasn’t that far off from what we’ve seen traditionally. [But] there’s a lot more attention on the polling today then there was, and on the perceived accuracy of polling.” He suggests that the public has high expectations of prediction accuracy thanks to sophisticated reporting on poll numbers, but—apart from political scientists like himself—consumers rarely bother to read the fine print or check poll methodology. Combined with races that are closer than ever, the copious information available to casual observers and circulating on social media has really skewed perceptions about how to evaluate and understand the data.
“The popular vote for President and the popular vote for the House continue to get closer and closer. There are something like less than 20 districts where Trump won and a Democrat won in the House, or Biden won and a Republican won in the House,” says Arbour. With national polarization so rampant, the number of voters who might change their minds or swing an election is shrinking. Because the country is so evenly divided between the two major parties, these swings might change a result even within the normal margin of error inherent to polling of all types, which is what happened in 2016. Arbour emphasizes that polls should be taken with a grain of salt, and a greater understanding for how probability works. “As much as we rely on these things, even when they do everything the right way with a 95% confidence interval, they are going to get it wrong sometimes,” he says. “When a polling number looks better than you think it is for your side, look for more people and how they’re talking about it.”
Arbour also hopes that some of the current factors that have undermined polling accuracy might be on their way out. The COVID-19 pandemic had a huge effect on the economy, which Arbour cites as the biggest factor affecting presidential reelection chances; also, Democrats may have been more likely to stay quarantined at home, making them easier for pollsters to reach by phone. Another unusual factor is what he calls the “Shy Trump phenomenon:” because former President Trump is such a polarizing figure, some likely Trump voters may have felt reluctant to share their intentions. “The 2018 polls were better than the 2020 polls,” Arbour says, referring to their accuracy. “That’s probably an indication that it’s a Trump-specific issue, so maybe that means we don’t have a long-term problem.”