But What About the Margin of Error?!

An MIT professor explains this and other tricky questions about polls.

Whichever way they go, we expect both the Massachusetts Senate race and the presidential race will be quite close. While some polls find both races nearly tied or at least well within the poll’s estimated margin of error, Five Thirty Eight’s Nate Silver says Obama has a 90 percent chance of victory and Warren has a 94 percent chance. Other data-crunching poll watchers draw similar predictions. For non-statisticians confused as to how Silver could be certain that the election will be narrowly decided but also just as certain that it will be decided for the Democrats, we reached out to MIT Political Science Professor Adam Berinsky. Here are some of his take-aways:

First, a poll’s margin of error is there because we can’t ask everyone in the country who they are voting for every day.

“We want to know what percent of the country favors Obama. One way to do this is to ask everyone in the country. This is really hard to do, so instead we can take a random sample of people from that population and ask them questions and get basically our best guess of what percent of people favor Obama.Where the margin of error comes in is that because we’re not asking everyone, we don’t know the truth but we have an estimate.

Let’s say 49 percent of people are going to vote for Obama. But we have a margin of plus or minus 3 percentage points. What that means is if we kept taking samples, 95 percent of the time, we’ll get an estimate in that range of 46 to 52 percent.

Now the reason we say 95 percent is that one out of 20 times we could get something really weird.”

The more you sample, the better … but only to a point.

“You see most polls sample around 1,000 people. [Because of the formula for calculating the margin of error], that’s kind of the sweet spot for polling. You get enough that you get a reasonable margin of error. But to get it down further, every 1000 people you do after that is just an incremental gain. And that’s why we don’t interview 10,000 people.”

[Ed note: People who paid attention in Algebra II, this is because the margin of error formula uses the square root of the sample size in the denominator and that graph sorta looks like this.]

Polls are useful, even when the difference between the two candidates is within the margin of error.

“It’s not statistically distinguishable from a tie. It used to be no one would talk about margins of errors. They’d just talk about polling results. But I think journalists have gone too far the other way. The polling numbers are still our best guess of the true state of Obama’s approval or the number of people voting for Obama.”

Poll aggregators are a better tool than any one poll.

“If it’s a close election, we’re never going to have some result outside the margin of error. We still have good information about what is the true level of support for Obama. Take advantage of the fact that. Years ago there were only a couple polls out now we’ve got 20 or 30 polls a day.

If I’ve got 40 polls showing Obama in the lead and two saying Romney is, though none of those has them outside the margin of error, that’s a lot of information. It’s going to be a close election, but Obama should win unless the polls have gone horribly wrong.”

 

The interview was condensed and edited a bit for length and clarity.