Tuesday, October 28, 2008

The Problem With Polls (and How to Read Them)

The British Prime Minister Benjamin Disraeli is famously reputed to have said that statistics were worse than 'lies' or even 'damned lies'. That said, most modern politicians and journalists know that opinion polls can prove remarkably accurate indicators of how an election will play out.

The problem with polls is how they can often be misrepresented. Take the example of a journalist who writes that the White House race is 'tightening' because Obama's lead over McCain is (say) 50-44, when last week it was (say) 52-44. In fact, such numbers are often what pollsters dub 'statistical noise' - normal variation in how well a sample will represent the population it is taken from.

Most polls have a 'sampling error' of around 3 points (plus or minus). This means that if a poll shows Obama on 50 percent and McCain on 44 percent, the Democrat likely has the support of between 47-53 percent of voters, and the Republican the support of around 41-47 percent.

An eager McCain supporter might point out then that, according to these figures, his candidate might have as much as 47% and Obama as little as 47% - making the race a virtual tie. Yes, but the 47-47 split is only one possible outcome, according to the above figures, among many others:

Obama 47 - McCain 47
Obama 47 - McCain 46
Obama 47 - McCain 45
Obama 47 - McCain 44
Obama 47 - McCain 43
Obama 47 - McCain 42
Obama 47 - McCain 41

We could reproduce the same table above with Obama on 48, compared to McCain's 47, 46, 45, 44, 43, 42, or 41 percent share. And again with Obama on 49, or 50, or 51, or 52 or 53 percent. Taken all together, of the 49 possible combinations McCain ties Obama in just 1 and is behind in the other 48. Worse for the McCain supporter, 21 of the 49 possible combinations have McCain's deficit at even greater than 6 percentage points. In 39 of the 49 combinations, Obama still leads by at least 4 points.

McCain really might be neck-and-neck with Obama (47%-47%), but he could also really be as much as twelve points behind (53%-41%). The fact that the real figure is probably somewhere between these two extremes is not going to be any comfort for Republicans.

3 comments:

Steven Piziks said...

How much media attention is the American election getting over there? In the States, of course, it dominates the news and other airwaves to the point where many of us are getting sick of hearing about it.

Rehan Qayoom said...

Ach, Statistics. I was too bad at bottom-set Maths at school, wasn't allowed to sit my GCSE Maths exams, had to take a numeracy test and failed that too -

Jess said...

It was not much of a comfort, no.

And although the elections are over, and I heave a great sigh- I suspect it's just one of many elections, this being my first, where I am going to lose many bets, I am still hopeful for the future of the Americas...