Pollster Points to Track Record

As a close-run and hard-to-call US presidential election draws near, Harris Poll Chairman Humphrey Taylor acknowledges that there is no one 'right way' to do surveys and that 'the first rule of credibility is to admit to fallibility', but says the track record of polling companies is the best reason to trust them.

The 'fallibility' quote comes from company founder Louis Harris, who became famous as John F. Kennedy's pollster in 1960 campaign. In a statement released last week entitled 'Why Should We Believe The Polls?', Taylor says pollsters preparing for their final election forecasts would do well to remember these words.

Taylor says that the best known reason for inaccuracies is sampling error - 'this is what the media often, and incorrectly, describe as the 'margin of error' (e.g., 'plus or minus 3 percentage points'). However, there are many other sources of error in polls which, unlike sampling error, we cannot quantify, and which probably contribute much more to the inaccuracy of polls than sampling error'. He lists, among others: sample design; respondent selection method in households with two or more adults; time in the field; the wording of the questions and the order in which they are asked; the quality of the interviewers where used; whether attempts are made at 'refusal conversion'; and the definition of 'likely voters', much discussed of late and in Taylor's words 'probably the trickiest issue of all, given that so many people who say they will vote, do not always do so'.

Taylor points out that no two polling firms use the same methods. He adds that 'Even if a polling firm is both skillful and lucky (and, yes, luck is a factor) in producing a perfect estimate of how people are likely to vote, it can still 'get the election wrong' if candidate preferences or the likelihood of voting changes after its final poll is completed as happened, most notoriously, in the Truman-Dewey election in 1948'.

'Critics of the polls may well ask, given all these possible sources of error, why should anyone believe the polls? The answer is 'their track record'. Since 1956, the average error of all the national polls on the gap between two main candidates' share of the votes has been 3.3 percentage points, and in 2000, the average error was 2.2 percentage points. Not perfect, but pretty good. And they do better in some years than in others. So pollsters should always run scared.

'To re-phrase Winston Churchill's famous remark about democracy: 'Opinion polls are the worst way of measuring public opinion and predicting elections - except for all of the others'.

The full text can be found on the company's web site at www.harrisinteractive.com

All articles 2006-23 written and edited by Mel Crowther and/or Nick Thomas, 2024- by Nick Thomas, unless otherwise stated.

• Harris Interactive Inc.
• All news
• MrWeb info