UK Election Polls - If Only We Hadn't Had the Enquiry

Another UK General Election has come and gone, with winners and losers amongst the survey firms as well as the politicians. Once again most people seem to be stunned by the result; while the adjustments made after the last election look like the key reason they didn't get this one broadly right. Who'd be a pollster?

Less than 24 hours after the results started to come in, you can if you wish already read a vast amount about what worked and what didn't, and critiques ranging from scholarly to poll-bashing - so we make no apology for quoting others liberally in our six-point summary.


1. There were a very broad range of final predictions. Some of them were bound to be wrong, even before we saw any results! In the event, the margin of 2.4% between the parties was at the low end of what pollsters said, but within the range. See Wikipedia.


2. Adjustments made to the initial polling data following the BPC / MRS inquiry into the 2015 errors actually pushed most of the companies in the wrong direction (see chart, courtesy of Ipsos MORI in the Huffington Post) - they would have been relatively accurate if they had not factored in 'Shy tories' and the tendency of young voters to stay at home, as Peter Kellner points out in an article for the Evening Standard. Arguably Kellner's title is rather harsh (General Election polls 2017: How the pollsters got it wrong) given that the article goes on to point out that a number of companies got things broadly correct:

With one exception the polls tried too hard. Their raw figures generally pointed to a bad night for Theresa May. Had they stuck with these numbers, they would have redeemed their reputation.

Instead, burnt by their failure to predict David Cameron's victory two years ago, they adjusted their data in the belief that many young Labour supporters would end up staying at home.

3. YouGov was the subject of an attack we reported just three days ago, by former Conservative Party election advisor Michael Moszynski, who said its forecast of a hung parliament was 'irresponsible'. Ketchup with those words, Mr Moszynski?

This method now becomes one to watch, at least, although commentators have pointed out that errors in the model cancelled each other out, perhaps a little fortunately. And it should be said that just because you have a new model and get one election right, it can't be assumed you're going to get the next one right, never mind all of them. SurveyMonkey did very well with the 2015 election and talked much about the death of traditional polling - they were quieter after the EU Referendum. Their later polls proved fairly accurate this time, so if two out of three ain't bad, they're in the running - but we do wish people wouldn't equate getting one election right with having discovered a magic formula.


4. The smart analysts of this and many other sets of stats generally look at trends rather than absolute figures. With this in mind, there was plenty of useful information in the polls, which clearly showed the narrowing of the gap right the way through the short campaign, and in particular following the manifesto launches - draw your own conclusions. And talking of smart analysts, news site fivethirtyeight.com published this article by polling legend Nate Silver as long ago (seems it now) as June 3rd:

So to borrow our phrasing from the U.S. election, when we said that Donald Trump was only a 'normal-sized polling error' away from winning the Electoral College, May's Conservatives are now only a normal-sized polling error away from a hung parliament. On average in the UK, the final polling average has missed the actual Conservative-Labour margin by about 4 percentage points. (This is twice the average error in US presidential elections.) If Labour outperforms its polls by that margin, Conservatives would win the popular vote by only about 3 points - and May would probably have to find a coalition partner to form the next government. If the polls were to miss by any more than that in Labour's favor, a variety of yet-more-unpleasant scenarios could crop up for May, including some where Labour leader Jeremy Corbyn tried to form a government instead.

5. More generally however, it's a brave man or woman who says they're confident about the outcome or the implication of political polls. As Silver says:

...These experiences have given rise to what I've called the First Rule of Polling Errors, which is that polls almost always miss in the opposite direction of what pundits expect.

By making any kind of forecast, you're putting yourself up to be shot at. And if in the old days there was something reassuringly linear about British political opinion, the picture only gets more, and more complicated, as David Cowling, former editor of BBC political research, points out in a good summary:

It seems that whether people voted Leave or Remain in 2016's European referendum played a significant part in whether they voted Conservative or Labour this time. Did the 2017 campaign polls factor this sufficiently into the modelling of their data?

If younger voters came out in bigger numbers, were the polls equipped to capture this, when all experience for many years has shown this age group recording the lowest turnout? (DRNO Editor: see some good analysis by Kantar of the age factor).

Also, the variations in voting across Britain at this election appear much more complex than in previous elections and that poses real challenges to polls seeking to reflect the national picture. We have seen great movements in Scotland; Labour taking Canterbury and Enfield Southgate but losing a core seat such as Mansfield and sustaining a big swing in Bolsover.

The future seems very challenging for the pollsters.

6. However we'd also echo Cowling's conclusion about their importance:

It is important for all of us that the pollsters succeed. If their reputation collapses then a space is created that will be filled by any number of spinners and fraudsters. Politics, like nature, abhors a vacuum and if polls are discredited then any number of people will step in to give their partial and partisan interpretation of events. National samples will be elbowed aside by the wisdom of tweets and Facebook friends.

I hesitate to call for inquiries - after all, we had one post-2015. But I think post-2017 we have to allow more time for proper scrutiny of the various methodologies. In 2015 there was great pressure for a quick answer: 2017 suggests there is no such thing.

In short, pollsters need to keep trying - and one final thought is that despite the perils they face, there's no sign they're giving up or that people are losing interest (as opposed to confidence) in them. In January, we reported that French newspaper Le Parisien had announced it would no longer commission its own polls, in the run-up to the country's Presidential elections, but would 'rely on journalists' feedback from 'working on the ground', along with references to external surveys'. British papers don't seem inclined to follow this French lead - the number of different published surveys for a UK election has never been so great.

So in answer to the question in our first paragraph, quite a lot of people would be and will continue to be pollsters, we suspect. See www.mrweb.com/jobs for the best list of current opportunities :)

Nick Thomas

All articles 2006-23 written and edited by Mel Crowther and/or Nick Thomas, 2024- by Nick Thomas, unless otherwise stated.

Register (free) for Daily Research News

Simple printable version of this article

Email this article to a friend