In his award-winning ESOMAR paper, John Kearon, CEO & Chief Juicer of BrainJuicer, explains his attempts to apply academic thinking on 'research markets' and the 'wisdom of crowds' to concept testing. The results, while still inconclusive, threaten conventional MR techniques - suggesting that a diverse, non-target group can predict success as well as a target sample.
Kearon's paper 'A Fresh Approach to Concept Testing: How to Get More Research for Less Time and Money' was inspired by the theories of James Surowiecki's best-selling US book 'The Wisdom of Crowds'. The book challenges modern corporate decision-making models, arguing that we should 'stop hunting the expert and ask the crowd instead', stating that under certain circumstances, crowds make better decisions than even informed individuals. For a crowd to be wise, he states, it must be diverse, independent and faithfully aggregated.
Surowiecki raises the possibility of using 'decision markets' or what Kearon calls 'research markets' to predict and make decisions. One successful example is the Iowa Electronic Market (IEM), where betting on the outcome of elections has proved significantly more accurate than conventional political polling. The IEM's election-eve predictions in presidential elections between 1988 and 2000 were, on average, off by just 1.37%. Similarly, the Hollywood Exchange web site - again based on betting - managed to predict 40 out of 40 Oscar winners - more than any newspaper pundit.
Kearon says that applying such 'wisdom of crowds' thinking to MR would 'reinforce the wisdom of using large, quantitative samples for decision-making' - but would of course also throw a huge question mark over the necessity of using targeted samples to predict in-market performance.
His first experiment, carried out in January for Unilever, consisted of comparing the results of two pieces of research:
All articles 2006-23 written and edited by Mel Crowther and/or Nick Thomas, 2024- by Nick Thomas, unless otherwise stated.
Register (free) for Daily Research News
REGISTER FOR NEWS EMAILS
To receive (free) news headlines by email, please register online