Online Survey Addiction Skewing Results?

Can results from online panels be trusted? Gian Fulgoni, Chairman of US-based comScore Networks set alarm bells ringing at this year's CASRO Conference with statistics about three-a-day survey respondents; but panel firms say the industry is taking a mature approach to the issue.

While speaking at this year's CASRO (Council of American Survey Research Organisations) conference in Los Angeles in October, Fulgoni presented figures generated from his firm's research, which highlight the problem of skewed results caused by panel respondents who regularly conduct online surveys three or more times each day.

comScore's analysis shows that 1% of the top ten online survey panels produce 34% of the completed questionnaires. The trend is for members of the top ten panels to join an average of five or more other panels; which is hardly surprising given the generous membership incentives offered by some MR suppliers.

However David Day, European CEO of Lightspeed Research, told DRNO he does not believe this situation is endemic in Europe. 'This piece of research indicates that the online panel industry is growing up, clients are beginning to ask more questions about sample quality, and there's now a greater need to introduce robust recruitment policies. We're confident that with the many checks which Lightspeed Research has in place to ensure that panelists are supplying accurate data, the possibility of respondents actively distorting the results is pretty remote. It's now up to all of us in the industry to share our processes and be transparent about how we manage data quality. We need to set the bar higher to ensure we maintain customer confidence.'

Outlining results from a study which Research Now has conducted into the issue, Kevin Goldberg, Director of Development and Panels, confirmed Day's belief that multiple panel membership is much less common in Europe than it is in the US. 'Through the enforcement of panel rules, we prevent panelists from completing multiple surveys. We do not use leading invitations where panelists are given the screener information they need to qualify, and we do not have member's areas where panelists are encouraged to take as many surveys as possible. Our rewards are made as a token recognition of a panelist's time, which means that nobody can get rich or earn a living by taking part in our surveys.'

Lightspeed Research and Research Now are online at www.lightspeedresearch.com and www.researchnow.co.uk . comScore is at www.comscore.com .

All articles 2006-23 written and edited by Mel Crowther and/or Nick Thomas, 2024- by Nick Thomas, unless otherwise stated.

• comScore, Inc.
• All news
• MrWeb info