David Day David is President and Global CEO of Lightspeed Research. He joined the company in 2006 as CEO, Europe to oversee the expansion of the European online panels and market research business. Read the full biography here.

Online's Balancing Act
Panel providers must tread a path between investment in quality and competitive pressure

10 March, 2010

Introduction

The online access panel business is now a decade old and accounts for more than 20% of all data collection. Yet despite its increasing market share and relative maturity, the debate surrounding online panel quality and standards refuses to go away. Whilst perhaps not openly discussed, the truth is all research companies face a dilemma at some time or other between the demands of their businesses and the standards required by the industry. I believe this is more pertinent for the online panel business than other areas of the market research sector.

..the truth is all research companies face a dilemma at some time or other between the demands of their businesses and the standards required by the industry.

The move online was driven by the desire of market researchers to increase sample sizes whilst at the same time reducing costs. It seems that even after all this time, these desires are still at the top of the mind of many researchers, particularly in these straightened times.

Over the years, as more and more companies have entered the online access panel markets by design or accident, survey research has become a commodity to many clients and prices have fallen dramatically. I would argue that this has put pressure on quality standards in the industry.

……but some are more equal than others

While panel providers are almost innumerable, high quality panel providers are not. It is only the bigger players, Lightspeed included, that can afford to invest the significant sums needed on an ongoing basis to address quality issues and establish best practice. Whether its technical measures or rigorous checking by research specialists, quality costs.

Compare the meerkat?

For me as CEO of a global panel business, the challenge remains balancing the need to run a viable business that delivers a premium product and service that clients will want to come back for, and at the same time remaining true to the standards that the industry sets and clients demand. It is not just about us getting the most business out of our clients or the most completes out of our panellists. We need to think longer term than that, and ensure we have a sustainable business.

Some research buyers simply ring round to a number of different companies and get a price per complete then choose the cheapest rate. Quality is rarely questioned and the difference between the panel providers even less understood.

Size isn't everything

Five years ago the quality of an online panel was determined by its size. To be totally frank, coming to Lightspeed from the internet audience measurement business, I was well aware of the business of ‘bigging up’ panel sizes. But as the industry has evolved, so should the questions research buyers ask of their panel provider. Some of the panel sizes I hear quoted by other providers are quite frankly ridiculous – and irrelevant.

But for me, it is much more fundamental – and worrying. Boasting about large panel sizes belies a serious failure to understand the business of research. The killer question to ask a potential panel provider is not ‘how big is your sample?’ it is ‘what response rates do you achieve?’ and ‘how deeply do you screen your panel’ and ‘what do you know about them?’ The move to ‘samples sizes of one’ in qualitative research may perhaps never be taken that seriously, but it does underline the view of some in our industry that big isn’t always beautiful. Or representative.

Know thyself; know thy panel

At Lightspeed we have invested significant time and resource to know our panellists. And whilst unique machine ID, postcode verification, IP checking, identifying and filtering out robots and the de-duplication of panel members may not sound like the sexiest of tasks, they are the tools that help us practice what we preach: a panel we know, profiled, responsive and relevant. The old adage ‘you get what you pay for’ may be a cliché, but when it comes to online panels, it is spot on.

The good news is things are changing. Clients are beginning to ask questions about online panel quality and in turn research buyers are doing the same. Access panel providers will have to start demonstrating quality in addition to claiming it. They need to be open and honest and willing to have the conversation, without resorting to marketing spin.

It takes two to tango

Everyone in the industry is responsible for ensuring that the research we manage is reliable enough for our clients to base their business decisions on. At the minimum, everyone should complete surveys themselves before sending them to panellists to double-check the entire process, not only in terms of the survey working technically, but also to understand the ‘survey experience’ the respondent is facing. Respondents are a valuable resource. They certainly are not a commodity.

Everyone in the industry is responsible for ensuring that the research we manage is reliable enough for our clients to base their business decisions on.

Not all panel companies will challenge their clients on the quality or length of a questionnaire, let alone turn away a piece of work if its borderline feasible. In these challenging times perhaps it's not surprising given the ranks of competitors out there. But from a quality and sustainability point of view we should all be prepared to challenge poor practices wherever we see it in the research process. If we don’t it will be our clients that will punish us.

There is a cost to high quality, but the cost of poor quality to the industry is far higher.

David Day

Comments on this article

Want to share your thoughts...?

Want to share your thoughts?

NOTE: Please note that this board is moderated, and comments are published at the discretion of the site owner.