Report on the ASC Conference

The Association for Survey Computing’s Sixth International Conference was held in conference halls and dormitory accommodation belonging to Bristol University, and the academic setting was entirely appropriate for a conference where the papers are always entertaining, educational and actionable.

With a delegate list of over 100 this was a bumper crowd for a ‘techie’ conference and the eclectic mix of academics and market research superstars was no doubt due in some part to the halo effect of two other conferences in the same week: ESOMAR in Amsterdam at the beginning of the week, and the Triple-S conference (also at the university) the previous day. Teresa Lynch, our Features Editor, having missed both the others attended the ASC for a nice nerdy conference.

The first key note speaker was Tom Ewing who had managed (accidentally) to generate all kinds of gossip by choosing to change jobs in this conference-packed week. By the time he gave his talk on Thursday everyone knew that he was moving from being Kantar’s Social Media Knowledge Leader to a new role at BrainJuicer (Digital Culture Officer as it was revealed on Friday). In his paper entitled ‘Twilight of the Respondents’, he looked at current concerns about how MR is interacting with its respondents.

Ewing used the example of three types of stories we in MR tell ourselves about respondents: the ghost story – this most commonly accepted story says that the respondents have been so ill-treated they have gone, never to return; the sci-fi story - in which he compared habitual non respondents to ‘the dark matter of MR’ and social media as ‘first contact’; and lastly the spy story – not Bond but Tinker Tailor, full of paranoia and mistrust and the belief which is the backbone of neuroscience and behavioural economics that we can’t trust respondents to self report. This last story also ends with the respondents gone forever.

Ewing sees all this leading to a post-respondent era where we will have data brokerage which values some respondents more than others and gives individuals with a high social media credit rating perks for sharing their opinions [please let this be earned by posting a low volume and not a high one...]. We will also build data ecosystems, leaving them alone to see what happens and find applications for all the data we have already collected. He also predicts that market researchers will start to tell new and more compelling stories – but warns that we need to avoid (because of competitors in our space) becoming the body in the library in our own murder mystery.

This really was a keynote for the first day with talks following on ‘Engagement, Consistency, Reach’ from Alex Johnson and Guy Rolfe from Kantar and ‘Online research – Game on!’ from Jon Puleston of GMI. Jon is involved in researching gamification as a methodology for enhancing respondent engagement (see an article by him here). Puleston always presents compelling figures for the data ROI on gamification: for example asking respondents ‘what brands of deodorant come to mind’ elicited an average of two answers whereas ‘how many brands of deodorant can you guess?’ yielded an average of more than six. Along with his suggestions he also warned us not just to drop random game elements into questionnaires – this prompted some agreement from the floor with the audience remarking on ‘bad’ gamification they had seen.

The ASC, of course, is a very broad church and some of us were drawn to the Chapel (see picture) where alternative themes of a slightly more esoteric nature were running. These dealt with ‘Visual and cultural engagement’ and ‘Improving the quality of achieved samples’. In the first of these themes Steve Hales from Synovate talked about ‘the third question type’ – asking respondents to answer using only visual images which had been pre-coded for their significance - and in the second Tony Dent of Sample Answers talked about sample bias. In the designated place of worship there was also a paper from Andrew Grannell of Statistical Solutions Ltd on ‘Using multiple imputations to adjust for survey non-response’ – totally ASC.

Also making an appearance after lunch was Ed Ross with some feedback on the Triple-S conference. This mostly involved questioning the next steps for the data standard which is vital for data processing and developers. Typically, during the conference there had been some talk about whether a particular piece of code would work and two delegates had worked on it that evening and produced the finished product in the morning.

Winding up the first day was the second of the keynotes, given by Jeffrey Henning of Affinova. The original title of his paper had been ‘Crowd Shaped Surveys’ but he changed it to ‘Dawn of the Respondents’ as a counter point to Tom Ewing’s opening presentation. He said we had been telling ourselves Ewing’s ghost stories since he joined the industry in 1987. Henning also suggested that we had abused the respondents somewhat and we should stop showing the last respondent in a study exactly the same questionnaire as the first one that we probably didn’t need to ask some of these questions and we should consider how much open ended material we wanted to collect. He pointed out that ‘more than any other industry we are cognisant of the problems with our methodology’.

Dinner was a jolly affair held in the old style refectory, seated on wooden benches. No one sat at the high table. This was followed by Mike Ducker’s Jazz Band in the bar.

Next day we were treated to surely the most engaging keynote of the conference, from Glen Watson of the Office of National Statistics (ONS) who talked to us about the 2011 Census. Absolutely fascinating stuff, from the department's attempts to reach young men (adverts in NUTS magazine, pub crawls, and sticking up posters in gents toilets), to dealing with a hoax about a leak which made the national press, to cartoons which read ‘I’m tweeting that I object to disclosing personal information’.

The numbers were staggering: £500m costs; 26m questionnaires; system set to withstand 180,000 concurrent users online; 700,000 completed online on the day; 1.7m calls to the 500 person call centre and 3m visits to the online help. All the government’s targets were met except they had hoped for over 20% completion online and got only 16%. Watson also presented a preview of how users would be able to access the data with interactive swipe maps and access to the API to produce survey specific output. When asked about the waste of paper he commented that it was equivalent to 3 days worth of the Daily Mail.

Back to the Chapel for ‘Using metadata for reporting: XSLT beats Excel (and everyone lives happily ever after)'. This was presented by Steve Taylor of We Work With Data, who when asked about the more esoteric points was happy to concede that he was merely Paul (or maybe even Ringo) to Mike Trotman’s John and got Mike to field them from the floor.

The Association’s AGM apparently broke all records after lunch by lasting less than 5 minutes and assured the assembled that the ASC is solvent and does not lack for officers.

Using a word completely new to most of the audience (Infoveillance) Joe Murphy of RTI international presented ‘Twitter feeds and Google search query surveillance: Can they supplement survey data collection’ This paper was about a project which attempted to prove that trends which could only be reported historically by government statistics could be quantified earlier by Twitter and Google. Murphy showed that the case was pretty much not proven and that the hypothesis needed further investigation with the only obviously non PowerPoint presentation at the conference.

James Turner of ICM finished off by making the case for the survey, 130 years old and going strong, which brought us as far away from the first paper’s ‘twilight’ as possible.

This is just an overview of a very full conference. Some ideas that were touched on more than once were how badly market research had treated survey respondents in the past, how important it was to chose the correct tool for both data collection and analysis and how nobody had a good word to say for sentiment analysis tools (best figure quoted 60% accuracy, worse figure quoted 49%). The papers were substantial and eschewed the ‘smoke and mirrors’ effects type of presentation. There were papers that could be expected at the ASC such as those on metadata and papers that were a bit more unexpected such as some of the social media pieces. It was difficult to make decisions about which sessions to attend and overall it was a very stimulating two days.

Teresa Lynch

Comments on this article

Want to share your thoughts...?

Want to share your thoughts?

NOTE: Please note that this board is moderated, and comments are published at the discretion of the site owner.