It’s snow problem!

Sheila Wilson while out sledging

Sheila Wilson while out sledging on the outskirts of London last Thursday

Europe is gripped by the hardest, snowiest, coldest winter in decades. Most of the UK, particularly the London area, typically has mild winters with an occasional dusting of the white stuff.

With Londoners unprepared, the city’s transport network has siezed up, many schools have closed and hoards of office workers remain stuck at home.

At meaning, we work mainly at home anyway, beacuse it is more productive and cost effective. It also means that whatever the weather, we’re still here to help our customers. So, we get to enjoy a bit of sledging and still carry on business as usual!

Panel crazy

Just before the end of last year, I decided to join one of the online panels to get a better understanding of their workings. I have to say that the whole experience has left me open-mouthed in shock.  In the short period since I have been a member, I have encountered error after error as well as, frankly, poor questionnaire design and basic scripting errors.

Typical errors I have experienced are:

  • Emails in various foreign languages inviting me to participate in surveys
  • Numerous broken links in emails
  • Requests to enter passwords that I don’t have
  • Another time, I clicked the link and got a message saying ‘this survey has not started yet’. I went back later and clicked the link and got a message saying that I had completed.

For those few surveys where I have not been screened out, I have encountered numerous impossible lines of questioning. There never seems to be a way of skipping a question, so your only options are to just click anything (to make sure you get your payment for completion) or to give up.  Some questionnaires allow you to make a comment at the end, so you can at least explain that you were forced to give an opinion on the personalities of different washing up liquid brands, even though you have no opinion on the matter because you regard washing up liquid as a commodity and just buy the cheapest, irrespective of its name, color, perfume etc.

In another survey, which was about radio stations, I was presented with a list of radio stations and asked which one I had most recently listened to. I picked a talk and news station. From then on, I was asked endless, detailed, non-applicable questions such as whether the presenters interrupted the music or how fun the competitions were.

And perhaps, even worse, I have recently seen a question which asked me which of a list of brands I had  bought. The one I had bought was not on the list and there was no ‘other’ option! Again, I was forced to select an incorrect answer or give up.

I did think that perhaps I was just biting into the one bruised apple of the basket, but coincidentally, I spent New Year’s Eve with a friend who is managing a new panel.  She has worked with numerous panel suppliers and was not in the least bit surprised by my tale of woe.

Having had this discussion with my friend, I was wondering whether market researchers are accepting unrealistic deadlines. Clients know that with the technology we now have, it is possible to script a project and complete the fieldwork online in a matter of hours. But perhaps they don’t know about the amount of thought, effort and creativity that is required for decent questionnaire design and in particular thorough quality control. There’s no point in conducting a survey if the questions are impossible to answer honestly and accurately. Presumably, some important business decisions are based on some rather dubious survey results.

What I’d like in my Christmas stocking

I’m hoping that Santa is going to stuff my stocking with a case of champagne and one of those glittery, sequined dresses that are all over the shops this year.

Apart from that, I’m wondering if he has the power to make the following magically appear in my office over Christmas:

1000 names and email addresses

I’d like 1000 names and email addresses of senior market researchers from around the world.  At meaning we conduct an annual online survey about market research technology and every year it becomes a little bit harder to achieve our goal of 230 completes, which should not come as a complete surprise since as the survey has pointed out in both of the last two years that ‘falling response rates’ is the top challenge the industry faces.  This means that one of my personal top challenges is building en ever longer list of people to invite to take part in the survey.

In my gift-wrapped Christmas box of sample, I’d particularly like some from France because, even though we translate the questionnaire into French, for some reason, our response rate in France is far lower than everywhere else. In 2008, 7% of our French sample completed the questionnaire, whereas in other countries this figures was typically between 10 and 20%. And I’d also like to ask Santa for Japanese sample because we quite simply never get enough of it!

And why bother Santa with this awesome task, when we have panel  providers, I hear you cry. Aren’t they ideal for wheedling out those hard-to-find groups? Yes, but this is just too specific even for them.  The thing is, our survey is about trends and issues in the use of market research software, so while we are happy for the IT boss, the Research Director or even the CEO to complete the questionnaire, it is no use if the HR, Finance or Marketing Manager participates (since they have nothing or very little to do with MR technology issues). We have had some success with panel providers in the past, and believe me, they have certainly tried hard, but the number of completes they brought in, while extremely valuable, never quite matched their effort.

A new piece of tabulation and charting software that hasn’t yet been written

Talking of our annual technology survey, I don’t know if elves are good at programming, but I would really like a new piece of market research software that produces a few cross tabs and publication-ready charts without having to endure a week of blood, sweat and tears to create them.

Once we have finished the fieldwork, Tim Macer and I have an annual battle with Microsoft Excel, which we do always win, largely thanks to Tim’s superior level of skill with the aforementioned package.  We have tried some market research specific packages, but have always fallen back on Excel because that’s what we have to provide the charts in to the various publishers… To be honest, Excel is not setting the bar very high, so maybe Santa’s little helpers can come up trumps.

So, champagne, a sparkly dress, 1000 names and email addresses and some brand new software. I think that is a modest wish list.