Author Archive

How to read software websites

A fortune teller's crystal ballI regularly visit the websites of the market research software companies.  They all have the most “powerful”, “flexible”, “easy-to-use”, and “open” software solutions. If you are looking for some new kit, especially if it is something they all offer, like Web interviewing, CATI software or analysis and reporting tools, the choice can seem overwhelmingly daunting. So, how do you step out of the marketing fog and compare them a little more objectively? Obviously, you are not going to be able to make a final choice by looking at websites alone, but it would be nice to use websites to narrow the field a little – by looking at exactly what they do say, and sometimes by looking at what they don’t say.

Even creating a ‘longlist’ is difficult. (And I do truly know because at meaning we often help companies do exactly this). One of the key differentiators, cost of ownership, is never on websites! Although I guess the software companies would argue that it is not how much it costs but how much money it makes you that counts…. and that is never on websites either!

There are probably several hundred Web interviewing systems on the market now and over a dozen significant ones. It seems doubtful that each of these products has been tried and tested in a wide range of situations by a large number of companies, which means there is bound to be software that is lacking in certain areas, and those areas may be just the ones you need. So, three of the key questions I ask when looking at a manufacturer’s website: is what evidence is there that real customers are using this product?  Are these customers doing the kind of things that I want to do? And thirdly, do these customers have similar kinds of operations and are they doing projects of a similar levels of complexity?

Beyond that, I’d ask What about the software company itself? Am I likely to need consulting and support? If so, are they at least in a similar time zone to me?

Of course this is just a start and you will need to consider at least one or two more factors that are ‘must haves’ for your business. All this may help you reduce hundreds to something slightly more manageable.

It’s snow problem!

Sheila Wilson while out sledging

Sheila Wilson while out sledging on the outskirts of London last Thursday

Europe is gripped by the hardest, snowiest, coldest winter in decades. Most of the UK, particularly the London area, typically has mild winters with an occasional dusting of the white stuff.

With Londoners unprepared, the city’s transport network has siezed up, many schools have closed and hoards of office workers remain stuck at home.

At meaning, we work mainly at home anyway, beacuse it is more productive and cost effective. It also means that whatever the weather, we’re still here to help our customers. So, we get to enjoy a bit of sledging and still carry on business as usual!

Panel crazy

Just before the end of last year, I decided to join one of the online panels to get a better understanding of their workings. I have to say that the whole experience has left me open-mouthed in shock.  In the short period since I have been a member, I have encountered error after error as well as, frankly, poor questionnaire design and basic scripting errors.

Typical errors I have experienced are:

  • Emails in various foreign languages inviting me to participate in surveys
  • Numerous broken links in emails
  • Requests to enter passwords that I don’t have
  • Another time, I clicked the link and got a message saying ‘this survey has not started yet’. I went back later and clicked the link and got a message saying that I had completed.

For those few surveys where I have not been screened out, I have encountered numerous impossible lines of questioning. There never seems to be a way of skipping a question, so your only options are to just click anything (to make sure you get your payment for completion) or to give up.  Some questionnaires allow you to make a comment at the end, so you can at least explain that you were forced to give an opinion on the personalities of different washing up liquid brands, even though you have no opinion on the matter because you regard washing up liquid as a commodity and just buy the cheapest, irrespective of its name, color, perfume etc.

In another survey, which was about radio stations, I was presented with a list of radio stations and asked which one I had most recently listened to. I picked a talk and news station. From then on, I was asked endless, detailed, non-applicable questions such as whether the presenters interrupted the music or how fun the competitions were.

And perhaps, even worse, I have recently seen a question which asked me which of a list of brands I had  bought. The one I had bought was not on the list and there was no ‘other’ option! Again, I was forced to select an incorrect answer or give up.

I did think that perhaps I was just biting into the one bruised apple of the basket, but coincidentally, I spent New Year’s Eve with a friend who is managing a new panel.  She has worked with numerous panel suppliers and was not in the least bit surprised by my tale of woe.

Having had this discussion with my friend, I was wondering whether market researchers are accepting unrealistic deadlines. Clients know that with the technology we now have, it is possible to script a project and complete the fieldwork online in a matter of hours. But perhaps they don’t know about the amount of thought, effort and creativity that is required for decent questionnaire design and in particular thorough quality control. There’s no point in conducting a survey if the questions are impossible to answer honestly and accurately. Presumably, some important business decisions are based on some rather dubious survey results.