Software reviews from meaning

Key Survey reviewed

In Brief

Key Survey version 7.1

Worldapp
Date of review: April 2009

What it does

Online survey data collection and reporting system provided on a Software-as-a-Service basis by a USA provider represented in the UK and elsewhere. Sophisticated mid-range web survey tool with a lot of flexibility and sophistication, at a highly competitive price.

Our ratings

Score 4 out of 5Ease of use

Score 3.5 out of 5Compatibility with other software

Score 5 out of 5Value for money

Cost

Annual fee for an unrestricted licence in US dollars: single user $3,950, additional users at around $3,500 (cost reduces by number of users). Annual usage-based licence for up to 10,000 completes at $1,950 per user and 25¢ for extra pre-paid completes.

Pros

  • Browser based and system independent – works on PC or Mac browsers
  • Simple GUI for most operations with a powerful scripting language in the background
  • Highly customisable and extensible through a series of plug-ins and a software interface (API)
  • Built-in error checking capabilities

Cons

  • No panel management
  • Analysis capabilities somewhat restrictive
  • Only exports to SPSS and Excel. No Triple-S.
  • Cannot create your own plug-ins.

In Depth

Researchers looking for a flexible web survey tool will be familiar with the conundrum that the more affordable entry-level tools lack the flexibility you need, while the grown-up ones not only cost a fortune but can be bewildering to use. Key Survey is a relatively new entrant to the UK market which focuses on the middle-ground and is aimed at researchers who want to build and deploy sophisticated surveys for themselves. Compared to other products on the market, Key Survey sits towards the high end of the middle ground. For a relatively low fixed cost, based on the number of users, you get a hosted solution with no restrictions on the number of interviews you do each year – something that is almost unheard of in the software-as-a-service market.

Key Survey not only covers all the basics in style – like all the standard question types, all kinds of survey routing and logic, text piping, logic on answer lists, question libraries, look-and-feel templates, multiple languages, survey invitations and reminders – it also leads the non-technical researcher into some very sophisticated territory. It offers analysis and online reports, but does not provide any panel management support.

It has a question library, which comes populated with hundreds of well-worded questions organised by subject, and very useful search capabilities that other tools often lack. You can add to the library, or treat any prior survey as part of the library. Using this could help you to standardise survey design and harmonise demographics across surveys.

There are over a hundred different design templates to choose from, and it is easy to take any of these as starting point and design your own templates too. These are all based on CSS (cascading style sheets) so they are sound at a technical level, but you do not have to get your hands dirty with any actual CSS coding, unless you actually want to.

One of the most versatile capabilities is the collection of plug-ins. These allow you extend the basic functionality into untold areas of sophistication – and there are no extra costs for using them. Each essentially goes off and performs a task and then returns any resulting data back into the questions you have defined in your survey. There are plug-ins for various kinds of flash animated questions, such as sliders, calendars for selecting dates, or using Google Maps to choose a location. There are plug-ins to validate and clean data on the fly, reach out to an external database, or to perform geo-IP checking on survey respondents, to detect those pretending to be in countries they are not. There are already over 20 plug-ins, and whenever a customer requests a new one, WorldAPP make it available to all customers too.

Logic-related errors are a common and expensive problem with surveys as they become more complex. I particularly like the Key Survey approach, which has a level of logical scrutiny built into it so that it will advise you of ill-formed logic, questions that may never get asked, and so on. Furthermore, wherever you make logic selections for routing, filters or text piping, a simple on-screen assistant anticipates the choices you are about to make, and presents you with context-sensitive overlay of the questions and answer options available – it is ingenious and very intuitive, and will actively help you stamp out most common scripting errors at source.

My only concern over the design interface is that the survey questions and answers are presented as a single scrolling page (though they can be displayed in the survey on different screens) and on long and complex surveys, this could become difficult to navigate through – especially the buttons and actions you need at the top of the page quickly scroll out of sight. It feels more web page than web app, in this respect.

The software does offer support for other self-completion streams of data too – there are additional cost modules offered for print/mail/scan surveys on paper, and for IVR on the phone, which share the same design and reporting environment.

The analysis and reporting side is a bit of a disappointment after all the capabilities in the upstream areas. It does make it relatively easy to create simple reports showing frequencies and charts, and these can also include a wide range of statistics. For some users, this may be sufficient, but the reporting formats are inflexible and most researchers will struggle to use the tool to probe their data fully and then build reports to communicate what is salient. What it does seem to be good at is letting you create quick snapshot reports that can be updated regularly as results arrive and which you can publish and share with your clients.

Users wanting to do deeper analysis are likely to want to move the data into other cross-tab or statistical tools, and here your luck runs out if it is not SPSS that you use, as that is currently the only export route that provides you with labels and definitions ready to run. WorldAPP is working on further exports, though the next one is likely to be SAS.

A particularly nice feature of the software is the live help button – this connects you immediately to a support representative who will answer any question you have using instant messaging chat. WorldAPP provides support from its three locations in Massachusetts, London and Kiev, in the Ukraine, so someone is usually available to answer questions around the clock. Support is also included in the fixed annual fee.

Customer viewpoint: Monica Coetzee, Research Manager, The College of Law

The College of Law is the UK’s largest dedicated postgraduate law school, operating from six campuses across England. It’s been using Key Survey for the last nine months in its research division, which carries out a broad range of surveys including student surveys on course quality, HR surveys, a continuous customer satisfaction survey on IT services and even ad testing, all done online with samples that are often in the thousands.

Monica Coetzee, Research Manager, explains: “We were previously using a desktop survey tool, but because there was only one person trained to use this software, it was causing a bottleneck. We wanted to restructure the department anyway and I wanted to change to working online.”

“As a non-profit organisation, price was critical for us when choosing a replacement, but we also needed something with a lot of advanced features. We reviewed several tools and Key Survey came out on top.

Monica’s team now have around 30 surveys done with Key Survey under their belts. “What I like about it is that it allows all three of us to have access to the software online from anywhere, giving us greater flexibility, for example when working in different College centres. Each person has responsibility for an entire survey project and not just for specific tasks.” This, in many cases, includes exporting report-ready data into SPSS. “It exports it directly with all the data and value labels correctly filled in”, she reports.

“We now know there is nothing that we cannot do – and we use all sorts of advanced features like show/hide options and complicated skip routing. We very often bring data we already have into the survey and use the autofill function to use it for verification or routing. And it all works very well.”

A particular requirement for Monica is the visual appearance of surveys, and she was pleased to be able to produce a set of templates matching the corporate visual identity of the College that could be used consistently across all surveys. “Although they have a good range of templates, there is the option to go into the HTML programming, so I can play around and see what happens to make my own. With very little HTML knowledge we have produced some really nice-looking surveys.”

Monica also appreciates the built-in warnings and safety features of the system. She remarks: “If you try to do some things that would result in an error, it will give you a warning or prevent you from doing it. But once you have gone live with a survey, if you need to make a change, you can react so quickly with this – it will make that change instantly. It must be said, my favourite function is the live help button. They almost always answer within a few seconds, and they are always very friendly and helpful. I actually prefer using a typed interface to a phone calls – I find I am generally more comfortable communicating online in this way.”

Published in Research, the magazine of the Market Research Society, April 2009, Issue 515

Tim Macer

Tim is a world-renowned specialist in the application of technology in the field of market and opinion research and is probably the most widely-published writer in the field. His roots are in data analysis, programming, training and technical writing. These days, as principal at meaning he works with researchers, users of research data and technology providers around the globe, as an independent advisor. He is quite passionate about improving the research process and empowering people through better use of technology.

Leave a Reply

You must be logged in to post a comment.