Software reviews from meaning

Vision Critical reviewed

In Brief

What it does

Web-based, hosted solution for high quality web surveys with an integrated, fully featured panel management solution and some support for online qualitative research.

Supplier

Vision Critical, Canada

Our ratings

Score 3.5 out of 5Ease of use

Score 4 out of 5Compatibility with other software

Score 4.5 out of 5Value for money

Cost

Per month: approx £1,250 for first 5000 panelists, £500 for each additional 10,000, with no cap on number of surveys, interviews or users.

Add-on modules for analysis, communities etc between £150-450 per month.

Allow £7-10,000 on-off for set-up, design and customisation.

Pros

  • Complete range of panel management and community building toolsSophisticated Web 2.0-like survey capabilities
  • Links together all a panellist’s responses to any survey across the database
  • Lets in-house researchers or smaller research firms run their own panels

Cons

  • Authoring and management is Windows and IE only

  • Limited animation support in survey tool

  • No enterprise version offered

In Depth

Given that many researchers wishing to do online research are likely to want to develop and run their own panels, it is odd that the Achilles heel of most online survey tools is still in providing decent panel management capabilities. Some get no further than giving you a database, a recruitment survey and some invitation management capabilities, others not even that far. Not so with Vision Critical, a Vancouver-based company that has spent the last three years building a very different online survey tool from the normal fare. Panel management lies at the heart of its Vision Critical web-hosted online survey tool, with not only facilities to fine-tune the selection of respondents from the panel database, but a complete range of tools to aid building the panel, nurturing a panel community and communicating with panel members and keeping it in peak interviewing condition.

Vision Critical is the first foray into technology for veteran Canadian researcher Angus Reid, and former owner of Angus Reid Group, which was acquired by IPSOS in 2000. The benefit of having a researcher shaping the technology is apparent throughout this software. More than any other online research tool I have reviewed to date, this is a product that acknowledges almost equally the needs of researchers, research clients and, with impeccable grace and style, the often overlooked needs of respondents. If you are seeking a more respondent-friendly way of conducting research, then this is certainly one to review.

The software is provided exclusively as a hosted ASP solution – Vision Critical have no plans to produce a version to install in-house. Therefore all of its capabilities are invoked from a web-browser interface. It is clear that Vision Critical is pitching for the corporate user rather than the agency, though the product should also appeal greatly to smaller research companies with no desire to run their own IT. The software comprises five key functional components: panel member administration, sampling, survey authoring tool, survey deployment and two reporting modules: a simple real-time tool, and a more advanced tool, about to emerge from beta testing.

Though it is not released yet, the new Dynamic Reports module module is intriguing – allowing you to use the same tool to run one-off reports for instant viewing, building complex reports as PDF, Word or Excel files, or publishing them to the web in the manner of a data portal for controlled viewing by clients and stakeholders – which will then change dynamically when you release more data into them. The approach here, and throughout the suite, is to not try to be too ‘clever’ in terms of functionality – avoiding options for options’ sake and going for sensible defaults so that it is easy to get started and become more sophisticated over time.

The Panel Plus module is fully released and in widespread use. It contains everything you need to run your own panel without having to recruit an army of admin staff to run it. It works with the sensible assumption that panel members should be able to self-administer their profiles, participation and reward redemption for themselves through the capabilities you can build into the panel member site. Though you can do all of this yourself, if you wish, designing the site and customising it is part of the set-up service that Vision Critical offers.

It also takes pays more than lip-service to the notion of a panel being an online community. For example, it provides the means to edit and publish your own online newsletter for panel members, which could be daily if you wished. It offers a content management system to let you publish findings and reports to your panel members, which is critical when creating ‘professional’ panels. And most interestingly in the context of Web 2.0, it offers the opportunity to blur the edge between researcher-led and participant-led qualitative discussion by integrating online forums or even blogs. It will also interact directly with commonly used Web 2.0 social networking portals.

There is a nice survey authoring tool too, which is well laid out and easy to use, with a very full range of functionality. It will also let you integrate with Java or Flash components, though it is disappointing that this otherwise futuristic tool does not have more in the way of animation and interactivity at present – though these are supposedly in the pipeline.

It is in the connection between panel and surveys that another astonishing difference emerges. The classic organisation for survey tools is to make each survey a discrete set of records, maybe linked to sample or panel indirectly, but with no actual interconnection. Some provide the mean to post back key variables into a respondent’s profile, but only to a limited extent. With Vision Critical, everything is interlinked, with every survey question answered being attached directly to the respondent’s record, and every prior answer being available for re-use. In case this sounds like chaos in the making, with a minestrone soup of a million questions to choose from, the coup de grace is that the variables are still presented as organised by survey, even though it is effectively just a slice through the database. But it means that at any time, all prior responses that a respondent has given are available to you to use for routing, sample selection or quota control – and also at the analysis stage, such as for profiling or comparison. So long as the respondent was asked the question and provided an answer, anything can be cross-referenced by anything.

But isn’t that what panel-based interviewing systems should by like?

Customer viewpoint: Dru-Ann Love, Business Week, New York

Dru Ann Love is a Primary Research Analyst at the US Publication BusinessWeek Research Services, based in New York. In addition to carrying out surveys and polls that often appear in the pages of BusinessWeek, and surveys among their print and online subscribers, the research group operates as an independent full-service custom research provider. One of its prized assets is its BusinessWeek Market Advisory Board — a controlled access panel of many thousands of business leaders and opinion formers.

This year, BusinessWeek Research Services switched to Vision Critical in order to bring control of its panel in house – although the panel is actually hosted on Vision Critical’s servers, Dru Ann is able to carry out all of the management aspects concerning the panel’s operation for herself.

Dru Ann recalls: “When we had the demo, our reaction was ‘wow!’ you can do all of this. And once we started using Vision Critical, it was really good. I enjoyed it – it was really fun.”

One of the first tasks for Vision Critical was to migrate BusinessWeek’s high-profile panel into their panel management suite.

“Their role was instrumental as we migrated our existing panel,” reports Dru Ann. “And they did a better profiling questionnaire than we had previously.” This has enabled the Research Services team to obtain much more information about their panellists.

Dru Ann also points out the sample selection capabilities as being a particular strength of this software: “I have control over who I selected, and I can verify the source of my target markets. For example, if I need to select a target of ‘all Directors or above’, I can see exactly who I’ve got, and I know to my satisfaction that I’ve got the group that I am targeting.”

Her experience with the questionnaire design module is also that it is “attractive and easy to use”. Again, Vision Critical, as a part of the migration process, defined templates so that the panel members’ portal, and also the surveys have a consistent look and feel, reflecting the BusinessWeek design ethos.

Dru Ann uses the built-in reporting features while the surveys are active. She comments: “I really like their online status report which has a feature which shows completes, so it gives me an idea of how many people are taking my survey at any particular time. If I start noticing that I am not reaching my target, then I can send another email blast to get some more respondents.” More detailed analysis is carried out in SPSS, using the capability provided in Vision Critical to create a complete SPSS file of the data.

She concludes: “I have no complaints. The software is really nice, and the people have been nice too, whenever I have needed to contact them.”

A version of this review first appeared in Research, the magazine of the Market Research Society, July 2007, Issue 494.

Tim Macer

Tim is a world-renowned specialist in the application of technology in the field of market and opinion research and is probably the most widely-published writer in the field. His roots are in data analysis, programming, training and technical writing. These days, as principal at meaning he works with researchers, users of research data and technology providers around the globe, as an independent advisor. He is quite passionate about improving the research process and empowering people through better use of technology.

Leave a Reply

You must be logged in to post a comment.