Apply yourself

How effective are qualitative analysis software applications? Tim Macer wades through the code.

When Thomas Edison described genius as “1% inspiration, 99% perspiration”, he could have been talking about qualitative research, where often the objective is to reduce 100 pages of notes into a single, insightful sheet.
These days, computer software excels in taking the effort out of organising data, saving on perspiration and affording greater opportunities for inspiration. A number of affordable solutions are available (see below), so it is curious that in 2002 qualitative research remains the lost kingdom when it comes to using software.
Most qualitative analysis software works on a code-and-retrieve basis. You use your judgement to classify any important words, phrases or even whole paragraphs from a transcript. You can assign several codes, so an interjection from a focus group on life insurance, for example, could be coded as relating to “mortgages” and “family responsibilities” plus “negative attitudes”. The individual speaker could also be coded: classed by lifestyle and demographics. In addition, you can attach your own free-format annotations and commentary.
Once you have coded your transcript, various tools allow you to view the results or filter them by the other criteria you have included in your classification. The more effort you put into the coding, the more there is to retrieve and the better use you can make of some of the advanced analysis tools that the different packages offer.
“In the olden days we wrote reports that were very impressionistic. Now we can say definitely ‘It is!’”, says Professor Lyn Richards, an evangelist for computer-assisted qualitative data analysis (CAQDAS). “It gives you more confidence and rigour, and more incisiveness in the analysis. You can keep asking questions - pursue a hunch until it stops being a hunch and becomes a conclusion. That is precious.”
Gill Ereaut, who runs Gill Ereaut Qualitative Research, acknowledges that some qualitative researchers have an aversion to mixing computers with qual work, she says:
“There are also some good reasons why researchers are ignoring CAQDAS in commercial research,” she says. “You find these programs being used a lot in academic, social and government research. But my feeling is that most of these programs were designed to mechanise a form of qualitative analysis we do not actually do.”
Alastair MacLeod, director of the healthcare research specialist MacLeod & Associates says: “I’ve tried it [CAQDAS] and concluded it will not help me or my clients in a cost-effective way. It is surprising because the programs look great, but it is a mirage because of the complicated things you have to do. Despite that, I am still convinced that the right tool, properly designed, could increase the quality of the analysis that we do.”
So, what’s the verdict? Steep learning curves, a lot of effort up front for very little reward downstream, and a feeling that the tools just don’t do what quallies in the commercial world do with their data: I put this to Lyn Richards co-author of two of the leading CAQDAS products, N5 and NVivo. She acknowledges the method was not developed with the needs of the commercial researcher in mind, but remains convinced it could help researchers beyond the realm of academe.
“It is a fairly permeable membrane,” says Richards. “There are not really two different worlds. We need to re-present the software in terms of what commercial users would use. While the tools may be designed for someone doing a PhD, we have to recognise that commercial work is different. We have a lot to learn from people working under the constraint of market research.”
The solution, it seems, is not to rewrite the software, but to write new manuals for MR users – though Richards does acknowledge some software changes are necessary and has a few on the cards for her own package.
Just how willing the various manufacturers are to make these changes remains to be seen, but the perspiration/inspiration ratio will need to shift significantly before software goes mainstream for quallies.
NUD*IST N5

One of two solutions offered by QSR, Australian CAQDAS specialists and market leaders in the field. NUD*IST predates NVivo and offers a wide range of tools for coding and classifying unstructured qualitative data. It has a steeper learning curve than rival WinMAX and works in a less visual environment – many of the activities are driven by special syntax you must write. However, those who master it can pull out key findings and get a quick overview from raw transcripts astonishingly quickly, before using the other tools to delve deeper. In February, QSR announced N6, which will contain new ‘command assistants’ or wizards to write a lot of the syntax for you, and different templates designed for processing questionnaires and focus groups and may make it much more appealing to the commercial qualitative researcher.

NVivo

When NVivo came out a couple of years ago, users were expecting QSR to kill off NUD*IST, with NVivo’s more advanced, Windows-style analysis. Instead, both products have found different groups of users, largely in the social and academic research world, and both are still being developed. NUD*IST appeals more to those wanting rapid access to a lot of data; NVivo is better for anyone wanting to go more deeply into their data – and with the time to do this! NVivo features rich text and colour coding on screen, abilities to connect different passages in documents with hyperlinks and a wide range of new tools to help with very subtle analysis and interpretation of multiple transcripts.

Sphinx Lexica

French-developed (but fully English translated) package for handling quantitative surveys which includes a ‘lexical analysis’ module that can be applied either to openended questions in a qual/quant survey, or to depth interview and focus group transcripts. Taking a slightly different approach to the code-and-retrieve solutions, Sphinx works by word and phrase searches from your custom-created dictionary, to help you to identify themes and make connections between speakers, context and other concepts.

WinMAX

Developed in Germany a decade ago and now working in a full Windows environment, WinMAX allows you to import transcripts and then develop your own classification system of codes, and apply these codes to different parts of the transcript. It also supports, text searches, allows you to tag items and add your own annotations and even crosses the boundary into quant by letting you output data for statistical analysis in SPSS. WinMAX is fairly simple to learn, but lacks the finesse of some of the other solutions. Though text import is easier than NUD*IST or NVivo, you are unable to edit the transcript further, such as to correct obvious errors in it.

All four packages are distributed in the UK by the publishers Scolari Sage. Website: www.scolari.co.uk.

Published in Research Qualitative Review - March 2002 issue.

© Copyright Tim Macer/Market Research Society 2002. All rights reserved. Reproduced with permission.

top of page | Back to list of reviews