Cinders seeks prince for fulfilling relationship
Quac, Bellview and Verbastat, May 1995
Tim Macer finds open-ended coding has had a complete make-over and has turned out surprisingly attractive...
It is some time since John O'Brien, delivering a paper to the MRS Conference in 1981, dubbed coding as "the cinderella for survey processing". There have been a few brave attempts to produce computer software capable of automating this process in the intervening decades. But it seems Cinders is still waiting for her ball. This is perhaps surprising, since openended questions are expensive to administer. Jonty Campbell, who oversees the weekly omnibus at Audience Selection, tells me they charge double for a purely openended question. For questions with a few "other specify" codes, they absorb the cost, rather than pass it on. Although part of that cost is simply the addition time spent during the interview recording the answer, most is the cost of coding.
SPSS provide a coding module, as a part of their Quancept suite of computer-assisted interviewing software, called Quac (Question Auto Coding). Richard Kottler, at SPSS, tells me it has been a part of the package since the early eighties and is pretty much as it was then.
Andy Mendelsohn of On-Line Telephone Surveys, who is also the Quantime User Group chairman, described openended coding from CATI interviews as "a problem area." He told Research "Quac has its plus points, but it lacks a number of features it would be useful to have. Bellview seems to be more labour-intensive and is, if anything, worse". He thinks the software manufacturers tend to treat coding very much as an afterthought. However, Andy had not seen Mass coding which is available in the latest version of Pulse Train's Bellview and provides some useful new features to aid productivity that may give it an edge in this respect.
Bella Moller at Pulse Train explained "Mass coding is ideal for 'other specify' type questions and short openends." A report gives a sorted, summarised list of unique responses along with a frequency distribution. From this you must draw up a codeframe manually. After that, coders simply assign codes to the summary, not to each case, and the software applies them to all interviews containing that answer. If an item appears as two different entries (e.g. "BT" and "British Telecom") the same code can be assigned to both, and the answers brought together. The drawback is that it relies on interviewers having to separate different answers in the "verbatim box" on the computer screen by specifying a delimiter character, such as a semicolon. Adding these in afterwards, via Bellview, is very laborious.
A very satisfying solution has just arrived on the market from Datastat Belgium in the form of a complete coding package called "Verbastat". It is a PC-based system with two modules. The first is aimed at pencil and paper interviews, where the verbatim texts are entered into the computer. The second is the main coding module. Users of Quancept or Bellview need only use the second module, as it will load verbatims directly from the openend file and will work just as if coding had been done within the original CATI or CAPI system. It can be adapted to work with other systems too.
Verbastat takes a wholly different approach to coding. For a start, it is mouse-driven and is truly point and click. It presents a complete list of all the unique answers to a question, and invites the user to deal with them in terms of "splits" and "merges". Where one response contains several answers, these are "split" by introducing a delimiter (usually a dollar sign). Where answers are the same in spirit, though different in appearance or spelling, the coder merges them and selects the preferred version. It has the same kind of satisfaction as doing a jigsaw puzzle (or, to PC junkies, playing Minesweeper). As each merge is done, the picture becomes a little clearer You can see its immediate effect, as well as how much there is to be done still. It can be quite compulsive!
Verbastat does much to exploit the benefits of computers in eliminating redundant work. For example, using the data entry module, you only have to enter an answer once. The next time the same answer appears on a questionnaire, you select it from a list on screen without actually keying it in again. During coding, there is very little to enter on the keyboard, apart from mouse clicks and pressing function keys.
Richie Corion at Strategy Research in Miami used the package for two months before deciding to purchase it. His verdict was that Verbastat is "very good. It makes everything much easier to do. It could be used in a coding department as it is so easy to use - it takes very little instruction." Richie gave an example of one of their large continuous surveys which took four days to code. Using Verbastat it is now ready in a day. He found it worked best with short openends. "With long drawn-out openends, it isn't worth it."
Serge Luyens, MD of Datastat (now part of SPSS) and the program's author pointed out to me "The system is not a handwriting recognition scanner nor a semantic closeness detector. It's an efficient tool for reducing the cost of a task often considered as arduous, not very satisfying and not held in high regard, but which greatly affects the overall cost of research."
As a consequence, Verbastat is very rich in features. Merges and splits can be "undone" or recovered later without limit. New questionnaires can be added in to batches already worked on. You can use spell-checkers and can code automatically by "keyword searches" or get the software to suggest some keywords to use. Listings and reports can be produced at the touch of a button, as can the codeframes, data definitions and even a complete Quantum program for analysis. When you are happy with your coding solution, a new fully coded data file is ready for analysis.
Perhaps, at last, Cinderella has found a elegant gown to wear. Now we must wait to see who invites her to the ball.
Published in Research, the magazine of the Market Research Society, May 1995, Issue number 348.
© Copyright Market Research Society/Tim Macer 1995. All rights reserved. Reproduced with permission.
Copyright © 2006-2009 meaning ltd. All rights reserved.