Data editing and cleaning workhorse gets an update
by Tim Macer

Anyone new to market research could be forgiven for thinking that quality is a recent discovery in the industry. Yet it is ironic that, in these quality conscious times, so little time is now paid to that most basic quality control process: editing and cleaning the data. Most of the new generation data collection tools pay scant regard to it. So all credit to Microtab - one of the earliest self-drive tabulation products, with a 20-year pedigree and recently vastly improved with a new Windows point-and-click interface - for providing an end-user cross-tab tool with a whole range of tools for improving data quality.
Microtab was one of the first tab packages to open up desktop analysis as a serious proposition for the researcher rather than the DP specialist in-house or at the service bureau. It first appeared in the early 1980s at a time when desktop computers were new, terrifyingly expensive and still called “micros”, hence the name of the product. The PCs back then were extremely limited in their resources, which encouraged great frugality among the program developers. That legacy today means that Microtab uses mere teaspoonfuls of disk space and memory while running at lightning fast speeds.

Table specs without the syntax
The product is still aimed at researchers who want to produce their own cross-tabular reports, and is fairly traditional in the kinds of analysis that it produces. It assumes that you will want to produce tables in bulk. While you may spend several hours setting up the run of tables, the speed of the program means that, in most cases, the report will be ready to print in less time that it takes you to navigate the Windows dialog to send it to your printer.
It is different from many of the other desktop analytic tools, like SPSS, Quanvert or Pulsar, where you build and view one table at a time With Microtab, you can build the whole report before you see it. That is not to say you cannot it use to build one table at a time, but it has the advantage that you can build your reports in advance and handle standard reporting and re-runs with great ease.
Operationally, it is more like the syntax-based cross-tab workhorses like Quantum or Mentor, except there is no syntax: Everything comes from drop-down menus, checkboxes and forms that you complete. For researchers used to working with paper reports, this could provide the best of both worlds in one product.
All of the programs options are available from a series of drop-down menus and a toolbar. Though the program does now work as a native Windows application, the user interface makes little use of the graphical capabilities of the Windows environment and is generally rather text heavy, revealing its DOS origins. However, keyboard shortcuts can make it fairly quick to navigate.

Pick a card
Probably one of the biggest deciders as to whether you are going to love or hate Microtab is its reliance on the 80-column 12-punch binary card as the native model for its data definition. Fortunately, the program is largely variable-based when you define tables, banners or filters. But you are not able to avoid card columns when defining editing or cleaning your data, and their appearance when recoding variables or defining weighting seems unnecessarily clumsy. Of course, anyone using paper-based questionnaires marked up for cards and columns will find this old “VX0-9” syntax an advantage, and

Microtab XP 2003 - Professional Edition
  • relatively easy to use
  • fine control and great versatility over the content of cross-tabular reports
  • powerful editing, cleaning and weighting capabilities for end-users
  • good import routes from SPSS or triple-s
  • underlying data is still always based on 80-column cards
  • lack of drag-and-drop and right-button menus makes Windows interface feel dated
  • cryptic manuals with few examples
programming the definitions is fairly simple though a little tedious. Anyone used to more modern, database-driven or Web-based tools is likely to ask “Is this really necessary?”
To some extent, Microtab’s new add-on import module makes this less of a culture shock when importing from most variable-based data collection packages, as it has a full metadata import, transferring variable definitions, names, texts and codes without any need to retype anything. This not only saves a lot of time but eliminates the risk of most common definition errors. Two major import routes take you either straight from SPSS data - a typical export path from CATI or Web interviewing packages - or the more research-friendly triple-s format, which is the growing international standard for trouble-free survey data transfer. Triple-s is supported by CfMC, CI3, Bellview, VOXCO, Snap and Net-MR, amongst others. However, in the backgroud, the program is still chopping up continuous ASCII records into 80-column multi-card sets for Microtab to work on, and if you wish to combine data sets, you will have to look up their newly allocated card positions.

Making a clean sweep
The program is very good at taking you through the stages of preparing, cleaning and weighting your data, then producing the tables with the minimum of fuss. It almost assumes that there will be errors in your data and will actively help you to find them. If you are importing data, it will report on any unexpected values or mismatches, rather than fall over or reject the entire file, as other programs are inclined to do. This is particularly helpful if the data have been coded, or additional codes have been added for don’t knows and refusals which are not reflected in the record definition you have been sent. The reports produced when importing are clear and very detailed.
Most of the editing functions are to be found in the Columns drop-down menu (which does mean card columns), and these include tools for finding outliers, checking inter-dependency and a range of other clever checks and tests.
The iterative weighting tool is also impressive and unusual in a researcher-oriented program. It lets you define multi-dimensional target or rim weights, and again provides you with very comprehensive reporting. Such is the horsepower of today’s PCs that really complex weighting schemes, requiring multiple iterations to resolve, take just few seconds to ‘lose.
With the data defined, cleaned and possibly weighted, it is time to start defining tables. What Microtab refers to as a table, in its Tables menu is, in essence, the stubs. A second set of menu options allows you to define the banners and a third, Reports, is where you bring both banners and tables (i.e. the stubs) together to create the run of tables.
It is here that the range of options available makes most modern, drag-and-drop online tab tools look lightweight. You can fine-tune the presentation of values, define nets and subnets and even vary the percentaging and indexing of values on a row-by-row basis. Sometimes the options are a bit fiddly to perform.

New net lines, for instance, always appear at the bottom of the variable, and have to be nudged up into position by clicking the arrow key several times. The dialog for defining banner headers too could be made more user friendly if it responded to drag-and-drop instead of a strange combination of mouse clicks and the arrow key again.

Cranking out the tables
Ireland Consulting Group provides boutique research-based consulting and planning services to both businesses and not-for-profit organizations in and around North and South Carolina. “Working as we often do in the social services arena means producing results for people who are not researchers and need information to be presented in a very clear and easy to understand way. My goal is to make their life easier and not confound them with numbers,” says Gayle Ireland, the company’s president.
Ireland and her team achieve this through what she describes as the “ideal” combination of SPSS and Microtab, which she considers very complementary in their functionality. Charts and more advanced statistics are carried out in SPSS, but the tables are achieved more quickly and easily in Microtab. “With Microtab, you hit the report button and in a matter of seconds it has cranked out 60 tables. It uses a lot less paper than SPSS. Microtab really cuts the analysis time down. On a typical job, I can produce all the analysis for the report in two to three hours.”
Its strength is in its versatility
Microtab has been cranking out the reports for some 10 years at Houston-based Creative Consumer Research. According to Tracy Cryer who manages the firm’s Computer Department, the software’s strength is still its versatility. Today, the majority of projects arrive in SPSS format, from studies collected in CI3 and other interviewing packages, which are imported directly into Microtab, complete with all of the definitions and labels. Yet the program’s versatility remains an attraction, with the definition and layout of tables.
“I have pulled some pretty good maneuvering with the program over the years to get tables into some interesting formats and merging data from different sources. I think we have used pretty much every table facility there is. You do have to pay attention to what you are doing, but so far, I have not run into anything I have not been able to create,” Cryer says.
Both users commented on how useful and reassuring the error reporting facilities are, as they speed up the cleaning processes. Ireland says, “[Microtab] provides good idiot proof tools. In the frantic pace of trying to meet a deadline you can over look these things and you live in fear of having sent the wrong message or conclusions to the client. Any aids to make my life easier in that respect are welcome.”


Tim Macer is an independent specialist and adviser in the use of technology for survey research. His website at www.meaning.uk.com

Published in Quirk's Marketing Research Review - October 2003.

© Copyright Tim Macer/Quirks Marketing Research 2005. All rights reserved. Reproduced with permission.

top of page | Back to list of reviews