Article #35: ASC99 Conference report


ASC99 briefs

Statisticians often view data mining as "naive hope, vainly struggling against the cold realities of chance", said Professor David Hand of Imperial College. With an estimated 90% of patterns found coming from contamination in the data and another 9% down to chance, researchers must work closely with clients to establish the real findings

After metadata comes "paradata", an attempt to use information about the interview gathered automatically in CATI or CAPI. Poor design of questions and screens can have a profound effect on the data. Analysis of timers, keystrokes, errors and missing information can identify problems and help with design improvements. The good news is: this information is completely free.

Using interviewers as a source of paradata needs to be treated with caution, according to Geert Loosveldt from the Catholic University of Leuven, Belgium. Interviewers' assessments of respondent performance varied widely and did not match objective measures from other data sources.


Tim Macer reports from the International Conference of the Association for Survey Computing in Edinburgh

Time to set standards

 

The Association for Survey Computing annual conference last month saw a number of brave forays into the area of standards for survey data and questionnaire design. There was also a useful series of presentations on CAI (computer assisted interviewing).

This was no gathering of the geeks. ASC99 offered many original and well-presented accounts of all imaginable uses of technology in survey research, served with generous helpings of practical advice and refreshing candour about what works and what does not.

Computer assisted interviewing

The impact of C API and CASI (self completion) is now being felt across all areas of survey research in terms of better quality data, though the benefits can easily be lost if the appearance of questions on screen is poorly considered.

The effect seems strongest when respondents are viewing the screens directly. Elizabeth Fraser (Health Education Board for Scotland) and Helen Angle (BMRB International) had switched to multimedia CAPI for a continuous ad tracking survey, allowing video and sound to be played, and had seen variations of 25% or more in recall compared to identical surveys on paper using photo montage or "telepic".

While the gains in moving to CAPI are considered worthwhile (despite the high costs) there are dangers. End-user's expectations need to be adjusted if a new norm is established and the novelty of this one measure can distract from findings in more 'boring' measures such as spontaneous recall.

The second UK National survey of sexual behaviour (NATSSAL) moved to CASI after an extensive pilot srudy showed dramatic reductions in both incomplete and inconsistent data. Don't-know and refusal responses dwindled to one twentieth of the levels observed wieh the same questionnaire was administered on paper. But the anticipated imporvement in self disclosure of particularly sensitive data such as male-to-male sexual acts, did not materialise to any significant extent in the pilot. This may be more to do with the low incidence of these behaviours in the population - the main survey will shed more light on this when it is completed.


Technology showcase at the ASC Conference trade show and poster fair
Photo with thanks to: ASC

The clash of the standards

Triple-S, though reasonably well known in this country as a standard for the interchange of market research data, is little known elsewhere. Statistics Netherlands, along with four other EU states including our own Office for National Statistics, have received EU funding for their own TADEQ initiative - to standardise descriptions of computer-aided questionnaires, instruments which are frequently impossible to verify independently once they have been set up on computer.

The Royal Statistical Society, in a


We all need these standards, and daily waste time, effort and resources overcoming their lack

paper presented by Essex University statistical archivist Hilary Beedham, was marching under the banner of the University of Michigan's 'Data Documentation Initiative' or DDI - a standard to allow users to work with multiple data sources.

Also from the States, the US Department of Agriculture had produced 'Star Schema', which was another standard to allow users to work with multiple data sources. As if that was not enough, SPSS MR chief Ian Durrell issued a rallying call for 'Vision 2000', SPSS's new data model that is proposed as "the new standard".

Everyone was talking about XML, which stands for "extensible markup language": not a standard in itself, but a means of implementing standards in such a way that they will plug into existing computer technology: web browsers, office applications, and so on.

 

Verbal skirmishes soon ensued. These started with a question to Ian Durrell to clarify how open the SPSS 'open standard' will be. The answer: a lot less open than those working on the other standards would like to see. As Durrell informed us, SPSS will be consulting only its clients (not any competitors) on its Vision 2000.

As the conference progressed, it was clear all the other proposers of standards wanted to exchange ideas and look for ways to work together. For instance, the triple-S people are keen to work with the Michigan DDI and invite other bodies in to make it a truly international standard.

Some felt regret that SPSS had never expressed an interest in triple-S: it could be doomed without SPSS's commitment. We all desperately need these standards, and daily we waste time, effort and resources in overcoming their lack.

But if a super-standards body should emerge from ASC - one that is capable of getting the ISO number - is SPSS going to be serving its customers and itself best by staying out in the cold? Or will it be one standard for them and another for everyone else? It seems unlikely.

Tim Macer writes as an independent specialist in software for market research. His website is at www.meaning.uk.com.

ASC Conference Proceedings and Papers (542pp), available from ASC, PO Box 50, Checsham HP5 3HQ, £60 inc p&p, or email admin@asc.org.uk; website at www.asc.org.uk


Published in Research, the magazine of the Market Research Society, Novemeber 1999, Issue 402.

© Copyright Tim Macer/Market Research Society 1999. All rights reserved. Reproduced with permission.

Top | Index