`Home     Our Services     Your Resources     News & blog     About us     Contact

The 3rd Degree reviewed

Can SMS text messages become a viable survey tool? Tim Macer reviews The 3rd Degree to find out just how bright the future is for text research.

While SMS text-message surveys are unlikely break into the big time as an interviewing method, they could have a useful role in a supporting role or even as a recruitment tool. Texting is so commonplace and accepted as a communications medium among the elusive under-25 age group that it makes sense to consider SMS text-delivered surveys, if they can be made to work. The reason virtually no MR agency offers test message surveys is probably more down to the technical difficulties than the more obvious constraints of the method, or its novelty.

The 3rd Degree, a technology start-up company with a web-enabled software product of the same name, is certainly hoping that a latent demand for SMS surveys will be unleashed as it becomes as simple to administer a survey by SMS as it is to do one on the Internet..

The 3rd Degree (or T3D) is provided as a hosted service, which means you can drive the whole process from your favourite web browser – a very wide range of browsers and operating systems are supported, unlike many ASP tools that tie you into Windows. The system works very much like one of the entry level web-survey tools like Zoomerang or SurveyMonkey, with the same sequence of steps. You log in, pick your survey or start a new one, write your questions, test them, define your sample and send out the invitations.

The survey is delivered to respondents as a series of separate text messages, one per question. Questions can be open or closed, and if they are closed, the list of answers with a single or double-digit answer code is provided. The respondent then selects reply, and enters either the code from the answer list, or keys in a numeric or a textual response, if it is an open question.

The constraints of SMS messaging means that surveys are very limited in the kind of validation you can perform: think of the diverse range of responses a sample may provide on paper despite your skilfully worded exhortations asking them to enter “1” or “2”.

Like a CATI survey, you need phone numbers of participants to contact, which you load into T3D’s panel management tool. You can load in new contacts from CSV files or Excel spreadsheets and reuse existing contacts. You can also create additional profile fields in your panel and populate them directly from linked questions written into your surveys, then use these subsequently when selecting sample. It is a good start, but needs a bit more functionality to be really useful.

Overall, the survey design and administration process is well thought out. The interface is a pleasure to look at. It is very accessible, so that anyone familiar with any other survey authoring software would be able to pick this up in a few minutes. There are only three basic components: closed questions (for single or multiple questions), open questions (for text or numeric responses) and information (for explanations or thanks).

Each survey is considered to have its own questionnaire, which you can either set up from scratch or you can copy from an existing survey and then modify. You can also reuse individual questions from other surveys - a ‘reuse’ button will list all the questions of the same type. Over time, I could imagine the list becoming very unwieldy, though, without any facility to structure the library of questions.

SMS surveys tend to be short - T3D suggest no more than half a dozen questions. So there is often no need for routing or the clever tricks found in CATI or web surveys. Nevertheless, simple routing is supported, and there is a facility to rotate questions, called cyclic questions. Set a group of questions to be cyclic and only one randomly picked question will be sent to each respondent. As a feature it would be massively more useful if you could ask it to randomly pick, say 3 or 4 questions, and rotate the order too.

In many places, the product has that newly-built hotel feel where not everything is quite open yet. It needs rounding out in several areas, but it is definitely functional, and the basics are well provided for. This is particularly true on the telephony side.

On the same screen that you define your questions, you also pick the sample from your panel database, and set the date and time at which you want to despatch the questions. You can set a delay here, so that questions do not come bouncing impatiently back as soon as the respondent enters the answer. Even a delay of a minute makes the survey feel more considered and less intrusive. Allowing differential delays between questions - something else the current version does not allow - would open up the scope to do daily diary and tracker-type studies, and go beyond the theoretical 6-question limit.

Results come back into one database, where you can easily view topline results and even do very simple cross-tabs and charts. It was great to see that this product also included an export to triple-s. However, the data do seem to need careful checking. Multicoded responses to closed questions can get mangled, and if a respondent texts back the same reply twice it can also get the data out of step for subsequent questions.

It is a case of thinking about the applications for it, and I am sure there are some very good ones out there.

The user view: AJ Johnson at MORI

AJ Johnson, director and head of data collection, MORI, has been experimenting with T3D, with a view to using it as a supplementary research method, or even as a useful recruitment tool for surveys by other methods. “It is great from the point of view of being able to set up simple surveys very quickly, for in the right arena, a short SMS survey can be ideal. This is good at doing the basic service, and I like the people. They seem willing to take on board suggestions, and they have clearly done their homework.”

For AJ, a concern remained over who pays for the call. T3D can offer a low-cost inbound tariff of 5p per message. But that would still cost the respondent 25p to complete a 5 question survey. T3D is planning to set up a toll-free number, but the network providers seem currently unable to offer what AJ would really like to see: reverse billing, giving respondents an instant phone credit for participation. One cute alternative that T3D is also considering is to send scannable barcoded gift vouchers by SMS. Respondents can simply redeem their rewards at their chosen retailer by presenting the message to be scanned at the till.

Published in Research, the magazine of the Market Research Society, May 2005, Issue 468.

© Copyright Tim Macer/Market Research Society 2005. All rights reserved. Reproduced with permission.

Complete solution for designing and administering surveys for SMS text messaging. It incorporates a basic panel management tool, some elementary topline reporting and full triple s export of data.


This clarification appeared in the July 2005 issue of research

In our recent review of The 3rd Degree SMS interviewing, we stated that respondents would incur a cost when participating. We have since learned the cost stated is that which applies to the toll free SMS number for surveys, and is incurred by the agency, not the respondent.

Furthermore, the limitation on multiple response is not as stated. There is no potential data integrity issue as the software permits only single digit or letter codes, which limits potential answers to 9 numeric or 26 alphabetic codes on single or multiple response questions.