Author Archive

« Older Entries
Newer Entries »

Mobile research growing up fast

By | Published on 20th April 2011

Hands holding out a collection of mobile phonesGlobalpark, organisers of the 2011 Mobile Research Conference asked me to chair day two of the event. I decided, rather ambitiously, to close the conference with a round-up of all the presentations that day. Here, in prose, is what I verbalised at the end of long day of very interesting presentations.

Don’t be surprised if you don’t recognise many of the names here. It is true to say that the early adopters of this method are not necessarily the usual suspects – there were some familiar firms present – but as the industry as a whole continues to see only problems with mobile research, it was illuminating to hear from those who are not only convinced of the value of mobile research, but are developing expertise, best practice and clients hungry for more.

Though organised by Globalpark, the event was software vendor agnostic, with examples from rival software providers presented too. In a few sentences for each session, here is what came up:

Bruce Hoang (Orange Advertising Network) – presented a multi-country study of mobile media consumption by mobile data users in the UK, France, Spain and Poland – countries with marked differences is adoption and usage. He has concluded that Web-optimised sites are more popular with consumers in mature markets than using apps to access content. “The web browser in the mobile device is the killer app” according to Bruce. He advocates sticking to web browser-based surveys on the mobile as it most closely aligns with respondents’ preferences and experiences.

Guy Rolfe (Kantar Operations) – asserted that Mobile apps for surveys definitely have their place. Kantar find participants are willing to download survey apps, which can enrich their survey experience. In parallel with this, many consumer product manufacturers and retailers are now creating lifestyle apps that capture a lot of useful data which are proving to be very popular with consumers – they don’t have a research purpose at their heart but the data they collect they could be very useful to researchers, if they can get their hands on it.

Jeroen de Rooij (ValueWait) – presented a lifestyle case study that proved it possible to use mobile research to ask 60-question-long surveys, with modest incentives, if you do it with care. The survey also asked respondents to email in pictures after completing the survey, and a very high proportion were willing to go to this effort.

Peter Lynn (University of Essex) – explained that from a social science and statistical perspective, the focus of scientific survey literature has tended to emphasise the negative – seeing mobile samples as a problem. This needs to be questioned. If you take a Total Quality perspective, there are many areas in which mobile samples are no better or worse than others – coverage may be better. There is also a one to one relationship between respondent and phone unlike landlines. Other sources of error are reduced, e.g. people are more willing to answer some kinds of questions, it avoids recall error, being in the moment, and overall, the responses are not otherwise fundamentally different from other modes. It’s strength surely lies in complementing other modes.

Michael Bosnjak (Free University of Bozen-Bolzano) and Sven Scherrer (Globalpark) – took us through some early results of study on how useful voice capture and voice recognition might be in overcoming the Achilles’ Heel of mobile research – capturing lengthy verbatim responses. Low response and high drop-off is often observed in mobile surveys when these questions get asked. The study pitched standard touch-screen entry with voice capture and voice recognition. From the preliminary results presented, voice did not come out well from a respondent perception point of view. Touch screen entry was preferred over voice entry – voice recognition was the least preferred and the spread of responses indicated a divergence of opinions here. Interestingly, respondents seemed to warm to those methods, particularly voice capture, when asked about it 5 days later. The actual effect on the data has not been measured yet – those results are due out soon.

Justin Bailey (The Nielsen Company) and Sean Conry (Techneos) — presented a case study on using BlackBerry Curve devices with a recruited panel of South Africans during the period of the world cup showed the extent to which low response really does not need to be a feature of survey based research. The study was monitoring media and advertising consumption and some brand recognition over the period of the World Cup. Very high response rates were sustained throughout an extended survey. Pictures were collected too, and Nielsen ended with a library of 60,000 submitted pictures. The case study offered a real feel-good moment for mobile research.

Thaddeus Fulford Jones (Locately) – has created a panel of mobile phone users in the US who are willing to allow the firm to capture location data and use this to model actual behavior. You can learn about the extent to which consumers do go to some outlets and often will drive past rivals in order to reach them. Raw location data is used to identify locations such as retailers, leisure destinations and other important consumer touchpoints. It tends to be most powerful when combined with other data to provide context. Location data also reveal useful temporal data – e.g. how long people really have spent in a store or even waiting at the checkout.

Hannu Verkasalo (Zokem) – spoke of  “on-device measurement” or using the mobile phone for passive data gathering. What came out was just how much you can measure passively, free from the response bias of a survey, when using a mobile device, from sites accessed, search terms entered and time spent on different activities to location data – what was accessed at home, at work or on the road. He also revealed  the very different ways that people consume mobile content on mobile devices compared to the web, and again the different profile of apps versus browsers in the content that people access. Hannu’s prognosis is that the mobile app is in the ascendant – which contradicted Bruce Hoang’s earlier analysis.

AJ. Johnson (IPSOS Mori) – chaired a panel session entitled “Debunking the myths of mobile research” and asserted that research needs to treat mobile very differently. People will be engaged, if you approach them via mobile research, but as researchers we have to be very transparent, open and honest with respondents.

Paul Berney (Mobile Marketing Association) – challenged research to take greater interest in mobile research. Mobile is the natural home of the digital native – the under 30’s who have grown up knowing nothing other than the internet and the mobile phone. It’s already changing the way that retailers are working and it fundamentally changes the engagement model for brands. It is a mistake to think that mobile is about the technology – it’s about people. Mobile is a two-way channel and if we don’t go there with our research, then others will.

To round up, a few common themes to emerge across the event that struck me as fresh:

  1. Mobile surveys can be a bit longer than we may have first thought. 8-12 questions is a common-sense length, but examples were presented of 30 and 60 questions, and much longer, when over an extended period. But is trying to push up the limit the start of the same slippery slope that has led to the downfall of online research?
  2. The experience in emerging markets and less mature markets is very different. The penetration of mobile is so high in emerging markets that it far exceeds every other channel except face-to-face – it is the natural equivalent of online research.
  3. In developed economies, there is an assumption that mobile research is a replacement for online. In reality, it seems to supplement it, and it is more of a replacement for telephone, face to face.
  4. Mobile research is not one thing – it’s a multimodal channel in its own right, embracing self completion, interview-administered, quantitative or qualitative, visual, textual, voice and image, or passive, observational, which can be augmented with location or temporal data.
  5. The sphere of mobile research is changing fast and it is continuing to evolve. It is not something that research can afford to ignore.

This entry was posted in Conference reports and tagged . No comments

Industry taking a twin-track approach to social media research

By | Published on 16th March 2011

Social media research continues to be one of the hottest topics in research. I’ve just been reviewing the abstracts for this year’s CASRO Technology Conference in New York in June, which I will be co-chairing, and of all the topics, its the one with the longest string of submissions. Not only that, but there is some diversity of opinion into what it is, how to do it, and whether it adds anything at all to the existing researchers’ toolkit. Closer to home, it’s a topic that will be debated in next week’s Research conference in London too.

Analysis technology used on social media research projects, based on the 17% of firms who are active in social media research

Social media research is also one of the new topics we focused on in our 2010 annual software survey, sponsored by Globalpark, the results of which are published today. There are some curious findings – and some predictable ones too – that add perspective to the current debate.

Our survey of over 200 research companies of all size around the world, shows social media research is still at the early-adopter stage,  accounting for revenue-generating activity in just 17% of the firms surveyed. Close to the same number – 19% – say they are unlikely to offer social media research, and of the remaining 63% who gave an answer, 31% say they are either experimenting with it and 32% are considering it for the future. Small firms and research companies in Europe are the least likely to be doing social media research and are also the most likely to have ruled it out, whereas large firms are the ones that are most active. The actual volumes of work are still low – we also asked how much revenue social media research accounted for. It is 5% or under for  two-thirds of the agencies that do it and tails off beyond that – but there appear to be some specialists emerging, with a handful of firms deriving more than 20% of their income from it.

Many firms are bullish about the future, though, with 20% predicting strong growth, and a further 52% anticipating some growth, with North America, and again the larger firms, most optimistic about its future.

As a technologist, I was most interested to see what technology firms were applying to what is, after all, something born out of technology. Were the tech-savvy gaining the upper hand, or were researchers taking the conventional, low-tech approach beloved of qualitative researchers. Again, it’s a bit of both. Of all the software-based or statistical methods we suggested for data analysis, the one that came top, was “manual methods”, used by 57%. For analysis, this followed by 54% citing “text mining” (as correspondents could pick all that they used). Text mining, though it uses some computing power, is also very much a hands-on method – but it’s good to see more than half turning to this method. Other methods make much less of an appearance, and the method that I consider shows most promise for dealing with the deluge of data, machine learning-based text classification, was bottom of the list, cited by one in six practitioners.

For data collection, technology was much more apparent – although it is hard to avoid here. We were still intrigued by the massive 54% who say they are using manual methods to harvest their social media data from the web; 57% were using web technologies to collect the data, and the more exotic methods were also fairly abundant, including using bots (43%), crowdsourcing (41%) and avatars (24%).

I’ll pick up on some of the other intriguing findings from the study later. But as the report is out now, you can pick up your own copy by visiting this webpage – and there will be a full report in the May issue of Research magazine.

This entry was posted in Blog and tagged , , , . No comments

Research firms reveal nine month technology lag (and a secret affair with the Mac)

By | Published on 14th December 2010

It’s not the first time I’ve postulated that MR firms can be laggards with their technology. An interesting early finding to emerge from the 2010 Globalpark MR Software Survey, a survey among research agencies worldwide carried out annually by meaning, provides some supporting evidence for this by looking at the actual technology being used to access the survey.

We’ve been able to analyse the browser string returned by the 550-some participants who responded to our survey invitations – which were targeted exclusively at MR companies across the globe. The browser string, which many MR survey packages capture automatically, reveals exactly what browser and also what operating system the survey participant used. It’s hard data, free from any response bias, as it is picked up from the routine chattering that goes on in the background between web server and web browser.

We thought it would be interesting to compare this with the current state of worldwide usage to see if MR differs, and if so, how. Overall, the figures are very close with respect to operating systems in use. We compared our figures with those from GlobalStats Stat Counter who measure usage in the same way, only on a somewhat grander scale: typically 15 billion page impressions from 3 million websites per month across the world.

The headline figure for Windows, at 93.3% among our MR participants, against the GlobalStat’s worldwide figure of 92.0% is within a whisker. Perhaps surprisingly too is the 5.9% figure for Mac OS X usage – a squeak away from the global figure of of 6.2%. Most MR software providers produce only Windows versions of their software, and even web-based software, such survey authoring tools or online analysis programs, which could be platform independent, are often locked down to Windows-only browsers.

Chart showing OS usage 2009 to 2010

The trend, which we can see by comparing browser string data from our 2010 survey with the equivalent from our 2009 survey, shows Mac usage has surged from 2% to virtually 6% in the last year. We cannot tell how many of these users were taking the survey at home on their own Macs (which is quite likely) but it’s clear they were responding to a work-oriented email on a platform that most MR software managers choose to ignore. If this is a continuing trend, this minority will be increasingly hard for software developers to dismiss as undeserving of their attention.

We can also see some interesting things happening with the flavours of Windows being used. It is well known that most corporate IT departments gave Windows Vista a miss when it came out in 2007, choosing to stick with the the ageing but more reliable XP. Even now, as XP approaches its tenth birthday, it is to be found on 70% of the PCs in our study that have Windows installed on them (65% overall). Among our 2009 participants, it was found on 74% of the Windows PCs. In the meantime, what little share Vista had accumulated has now mostly been ceded to the newest Windows 7, which launched in October last year, just before our fieldwork period. It has now grabbed nearly a quarter (23% among Windows PCs, 21% overall).

Mobile devices barely made an appearance – less than 1% of our sample. There again, we’d expect most researchers to realise that taking our survey on a handheld device was unlikely to be a joyous experience. The same is not true for iPad users. We tested it on iPad and it looked good: but only two showed up among 554 research professionals who clicked the invitation link.

Chart showing web browser usage in MR firms vs global usage

Browser usage was also remarkably close to the global figures overall. Between last year and this, the major trends are that Internet Explorer has been losing share to both Firefox and Google’s own Chrome browsers – and our MR sample show signs of the same trend. However, when we compared the trend with the global data, our November 2010 sample showed a remarkably close fit with what the rest of the world looked like around March and April this year. Worldwide, Internet Explorer dipped below the 50% mark in August and by November it had eased down further to 48%. Our November 2010 snapshot shows IE as the browser of choice among 54% of research users, down from 62% last year, while Firefox has gained strongly, and even Chrome and Safari have picked up users.

We’ve just started to look at the actual questions in the 2010 MR Software survey – we will have the results out in March 2011.

This entry was posted in Blog and tagged , , , . No comments

« Older Entries
Newer Entries »