The Economist recently ran an article on “Big Data” in a special report on International Banking. Its assessment of banking elsewhere in the report is that the industry has been surprisingly resistant to embracing the Internet as an agent of change in banking practice. It reveals, counter-intuitively, that number of bank branches has actually risen by 10-20% in most developed economies during a period when most customers pass through their doors once a year rather than once a week.

The newspaper explains this paradox thus: banks with a denser branch network tend to do better, so adding more branches is rewarded by more business. But it’s business on the bank’s terms, not necessarily the customer’s. It does not increase efficiency – it increases cost. And, as The Economist points out, banks’ response in general to customers using mobile phones for banking has been lacklustre, even though customers love it and tend to use it to keep in daily contact with their accounts. It’s a level of engagement that most panel providers would envy.

All of which is to say that there are parallels here with our own industry. Here at Meaning, we’re just about to release the findings of the latest annual MR Software Survey, sponsored by Confirmit. In a sneak peek, Confirmit blogger Ole Andresen focuses on an alarming finding about the lack of smartphone preparedness among most research companies.

But what interests me is the Big Data – both in The Economist’s report and our own. The former offered a fascinating glimpse into the way banks were using technology to read unstructured text and extract meaning, profiling some of the players involved and the relative strengths of different methods. This is technology which is improving rapidly and can already do a better job than humans.

In our annual survey this year, we have asked a series of questions on unstructured text. Research companies, in embracing social media, “socialising” their online panels and designing online surveys with more open, exploratory questions in them, are opening the floodgates to a deluge of words that need analysing: at least that was what we suspected.

Analysis methods cited by research companies for handling unstructured text, from the 2011 Confirmit MR Software Survey by meaning ltd

In our survey we asked a series of questions on unstructured text. Research companies – in embracing social media, “socialising” their online panels and designing online surveys with more open, exploratory question – are opening the floodgates to a deluge of words that need analysing: at least, that was what we suspected.

It turns out that half of the 230 companies surveyed see an increase in the amount of unstructured text they handle from online quant surveys, and slightly more (55%) from online qual and social media work. Yet the kinds of text analytic technologies that banks and other industry sectors now rely on are barely making an impact in MR.

Even a quick glance at the accompanying chart shows that most research companies are barely scratching the surface of this problem. It’s not the only area where market research looks as if technology has moved on, and opened a gap between what is possible and what is practised. There’s much more on this in our report, which will be publishing in full on the 30th May. Highlights will also be appearing in the June issue of Research magazine.