Big Data: It’s Not Just About Social Media

Data is kind of like alligators. When you’re surrounded by too much of it, it’s easy to lose sight of your original purpose. So it is with social media data. The fire hose of tweets, status updates, chat, photos, etc, has convinced us that there’s a new imperative to better segment and “understand” our customers, followers and other audiences. And a bucket load of new tools has emerged to help us do so. Hence the steady ascent of Big Data in many people’s priorities.

Ignore for a minute that there’s very little new about Big Data. (The definition gives it away: Wikipedia defines Big Data as “data sets whose size is beyond the ability of commonly used software tools to capture, manage, and process the data within a tolerable elapsed time”.

By that definition, people have struggled with big data ever since the invention of the stone tablet. The iPad is prettier, but no better matched to the data sets we now want to throw at it.) Our emphasis on these new interaction channels risks blinding us to the fact that our goal isn’t to segment customers; it’s to satisfy them.

In order to satisfy customers, it’s not enough for us to know what they want. We also need to be able to deliver it. That means our internal operations must be up to the task. I think this is where the tools and techniques associated with Big Data could be most valuable for many organisations.

Consider, for example, the following scenarios:

  • A university offering online courses. If we can spot signs that a student is having difficulty in certain aspects of a course, then we can intervene at an early point, before they get frustrated with the whole course and drop out. Those early warning signs are probably there, in the records of interaction patterns, test scores, etc. We just need to undertake the appropriate analysis to recognise them and bring them to a tutor’s attention. This simply wasn’t feasible for most institutions with the first generation of learning management systems a decade ago, but the “big data” tools we have available now are made for this type of analysis.
  • An online marketplace for second-hand household appliances and their associated spare parts and servicing. The trading patterns can tell us an enormous amount about the appliances – which ones people trade in quickly, which ones they hang on to for years, what parts need to be replaced regularly, what accessories people buy for different appliances. This is all invaluable input when we start designing new products, managing parts inventory, building service infrastructure, and so on. And again, the challenge of gathering and analysing the trading data has become a lot more tractable in recent years.
  • A logistics company running a large fleet of trucks. Data about fuel usage, engine performance, travel times, etc, could help us optimise service and refuelling schedules and suchlike, thus reducing operational costs while improving our ability to meet tight, just-in-time delivery schedules for customers. Rolls Royce has been doing this sort of analysis for jet engines for years. Commonly available “big data” tools are bringing it within reach of a far wider range of companies.

In each of these cases, the organisation can create a lot of value by improving the way it senses and responds to external events. By responding at the right time and by adjusting our response to the specific circumstances, we can reduce customer turnover, increase their satisfaction with our products, and reduce operational costs.

In general, the more we understand about our internal processes – the nature of the information flows, the bottlenecks, the communication breakdowns, etc – the better we are able to tune them so we can respond rapidly and in ways that add most value for our customers.

The key here is to understand the processes as they actually happen, not as the process documentation says they’re supposed to happen. This is where Big Data comes in. If we can use its tools and techniques to collect process data and analyse our operations, then we’ll have much more insight into which factors drive operational effectiveness, which bottlenecks need to be eliminated, and which types of response are most effective in any given circumstances.

Of course, listening to customers, understanding their needs, and hence segmenting them more effectively, is important too. Applying the tools, skills and mindset of Big Data to that social media data could almost certainly create immense insight for many organisations. But insight is only valuable if you can act on it. That requires internal capabilities. Put some of your Big Data investment into understanding and improving your internal operations.

Dr Graham Oakes is a highly skilled systems engineer and project manager with over 20 years’ industry-proven experience backed by a track record of delivering highly innovative and effective solutions. Graham helps people untangle complex technology, relationships, processes and governance. As an independent consultant, he helps organisations such as Sony Computer Entertainment, The Open University, the Council of Europe and the Port of Dover to define strategy, initiate projects and hence run those projects effectively. Prior to going independent, he held positions including Director of Technology at Sapient, and Head of Project Management for Psygnosis (a subsidiary of Sony). His book, Project Reviews, Assurance and Governance, was published by Gower in October, 2008.