Big Data: The Challenge

Big Data Challenges

A topic receiving a lot of coverage in the media this year is Big Data and all things data related, and yet we have not really seen the high levels of adoption that was expected. Gartner stated that 42% of businesses will have invested in big data technology within the next year, but there does seem to be a delay in the uptake. Why is this?

There is a wealth of unstructured information in the digital world, revealing consumer’s thoughts, preferences, and feelings on products and services. Corporations would be foolish not to capitalise on this freely available data in order to review strategy and gain competitive advantage, and yet results from a recent poll of CIOs revealed that most felt although Big Data might be useful, it was not a real game changer. Perhaps the corporations are not seeing the potential benefits, or do not truly understand how to harness the data and make use of it once acquired.

Confusing Historic Data Analytics With Unstructured Data Benefits

Data Analytics has been around for a long time, evolving as technology has evolved. Capturing information and performing statistical analysis in order to drive decisions or determine how to invest, is not a new concept.

What is new, is the many unstructured sources that can now be mined for valuable information. Perhaps this is what the organisations are failing to see. Historic analytics focused on prescribed responses to known questions or inputs. For example, a car can send a vast amount of telemetry data to a garage computer, generating a lot of data to mine, but this data is a known and expected type and quantity.

Unstructured data analytics looks at the commentary, the discussion, the preferences and views expressed by consumers based upon their personal discoveries and interactions with their friends, and a global audience. Being able to tap into this world-wide ‘focus group’ can yield unprecedented strategic value if done correctly, so what is holding back investment in this concept?

Concerns Around Misuse of Data

Perhaps organisations are concerned about intellectual property law and how free this data truly is. Corporations need to be careful about what data is gathered, and how the different types of data are used. Customer specific data is gathered from the corporations own systems, based on consumption of their products and services, or the prescribed responses we discussed earlier, hence existing feeds. Public commentary exists on the social sites such as Facebook, Twitter, blogs, and forums.

Telecommunication companies have perhaps the greatest opportunity to capitalise on the Big Data phenomenon, but also the trickiest tightrope to walk. In May the Sunday Times published an article about a telecoms provider selling mobile phone data to IpsosMori for 27 million customers.

The data was said to include gender, age, postcode, websites visited, when texts were sent, and more. The telecoms company confirmed the data is aggregated and anonymised so cannot be used to identify individuals, but this sort of report leaves companies concerned.

Data Ownership Concerns

If you look at the unstructured data freely available on Facebook, Twitter, and social feeds, you start to see a different type of data, in consumer’s thought and views rather than their geographic location and texting patterns. Whereas copyright law will usually cover creative writing by individuals, for short comments like those used on Twitter and Facebook, unless explicitly stated, the same rules and policies do not apply.

It is a challenge to clearly describe the copyrighted content in these posts, hence someone declaring their preference for Coca-Cola over Pepsi does not contain anything that can legally be deemed IP. Will this change in the future, perhaps, as data privacy laws are in a state of constant flux, especially in light of snooping allegations in recent months. For now this sort of content is an untapped mine for product and service analysis, that can be used to develop corporate strategy.

Why Do Corporations Not See The Potential?

Perhaps the problem is understanding how to aggregate this data into something tangible that can be used to determine market views and demands in relation to corporate strategy. The collected data is only useful if the mechanism for mining the data is good, and in the world of unstructured data, this is dependent on those business analysts able to probe with the best questions and filters. If corporations are not shown the true capability of Big Data, it is not surprising that they are reluctant to take the next step.

Extended Corporate Knowledge

When mined correctly, the wealth of commentary and thought patterns extracted from this unstructured data can help companies to make agile strategic decisions, based upon real world up to date analysis. This knowledge can present a tangible competitive advantage over market rivals.

SHARETweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Pin on PinterestDigg thisShare on RedditShare on TumblrShare on StumbleUponEmail this to someone
Scott Cairns

Scott Cairns is CTO at T-Systems UK. Scott focuses on strategic innovation and future concepts.