Big Data And The Untapped Information Goldmine

‘Big data’ has become very much the business challenge of the moment. IDC recently predicted that the market is expected to grow from $3.2 billion in 2010 to $16.9 billion in 2015, representing 40 per cent annual growth.

The proliferation of consumer technology, particularly smartphone devices, in the workplace has meant that people are engaging with technology in new ways and as a result, producing more data today than ever before, and at a much faster rate. On a daily basis it is estimated that we collectively produce 2.5 quintillion bytes of data, and the growth rate is so high that 90 per cent of all information ever created was produced in the last two years alone.

The death of the relational database

In the business world, data has traditionally been housed in databases which have been a simple and effective way to keep information safe and stored in a uniformed fashion. However, the database was created as a way to work around the limitations of computers half a century ago.

As the majority of information flowing through organisations today is unstructured or ‘human information’ (90 per cent), such as text, email, video, and audio, and therefore impossible to box into traditional databases, being able to manage that information so you know where it is and process it so you know what it means is essential.

For instance, you could store call centre recordings into a database if you really had to, but you would have no idea whether the customer was happy or not. This is exactly the type of insight businesses want to know after all. To understand them organisations have to take a new approach to managing and uncovering the meaning of their data.

Understanding meaning

The next-age information platforms make sense, and understand the meaning, of information and use it to solve problems in order to give businesses a better understanding of themselves, their customers and the competition. This is what has got the analyst community so excited due to the sheer size of the market opportunity.

It is about time that we got the machine to fit to humans and not the other way around. Information can be a powerful tool to help organisations make better and more effective business decisions, get the edge over the competition and increase revenue.

The power of insight

To see just how human information can add value, just think of a marketer for an online retailer. At its inception, eCommerce was simply an alternative point of sale for traditional retailers; then it became an active channel to actually entice consumers into buying products and services.

Now, the availability of vast quantities of data on current or potential customers offers online retailers huge potential value – if they can gather and analyse information efficiently and in a timely manner. Customer browsing and purchasing habits create sizeable data trails, and, coupled with social media integration, can allow content to be effectively targeted.

Seeing beyond the bytes

The challenge is that having tens or hundreds of thousands of customers, each with potentially thousands of data points connected with them, results in a very large amount of data being collected. Part of this might be structured data that can be easily categorised, for example, gender, age, and geography, but the majority will be unstructured, human information.

The content and sentiment of a product review, for instance, is not something that can be slotted neatly into a standard database, but the ability to tap into information like this has huge implications for driving sales or safeguarding a reputation.

Unlocking real business value

The key is to understand that data – leaving it exactly where it is – in order to create an infinitely scalable platform, and a powerful basis for analysis and action throughout the entire enterprise. By applying meaning, we can cut through the Gordian Knot of trying to find that ‘perfect’ database, and get to the heart of the issue – being able to process 100 per cent of the information, structured and unstructured, to unlock real business value.

By applying the ability to understand meaning, businesses can get a view of all of their data, not only the 10 per cent of ‘neat’, structured data, but the whole hundred per cent. What we can do now has never been possible before: the next IT revolution is happening in the ‘I’, not the ‘T’.

Nigel is Autonomy’s subject area expert advising customers across EMEA on how to meet key enterprise infrastructure challenges through solutions for Pan-Enterprise Search, Search-based Applications and Analytics and Collaboration Tools. Nigel is actively engaged with major Autonomy customers ranging from large government bodies to private organisations across all vertical sectors. Having joined Autonomy from Meridio in 2007, Nigel possesses extensive expertise in Records Management.