Does Your Business Have Obsessive Compulsive Big Data Disorder?

Big data won’t be referred to as ‘big’ forever. That’s not to say that it will shrink in size (quite the opposite in fact), but rather that organisations will be so used to dealing with this amount of data, that it will become the norm. In the meantime, many organisations are still struggling to come to grips with how best to manage the data deluge they are facing.

The main thrust of the conversation around data so far has been the need to collect it and store it, meaning that the focus for many has been almost exclusively on ensuring that they have the technology in place to allow them to physically hang on to it. A large government transport department, for example generates 0.5 terabytes of ticketing every single day and large investment banks can generate up to and over a 100 terabytes of risk data every day.

Data capture alone however, is not a data strategy. All the rhetoric around data capture means that a problem that organisations are increasingly facing, is knowing when to let go. Having gone to so much effort to find and collect the data, ‘data hoarding’ is perhaps an understandably easy (although costly) trap to fall into.

However, if businesses are to avoid getting trapped in a data collection loop, decisions must be made around which data is actually of use to the business, which must be kept for regulatory purposes, and which…is simply taking up valuable storage space.

The true value from data comes from being able to contextualise, consume and understand it. There is no point in collecting and storing data if it’s completely incomprehensible to the people who need to make decisions based on it. Most of us are proficient data managers in our day to day lives, navigating Twitter, Facebook and social internet search with ease to understand who said what, when, where, and in response to what.

This approach to data in our personal lives has naturally had an impact of our expectations of how we should be able to navigate enterprise data, and decision makers are starting to ask “why not?” of the data they handle at work.

Why can’t they link through to see who that person is connected with? Why can’t they simply connect through to a communication device by clicking on their name? Why can’t they see “trends” or sentiment in real time? The trick to making enterprise data ‘edible’ therefore, will ultimately lie in making business data look more consumer grade.

Data is not decorative, and should be kept for function only. Those organisations that can truly draw maximum benefit from big data will be those who are able to see data as an organisational asset and a business tool, helping organisations to be more scalable, agile and adaptable.

Ultimately it boils down to doing things for the right reasons. A big data “initiative” is not different from any other business initiative and therefore needs to be justified by the return expected of the initiative. Any Big Data strategy should be part of a wider Information Strategy within an organisation and if it isn’t then one should seriously question the initiative.

If it does make sense for the organisation to leverage Big Data at this stage, for reasons of strategic and competitive advantage, then doing nothing is not necessarily an alternative. Big Data is not just “bigger data”. Being prepared for Big Data requires more than cruising along with your current initiatives and infrastructure.

Big Data represents a paradigm shift in the way this data is stored, integrated and queried. Therefore, it does require an organisation to possess a set of skills, across people, process and technology that it may not have at the moment to cope with Big Data going forward. However, before making that leap, it’s essential to ensure that it’s really needed and that it will add value to the business in the long term!

SHARETweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Pin on PinterestDigg thisShare on RedditShare on TumblrShare on StumbleUponEmail this to someone

Launching his professional career within the Data Warehousing practice of PricewaterhouseCoopers and then spending a few years with an analytics product development and market research firm, Suranjan Som is currently Practice Manager focussing on Business Intelligence, Information Strategy and Capital Markets at Information Management Group, headquartered in London. Working with some of the largest names in their space, Suranjan is regarded an authority in the analytics space around Investment Banking Risk Management, Linear and Non-Linear Broadcast Media Analytics, Retail Sales and Logistics Analysis and Legal and Professional Services Performance Management.