Big Data: Moving Beyond The Hype

Hailed as the next big technology trend by vendors and analysts alike, Big Data is big news. It’s also big business. With data volumes growing exponentially, statistics are continually published in the effort to convey how big is Big when it comes to Big Data. According to IDC, 2.7 zetabytes of digital data exists today, a staggering increase of 48 percent from 2011. It’s also said that the data available for analysis is growing at a rate of 2.5 billion gigabytes per day.

While Big Data is clearly a key business priority that is expected to add significant value to organisations, it has also become difficult to get beyond the hype. I believe that Big Data is simply a natural progression of how organisations access, analyse and use information for the running of their businesses. Therefore I see Big Data as an evolution (rather than a revolution) that above all compels organisations to re-examine what they traditionally consider as business information.

Today businesses want relevant information at their fingertips and they want the ability to analyse this information quickly and easily. The challenge is that data has become so vast and varied that the traditional approaches to managing and analysing data can no longer meet the increasing demand. The good news is the technology is available to tackle these challenges, and Big Data tools can deliver new levels of insight fast.

However knowing where to start can be overwhelming. The real key to success lies in how you go about identifying the data that will be useful and relevant to your organisation, how you examine this data, and then understanding how to store, categories, organise and use it for competitive advantage.

There are plenty of real-world applications of Big Data today. For example in the insurance world, catastrophe and loss modelling are the two biggest data analysis challenges. Big Data is helping insurance companies better understand how events are changing and the effect this might have so that they can better manage risk.

Retailers are using Big Data so that they can provide dynamic pricing and predictive analytics. They are creating up-to-the minute customer profiles which allow them to better predict buying patterns. Big brands are also using Big Data to provide better customer service. By harnessing unstructured information that sits outside the organisation, they can find out what customers are actually saying about them at any given moment and respond accordingly in real-time.

While the benefits of Big Data are potentially immense, we believe that a good dose of common sense and pragmatism needs to be applied when approaching a Big Data project. Successful Big Data projects first clearly identify the business requirements before applying the analytics to support the business objectives. In this way new insight can be gleaned from existing sources of information rapidly and to a level not previously thought possible from the traditional methods.

Key questions I recommend asking are based around a best practice approach that any organisation should adopt when embarking on an IT project and include: What are the key Big Data requirements that will provide the most value to the business? Have I established a strong business case based on measurable outcomes? Do I have business leader sponsorship for my project and can I establish a pilot project that will deliver a quick win?

I recommend that those considering a Big Data initiative first focus on tangible business outcomes and then think small to think big. It might be counterintuitive given all the hype around Big Data, but it could make all the difference in achieving successful outcomes. Here I provide practical recommendations on how to approach a Big Data project:

1. Understand that Big Data is a business-driven solution

Success will be dependent on meeting the needs of lines of business – IT is the enabler. First identify the business requirements, then look to the infrastructure, data sets and analytics to support the business opportunity.

2. Establish a clear business case

For many organisations, the traditional approach to data analytics has limitations. Put a cost on it – it’s the difference between having information at your fingertips in minutes as opposed to days, weeks or even months.

3. Start small, focus on quick wins

Don’t try to analyse everything at once or you will struggle. Focus on a specific area that will deliver a return to demonstrate the capability of the technology. Then look to broaden your wish

4. Take a staged approach

Start a pilot programme by selecting a business unit or function where you think the Big Data opportunities and benefits will be. Develop proof of concepts or prototypes before you make huge technology investments. A gap analysis between your current state and desired outcome will be helpful! Where possible, benchmark yourself with industry best practice.

SHARETweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Pin on PinterestDigg thisShare on RedditShare on TumblrShare on StumbleUponEmail this to someone

David became CEO of Northdoor on 1st July 2010. Prior to that, as consultancy director, David led a team that cares about its clients’ businesses and whose approach is always to ask ‘how can we help?’ He sees his role as one of building a team with a broad range of expertise and a single culture of responsibility and a proactive attitude. He believes it is extremely important for clients to be able to trust the team’s business understanding as well as its IT knowledge and skills.

  • As Ballard suggests,there is much hype around big data and the difference, if any, it can make to your business. Big data has begun to enter our everyday lives and means that organisations are collecting vast amounts of information about their customers for analysis. However, with a focus on big data and a rush to embrace the unknown and often unnecessary realm of it, organisations are missing opportunities that lie within the wealth of information they already possess.

    The subject brings much controversy and as Ballard comments, knowing where to start with your data alone can be overwhelming. The swiftest and most accessible route into data analysis is the emerging discipline of visual analytics, which utilises our visual perception system and its innate pattern discovery;no-one needs a degree in statistics or mathematics to be an effective visual analyst. The best of the current breed of visual analytics software encourages exploration of data, of all forms and sizes, and enables simple, effective communication of insights discovered to non-technical audiences.

    For smaller organisations and in particular SMEs, the focus should be on utilising the readily available data they already possess more thoroughly, imaginatively and effectively before investing in the technologically and ethically complex domain of ‘big data’.

    Guy Cuthbert
    Managing Director
    Atheon Analytics