Why do we care so much about looking into the future? In business, the answer lies in the power of using advance planning for staying ahead of the game, beating the competition and adding shareholder value. Punditry is a risky business, but we like it. It makes people put their money where their mouth is and I believe that 2014 is the year that ennui will set in around the term ‘Big Data’ as companies and commentators alike see that it is Fast Data in context that is, in fact, key.
So far, much of the work around Big Data has been experimental and at an earlier stage than many predictions would have you believe. Its actual application in the mainstream is limited to a few verticals – 2014 is seeing that change. Several factors are allowing the three Vs, i.e. volume, variety and velocity of Big Data to be harnessed to great advantage.
First, it is now easy to integrate open source software, such as the Hadoop stack for storage and large-scale processing of data, into an enterprise application environment. More importantly, data integration across sources and the ability to get to the most recent data in near realtime is providing more accurate results with significantly less time between insight and action.
What we term Fast Data – and believe to be the actionable side of the Big Data challenge – is where companies are realising true competitive advantage as the timeliness of data is being captured and employed, rather than just its size, speed or variety.
Second, organisations are developing the ability to precisely identify patterns of events that lead to a business impact. We now see data scientists identifying patterns that are meaningful, so that real-time events (even from streaming data) can be consumed by event processing technology. Based on patterns identified by data scientists and combined with real-time events via integration, event processing and streaming technologies can spot these key situations.
Third, and in my opinion, most significant, analytics technologies and better modelling are allowing enterprises to extract the full value of Big Data. Data scientists can now understand the complete context of an area of interest and use this correlation to optimise revenue, or say for an airline, predict potential mechanical problems and schedule proactive maintenance rather than face disruption.
This context is vital in allowing organisations to prevent a bad situation before it happens, or foster a good one. This can only happen when organisations can see the leading indicators that matter the most, distil real insights and act in realtime.
The buzz around Big Data continues, with companies aiming to address the Big Data challenge. However, for Big Data and its currently ill-defined goals, to find more applications across all industries, the conversation needs to be refocused away from the catch all-term ‘Big Data’ towards how companies can harness the power of Fast Data and reap the rewards of achieving timely, actionable insight in context.