Why Do So Many Big Data Analysis Projects Fail?

Big Data Analysis

Many companies think that they can effectively extract value from their data, for business or operational advantages. Research among 1,650 US and European companies from Iron Mountain shows that 40 percent of big data analytics projects fail and three of four companies get almost no advantage from their business data. Is there an obvious cause for this problem?

Over the past 20 years, the information landscape has changed dramatically. Not only has the amount of data we store increased, but there are also many more types of data available. In addition to operational and commercial data, nowadays, more and more information is being collected through websites, mobile apps and IoT sensors.

Within these areas, all kinds of valuable insights are hidden and companies often don’t have the technical resources and skills to exploit the data. The underlying problem is usually an inadequate or absent data strategy. Only when organisations define a clear data strategy, will they invest in adopting the appropriate analytical software and skilled staff to implement that strategy.

Insights Are Not Shared

Modern organisations that want to be successful and remain so will have to filter value from their data by means of data analysis. Yet this is hardly done in most organisations. And if it happens, it is usually by a select group of employees who have to rely on the IT department for workable data. This means that the whole organisation is not rallied behind the data strategy, and analysis is done from a limited perspective.

This is exacerbated by the fact that people sometimes only use data and do analyses that suit their own business objectives. These insights are then rarely shared outside a department or team. This is a major reason why organisations struggle to get valuable insights from their data. If the organisation can’t support a data strategy and benefit from the insights, analytics projects will always fail.

Accessed Data Is Incorrect

An additional problem in organisations that do data analysis on a small-scale, is that the teams are often happy with the results, as long as they tell the story they want to tell. Whether or not the figures are correct, is less relevant to them. This is a dangerous way to deal with data analysis, because decisions may be made on the basis of incorrect data. In addition to being an organisation that fully supports data strategy, there is therefore a need for a comprehensive data analytics platform with an integrated governance framework to centrally manage and distribute data.

Within such a system, information and analysis can be shared safely throughout the organisation, but from one central source of information, and within the applicable legal regulations for the protection of privacy and intellectual property. Data is managed in a consistent manner, anyone can gain access to relevant information, and more importantly, it eliminates data silos with outdated or incomplete information.

Who Among The Information Elite?

It is increasingly clear that data yields the most value within organisations when normal business users can work with it, without time-consuming interactions with the IT division. In other words: by using self-service BI, aimed at uncovering new insights based on questions from the organisation.

However, research shows that only a very small portion of organisations are using data optimally. Only 4 percent of them belong to the ‘information elite’, as defined in the study. These are companies with a good information strategy, data governance and the knowledge and resources to make the best of data analysis. But the majority of companies still have a long way to go to extract real value from their data.

Yet, there certainly is still time to catch up. It starts with an open corporate culture and a data strategy. The value of data must be recognised, and the company must be willing to invest in the knowledge and resources for centrally managed data analysis. Big data and analytics projects will inevitably fail, if they are thought of purely from the perspective of IT and technological solutions.

SHARETweet about this on TwitterShare on LinkedInShare on FacebookShare on Google+Pin on PinterestDigg thisShare on RedditShare on TumblrShare on StumbleUponEmail this to someone

David Brierley

David Brierley is vice president of sales, EMEA and APJ. He is responsible for executing on Pyramid Analytics’ international expansion plans, which consist of scaling the field sales and partnering organisation across EMEA and the APAC region, along with building out the company’s partner ecosystem.