How To Ensure That Big Data Stays Out Of Big Trouble

Big Data

Once you’ve brought big data down to eye level, look closely, and you see what big data is made of: many smaller datasets. Alone, each dataset may provide value. Stitched together, they offer big value. In the consumer goods industry, for example, executives get a full understanding of customer behaviour only when they’ve blended sentiment data with purchase data.

You get a rich variety of data through loyalty cards, but it’s important to blend all this data together to understand why people enter the shop and fill their baskets, if you are a consumer goods company. It enables you to anticipate popular products and new trends.

The most value goes to organisations that blend relational, semi-structured, and raw data – with minimal up-front cost and without bothering business users about the technology. It’s done, and that’s good enough. Whether your data is in a spreadsheet, a database, a data warehouse, open source file systems like Hadoop, or in all of those, you need the flexibility to quickly connect to data and consolidate it. That lets you ask and answer questions as they come to mind – no matter how big, or small, your big data may be.

Ensure That Big Data Stays Out Of Big Trouble

Big data can be like a sandbox. You can get in there and build and shape things with freedom. That mass of data is valuable partly because it’s often about real people. Governments, not to mention ethics, have a lot of sway in this area. More than 80 countries now have data privacy laws. The European Union defines seven “safe harbour privacy principles” for the protection of EU citizens’ personal data. In the U.S., Sarbanes-Oxley puts all publicly traded companies on notice, and the Health Insurance Portability and Accountability Act (HIPAA) sets national standards for healthcare privacy.

So before you dive deeply into the big data ocean, look seriously at your needs for adhering to governance and privacy standards. Are you a healthcare organisation subject to HIPAA? Or operating in certain areas of the world? Or do you just realise that it’s smart to take precautions?

If your organisation must be compliant, one obvious solution is master data management, which tightens up data use around the organisation. If you’ve got it, you’re all set. However, coming to agreement on definitions and business rules can be slow for most who try it. Painful perhaps, but it’s certainly pragmatic. Don’t bypass governance for the sake of agility and fast results is the advice from Forrester Consulting in its 2013 report “Big Data Needs Agile Information And Integration Governance.”

Forrester recommends against adhering to “a single set of standards, policies, and practices,” which it found “stifles the value that can be achieved from big data investment and insights.” Instead, the report suggests adopting governance to match analytic capabilities and objectives, establish governance “zones” considering the data’s source, type, and test before you put rules into production. It’s an exciting field where a fast moving data-head can make their mark and a big difference to the business.

Bob Middleton is a Product Marketing Manager at Tableau Software.