How To Navigate The Information Lifecycle Minefield

Information Management Lifecycle

The current megatrends of mobile, social, analytics and cloud are driving digital business transformation. This is raising the stakes for existing businesses. Through these new technologies, companies are generating a huge amount of data but are unsure how it can be commercialised to gain competitive advantage, used to make better business decision or predict business scenarios. This is true in both B2B and B2C situations.

As the value and volume of an organisation’s data changes, IT professionals must manage it accordingly, choosing to retire or archive appropriate data sets, or ultimately remove them altogether. But when all that data lives in a private cloud or hybrid cloud environment, it adds a host of extra challenges around information lifecycle management (ILM).

Traditional operational structured data sources are being augmented with social, web, government, as well as historically hard to use enterprise dark data (unstructured doc libraries, archives) and vast volumes of machine generated data (sensor and log).

The Private vs Public Dilemma

CIOs must decide what bits of data can be tied to different deployment models – for instance private, public or maybe a combination of both – hybrid. There are a number of issues to consider when making this decision, influenced by the five V’s of big data – Volume, Velocity, Variety, Veracity and Value. Firstly, latency is an issue to evaluate; if you can’t afford to have latency issues then your data would be better stored locally. Use cases where real-time decision making is required. Live data sources traditionally fit this model, scenarios such as high frequency trading, A/B testing, real-time fraud detection.

Availability is another issue that companies must consider; CIOs need to ensure that they have the right back-up, recovery methods and right availability contracts in place to make the decision on where to put their data. It’s important to scrutinise those contracts and check that the provider has the provision and elasticity that they reference in your contract. Access to data is also key, it must be easy to get data in and out, vendor lock-in here is a significant risk.

Lastly, there are always cost considerations when taking data to and from the cloud – organisations need to clean up their data before it can go in the cloud otherwise they could be saddled with a huge cost burden. Consider whether the workloads are committed, predictive or variable before deciding whether they should go in the cloud.

Cloud Migration: Not Why, But How?

Cloud provides businesses with agility, automation, and it gives them a new set of capabilities to do things like re-stores. CIOs must consider the traditional ILM methods and how these can be extended when moving to the cloud so that the same level of governance and oversight is received from the traditional lifecycle methods you had before.

But with so many different providers in the marketplace, picking the right one can be daunting for business decision makers. Firstly, you need to consider functionality. Will a particular cloud provider be able to offer you the right ‘class’ of service, and can they advise you on the right cloud platform that is most suited for your business, for instance hybrid cloud? Other considerations include assessing whether the cost models and performance models are the right fit for your business? Senior IT professionals also need to think about other factors including the workflow, access, control, authorisation, auditing and reporting.

To ensure high costs aren’t accumulated when placing data in the cloud, the right offerings and capabilities need to be offered by the provider, such as object storage and parallel access and the Hadoop Distributed File System (HDFS).

Tiers For Fears

Modern ILM strategies involve a multi-tiered approach to handling data primarily for cost management reasons. Crucial factors in this decision making process are the availability of data, performance, back-up and recovery, and price. Companies must also find a provider with the appropriate ILM and security tools, but ones which don’t compromise the ease at which data can be stored and extracted.

A balance also needs to be struck between tiers, such as with mission-critical applications and older records. Archived data can be stored on slower cheaper storage. For business critical apps, trust and control are important considerations. The IT department needs to understand the controls required by the business, what controls are already in place, and then identify the control weaknesses before putting a remediation plan into action to fix those weaknesses. Understanding where and when to put business critical apps in the cloud will become even more important in 2014-15 as businesses undertake pilot projects for business functions.

Digital Evolution

Companies which get their ILM infrastructure right today, can avoid costly alterations in the coming years as big data becomes more established and the importance of real-time analytics grows. Fast, reliable and secure access to information will be key in the race to become a truly digital business and remain competitive in the cloud-enabled world.

Sean Catlin

Sean Catlin is currently CTO at Canopy, the cloud arm of Atos. Sean's had an interesting career and is incredibly knowledgeable having previously worked in IT for financial services - he was the chief architect at Investec, he's been the head of architecture at AXA and consulted for Nomura and Goldman Sachs.