2012 Prediction: Cloud Will Provide New Opportunity For Benchmarking Insights

Cloud Benchmarking

The big question for IT professionals is if your operations are efficient. Looking at some of the areas comprising IT operations – change management, performance management, service management etc. – IT managers are often left asking themselves without a frame of reference, “Is this process good enough?”

The same question comes up when assessing technical details of the process. For example, if I plan to change a parameter like Connection Timeout to 120, “is that good, or not?”

There are not a lot of vendors providing a knowledgebase for evaluating this. So IT has just gotten used to the idea that they need to make these evaluations based on their experience, or based on what they have learned from their peers.

Traditional Benchmarking Challenges

Traditionally, survey-based benchmarking has not focused on low level, minute details. At the same time benchmarking, established through research conducted in lab settings, produced very limited results and information that was hard to extrapolate.

Furthermore, typically research needed to be purchased, and the research was limited due to lab conditions, requiring setup effort – installing a server, setting it up, letting it run – further compounding the challenge of carrying out benchmarking.

Cloud Changes Everything for Benchmarking

Now, with the advent of the cloud, the whole subject of benchmarking will be changing. The actual, real-world data is accessible and right there. No surveys are required, and no servers need to be set up for research purposes. Due to expanding use of Software-As-A-Service, the data for setting benchmark requirements sits in the cloud, requiring just a bit of data mining.

The benchmarking process and effort can be automated. For instance, benchmark information can be extracted from a particular event, or from a day-to-day activity reporting to a central cloud based repository. Benchmark results can even be updated automatically at regular intervals.

The Added-value Of Software-As-A-Service

This is all the beauty of Software-as-a-Service, more than the cloud itself, where lots of data from diverse clients is consolidated into one facility. Vendors providing the facility can use this data to help clients operating in this facility.

There is added-value that will need to be unlocked, and some vendors have started to do this. This is evident by the fact that some SaaS applications are now publicizing value-added benchmarking services for their customers.

Moreover, the benchmarked data can be analyzed and used proactively to warn clients. Not just as a passive metric, but applied actively. Beyond the benchmark setting, vendors can convey not just the information that clients should measure themselves on, but now they will be able to actively recommend to their clients which setting to apply, actively involving the vendors in helping to manage the business.

Since all the data will be accessible, vendors can carry out top-down or bottom-up analysis, providing benchmarking data to the business service. For the first time in history, we will be able to apply an end-to-end perspective, and determine in terms of a business service how you are performing as compared to similar organizations.

Now with the cloud-based systems, we can look forward to consolidating all our data to improve everyone’s business.

Sasha Gilenson is Chief Executive Officer, co-founder, of Evolven. Sasha enjoyed a long and successful career at Mercury Interactive (acquired by HP), having led the company's QA organisation, participating in establishing Mercury's Software as a Service (SaaS), as well as leading a Business Unit in Europe and Asia. Sasha holds an M.Sc. in Computer Science from Latvian University and MBA from London Business School.