One of the most common factors I’ve observed, across all of the organisations I’ve worked, is that every one of them, regardless of size, acknowledged that testing should be, and should also be seen to be, independent. However, every one of them subsumed the testing organisation within the delivery function, which begs the question – where does testing sit within an organisation.
Clearly, any such organisational arrangement will invariably result in tension between the testing practitioners and the development and project management communities, many of whom view testing as, at best, a hindrance to the implementation of their projects.
In the majority of cases, this view has been reinforced by the tendency of the delivery organisations management to disregard the output from testing in order to ensure projects and programmes delivered on time, if not necessarily to the required quality.
Like a lot of textbooks and writers, I like think of the term ‘testing’ as short hand for ‘Independent Verification, Validation and Testing (IV&VT)’. I would argue that, whenever IV&VT is undertaken, the objective is to ascertain the risk to the organisation of accepting the item under test as being ready for use in the live operational environment.
This objective applies across the whole spectrum of organisations and associated IV&VT activities, whether the item being tested is an IT system, a fighter jet or a commercial airliner. In this context, IV&VT is a risk assessment tool for the organisation, and one that serves a clear, identifiable purpose with this in mind.
However, most organisations seem more interested in having the question of ‘have all the test scripts been executed’ answered in the affirmative, rather than ensuring that this risk is mitigated. As a result, there’s a perception of ‘testing’ as simply being the execution of test scripts.
The unfortunate truth is that the testing community has done a lot to foster this perception, with most processes and procedures addressing the techniques for deriving better quality test scripts and test coverage. For example, the ‘risk based’ approach to testing is usually presented as nothing more than a technique for prioritising the order of test scripts to be executed and, as such, appears to denigrate the use of IV&VT as a risk assessment tool.
In reality, the question of ‘have all the test scripts been executed’ is only one of a far more detailed set of questions that need to be asked. These should include questions regarding the scope and rigour of the requirements, the techniques and standards adhered to during the build phase, compliance to regulatory and audit standards; traceability of requirements through the various phases of IV&VT.
Most importantly of all, an assessment needs to be made as to the overall risk to the business of migrating the item under test to the live operational environment. With this in mind, limiting the role of the testing group within an organisation to the writing and execution of test scripts starts to look a little short sighted!
If we accept that IV&VT is an important tool for risk assessment within an organisation, then the question of where the testing organisation sits within that organisation becomes one of high importance. In the context of risk assessment, the role of testing becomes one of continuously assessing and reporting on the risk to the organisation associated with its processes and procedures for producing and delivering new and enhanced products and services.
Consequently, to maintain its independence and ensure that the focus is really on risk assessment and not just test script pass /fail statistics, it must surely follow that the testing organisation should be an integral part of an organisation’s risk, or at a stretch, audit functions.
Such a move would necessarily entail a step change in the training and development of testing professionals so that their techniques and processes align with the vision of providing an independent assessment of risk, but this is considered a small investment in relation to the potential benefits.
It is worthwhile to consider the number of projects that fail to deliver their anticipated benefits, and compare that to the potential benefit of having a truly independent IV&VT organisation highlighting risks throughout the project lifecycle. Such an IV&VT organisation would focus an organisations attention on early remediation of risks, and hence, result in an increase in the number of projects that deliver their anticipated benefits.