From a network manager’s perspective, Software Defined Networking is a revolution in concept – but the colossal investment in existing infrastructure means that in practice the impact of SDN has been more of a slow evolutionary change. Among the less obvious changes has been the introduction of software development strategies into the networking environment – notably the more agile and interactive practices of Continuous Integration and DevOps where applications get built, tested and upgraded on the fly.
In rolling out hardware on the scale of network infrastructure, you could not afford to get it wrong. So the network development process required exhaustive testing of the structure in the laboratory before deployment, and there is little scope for making serious changes once it is in operational hands. In the flexible world of software, however, the relationship between development and operations is far more agile, allowing any number of minor or major revisions to be made in response to the software’s performance in the field – hence the endless series of versions with numbers like V10.8.13.
So when SDN introduces the idea of a virtual network, it brings these two worlds into collision. The network structure can now evolve at software speeds, allowing development to be informed by its behaviour under operations conditions, and this allows a rethink of the relationships between Development, Network Testing, Operations, QA and related departments. DevOps has already become a trendy term in the applications sphere, and it is increasingly relevant to networking.
The Essence Of DevOps
Natural selection begins with a simple hardware model: Development takes place in the egg or maternal womb and gives birth to a unique genetic entity to be tested in the Operations department we call “Life”. It either fails the test, dying without issue, or it survives and breeds. It can also pass with flying colours, leaving masses of progeny.
In the simplest organisms – and this includes organisations responsible for pre-SDN networks – Development and Operations exist in separate silos. In nature this means that Development lays the eggs and walks away. In business it means that Development’s work ends when the product goes to Operations.
For higher organisms, however, Nature created the concept of Parenting: you don’t just leave your children’s future to fate, instead you nurture their development through life itself. It is an agile adaptive and increasingly intelligent process: even if the genes have produced a small or weak child, the parents can still play an important role teaching how to defeat bigger rivals.
In the business environment, this integration of Development and Operations is called DevOps. Like Parenting, it is a concept that makes a lot of sense, it is one that everyone would like to master – but can be a real challenge to implement.
For a start, you can get resistance from those who think that Development is trying to take over Operations as a mere extension of the development process. It is true that the parent has a “higher” role than the teacher or tester, in that a parent can become the teacher rather than the other way round, but no-one welcomes the “bossy parent”. So a wise parent learns to listen to the teachers and trainers – just as a good DevOps project is all about two-way sharing between developers, operations, QA and so on down the line, to build a culture of collaboration.
This process has been analysed into four key elements: Sharing, Culture, Automation and Measurement – unfortunately spelling SCAM. So these are normally presented in CAMS sequence as follows:
- Culture: as suggested, this is not about any one department bossing another but about encouraging a culture of sharing or parenting. It is actually something that emerges in the course of attempting DevOps.
- Automation: parenting is a hellishly complex on-going operation that would be impossible without the help of automation. Nature has had several million years to develop automatic parenting instincts (and it still drives youngsters wild when parents just sense that something is bad but cannot argue why). Industry, however, has had barely 3 years to develop automated systems to orchestrate DevOps.
- Measurement: testing has become a continuous process, like life itself, and must be integrated into the automation systems.
- Sharing: should really come first, as it is the feedback process out of which emerges the whole parenting culture. This happens both within a DevOps team and between them, as the whole DevOps concept has also evolved through the shared experience of many teams worldwide.
Migrating To DevOps
Telling different departments, which have each evolved their own cultures and ways of working, that they must now work together in harmony is a bit like ordering parents and teachers to do better for their children. Everyone can see it is a great idea, but how do we kick-start the process? What is needed is the tools to handle the heavy work – because the complexity of agile, iterative development makes for a lot of extra routine labour.
Consider the example of “Continuous Integration” (CI), the term used by software engineers for development built on plan/code/build/test/repeat iterations. This requires many inter-operating and well integrated elements: not only a highly efficient, development environment, but also one that can identify all new software updates, control versions in the repository, and schedule their execution. With a large team of engineers CI can lead to hundreds of software builds a day being put through their paces and needing to be revised and integrated. It also demands a massive amount of hardware on standby to handle all the operations on demand.
My company has a tool that turns CI into a workable solution it is called “CLEAR DevOps”. It succeeds by automating the complexity of CI and running it on virtual machines. Instead of racks of physical servers on standby, consuming electricity and generating heat while waiting to play their CI build and test role – the “CLEAR DevOps” environment simply spins up virtual machines automatically as required to execute automated tests.
In addition, the solution’s sophisticated test automation systems do away with the need for specialist coding skills. Instead a simple user interface allows staff to create, snapshot, start, stop or manage virtual machines, while test assets are easily integrated into the workflows to integrate behavioral automation, data parsing and analysis into the DevOps process.
It is a major practical and cultural shift, but we’re already showing results in the real world. One electronic equipment manufacturer, with R&D teams across the globe, had been investing $12.5 million a year on development and testing and $4 million on automated tests and yet in a single software release they still had to correct over four thousand bugs, finding it exceptionally difficult to identify where the problems were located. The benefits of migrating to our solution were immediate: within a couple of months they reported a 70% reduction in bugs, 66% reduction in release time and massive savings in R&D, QA and CAPEX costs.
The Impact On Networking
DevOps is hot news in applications development and it is seeping into other regions of our increasingly software defined universe. As the parenting example suggests, it calls on a lot of experience and is not something you can simply pick up from a book.
Bear in mind the three great drivers – Quality, Cost and Speed – before deciding how best to proceed. DevOps can improve all three, as our example shows – the customer not only reduced the number of bugs, but also saved money and reduced time to release,. There is no shortage of consultancy companies who can help you save money and get quicker results by migrating to the DevOps approach, they can also help to establish the culture that holds the whole thing together.
But if improving quality is a key driver, then testing must be the central component of the DevOps process, and that requires integrated automatic testing to lift the burden or, as one team put it, “to take the strain out of Development”. According to IDC’s December 2014 survey of Fortune DevOps Best Practice Metrics, approximately 21 percent of the IT organizations surveyed said they were looking for Testing/QA tools to accelerate DevOps, but those that tried to custom-adjust current tools to meet DevOps practices have a failure rate of 80 percent. So choosing the right test systems is a critical step.
Testing will then be integrated into the combined development and operations, and no longer be a separate, specialist department. So this requires test systems that are simple to set up and operate without coding skills. The results of the tests will become more widely relevant, so it is also important that the system reports results in a consistent and clear format that can be readily assimilated into other parts of the DevOps system.
So, if quality is your objective, you must choose the right test tools, and get advice on how to incorporate them into the heart of the DevOps system, to provide the core around which your dynamic DevOps culture can be formed.