DevOps done right: Securing grassroots support for a top-down DevOps enterprise push
As more and more enterprises rush to join the DevOps movement, practitioners share details of how they secured grassroots support for their senior leadership team's top-down agile adoption mandates
Securing senior management support for grassroots DevOps initiatives is an absolute necessity if practitioners stand any hope of turning a small-scale department win into an enterprise-wide success story.
And for those that have, their accounts of how embracing DevOps has boosted revenue growth, improved business productivity and made their organisations better equipped to respond to looming competitive threats are not going unnoticed by their peers.
This is why the pressure to adopt a DevOps strategy is just as likely to come from senior enterprise leaders these days, as it is from frustrated IT workers who feel hamstrung by long-winded, waterfall-like software development protocols and respond by seeking out alternative ways of working.
“We’re seeing this velocity, this transformation happen, where the business – whether insurance, banking, finance, manufacturing or wherever you are – realises that technology and software is driving the organisation, so what you have to do is drive down [a DevOps agenda],” he said.
But a top-down mandate to adopt DevOps is unlikely to gain traction unless the developers, operations staff, IT security personnel and quality assurance (QA) staff responsible for delivering on the organisation’s agile ambitions are on-board too. And that’s not always a given.
People are protective of day-to-day job responsibilities, for example, and are wary of how any shift in the status quo might affect them, or they might not appreciate why there is a need to change their organisation’s approach to software delivery and deployment.
If those resistant to change feel the business is simply “doing DevOps for DevOps’ sake”, their support is going to be hard to secure, which is why it is important for the senior leadership team to identify a genuine business issue that it could help solve.
Perhaps it is taking too long for the organisation to act on user feedback and incorporate it into its products and make improvements, resulting in lost sales and revenue, for example, because of bottlenecks forming at various stages of their existing software delivery process.
For the team at Dutch online retail giant Bol.com, the need for change was driven partly by a realisation that the IT department’s rate of work had started to stagnate, despite its headcount increasing.
If those resistant to change feel the business is simply “doing DevOps for DevOps’ sake”, their support is going to be hard to secure, which is why it is important for the senior leadership team to identify a genuine business issue that it could help solve
“The business was scaling, but we couldn’t grow very efficiently or effectively … and there were dependencies all over the place, so you were always waiting on somebody else. It was very frustrating for people,” recalled Frederieke Ubels, director of IT innovation process and agile coaching at bol.com, during a breakout session at DOES 2018.
“Time to production was our most persistent bottleneck. From the end of the sprint and releasing to production, there could be a long time between,” she said.
There may be trouble ahead
The emergence of these challenges came as something of a surprise to Ubels and her team, given the company had recently taken steps to eradicate a number of other barriers to innovation.
As such, in 2014, the company began to insource its web operations, after completing the build of a new datacentre, having realised handing off the management of its production environments was starting to affect its business agility and ability to recruit new staff.
“It restricted us in many ways,” she said. “For instance, we were only allowed to deploy once every four weeks, and we had trouble getting the best engineers because when you’re not allowed to touch production, where’s the fun?”
Along with the datacentre, the company had also moved to adopt a number of “state-of-the-art principles”, such as infrastructure as code, continuous integration and a number of “cool tools” that should – on paper, at least – pave the way for DevOps to flourish in the organisation.
“In our minds, we had Dev and Ops, and could sit back and watch DevOps automatically happen. Well, it didn’t,” she said.
Defining DevOps
There were a couple of reasons for that. First of all, Bol.com’s insourcing strategy unintentionally resulted in its developer and operations teams organising themselves into silos, which needed breaking down.
It had also become clear that DevOps meant different things to different people in the organisation, and people were unsure of what it was they were supposed to be working towards.
“Everyone had their own vision of what DevOps should be. Was it automation? Was it culture? Was it tooling? It could be everything,” she said.
To clear up the confusion, the company realised it needed to better define what it was trying to achieve in a succinct way, leading the firm to define its DevOps push in similar terms to what went into putting “man on the moon”.
“We found a great metaphor – to put a man on the moon – because it’s a clear goal, you can see it every night, but you don’t know how you’re going to get there or what the journey is going to look like,” she said.
“Our goal was not doing DevOps: our goal was to be scalable and productive, and have fun along the way. That was our moon.”
Following Spotify’s lead
To get people to get on board with the strategy, the Bol.com team encouraged everyone to read up on how other companies have approached the move to DevOps, with music streaming site Spotify being called out by Ubels as a source of major inspiration.
This led to the creation of a series of multidisciplined teams at Bol.com, each with responsibility for running a particular product or service.
The teams were also asked what they wanted to get out of the move to DevOps, with the vast majority asking for more autonomy in their daily work, paving the way for them to oversee the deployment of their own code into production, and assume responsibility for its aftercare.
“We got back on track, with the number of user stories more in line with the number of people in our team,” said Ubels.
“That’s not just great for our teams, but also for our business, because our teams are the driving force, and if it were not for this man on the moon, we wouldn’t have had the revenue growth figures we have had over the past three years.”
Inspiration to change
Taking inspiration from other companies is actively encouraged within the DevOps community, with its members urged to share what approaches have and have not worked for them at informal meetups and conferences, such as DOES.
US retail giant Target is often name-checked by practitioners as a major source of inspiration, and that is certainly true of the DevOps team at credit card giant Capital One.
Target offers its teams help in getting to grips with the principles of agile and DevOps in an immersive learning environment over the course of six or more weeks, while tackling the “real-world” problems that are preventing them from being as productive as the business requires.
The approach is referred to as a “dojo”, and it was after attending a Target-hosted one that the team at Capital One hit on the idea of replicating the initiative in-house, as a means of securing grassroots support for the company’s wider DevOps ambitions.
The company is several years into a wide-scale digital transformation project that has seen it move to adopt DevOps, embrace agile ways of working and start moving some of its workloads to the cloud.
Aimee Bechtle, senior manager for next-generation infrastructure business strategy at Capital One, joined the organisation in December 2015 to lend a hand with some of this work, and in early 2017 was asked to lead the company’s DevOps Acceleration Service, at the behest of senior management.
Continuous integration
Its aim is to provide the credit card technology organisation within Capital One with DevOps engineers to help them get to grips with the principles of continuous integration and continuous delivery in the cloud.
“I had a new team, a new service, and I had no customers yet, so I had to go out and start talking to my technology colleagues to get some business,” Bechtle told DOES attendees. But she soon discovered the people she was talking to were not all that sold on the idea of DevOps. “I was surprised, when I started talking to them, that I was hearing, ‘We don’t have time right now’.”
A lot of the people she approached also advised her to speak to the product owners who were responsible for overseeing the output of its tech teams, rather than them, prompting Bechtle to do just that by organising a pro-DevOps presentation.
“I prepared this great deck, filled with facts, espousing the benefits of DevOps and the service, and how my team was going to help bring in these software engineering teams to help them,” she said. “I was surprised after briefing these product managers that I received silence and glazed-over looks.”
But not from everyone, namely her colleague John Schmidt, who works as a software product innovator at Capital One, and had been running into productivity issues within his own team. He had found that it would often take weeks to get code into production.
Trialling agile and DevOps
Schmidt had previously taken part in a Target dojo, so the pair decided to trial one using his team as guinea pigs to see if schooling them in the ways of agile and DevOps over six weeks might speed up their rate of work.
They were asked to create continuous integration and continuous delivery pipelines for two different application programming interfaces (APIs) by following the principles of agile and DevOps over the course of the six weeks.
During that time, there were two points where the experiment could have potentially derailed, said Bechtle. “These are two dips that are going to be experienced in every transformation and you can decide to quit or decide to move forward,” she said.
The first occurred quite early on, with its participants starting to question why they were being put through this process, given that, while their old way of working may have delivered slower results, it still worked.
“This team had been working together for quite a while,” she said. “The anxiety was high. They were uncomfortable and they felt exposed.”
“They went over the hump, and they said, ‘We get it now, but we want to get back to coding and business as usual’. But we said, ‘No, you’re going to meet the challenge’. And they did. By the end of the six weeks, they had reduced three weeks of testing to three hours.
Learning from others – even competitors – is valuable in DevOps, and so is returning the favour by offering support, advice and suggestions that could help others overcome whatever barriers are standing in their way
“We run several dojos now, and I believe it is the fastest and most effective way to adapt a team to an engineering culture and build accelerated expertise in DevOps and agile,” said Bechtle.
Cooperation over competition
Learning from others – even competitors – is valuable in DevOps, and so is returning the favour by offering support, advice and suggestions that could help others overcome whatever barriers are standing in their way.
DevOps Handbook co-author Gene Kim reiterated this point during the opening remarks of the second day of DOES London 2018, where he remarked on how quickly DevOps had been adopted since he started studying the community five years ago, as a result of practitioners sharing their knowledge and experience.
“What we all have in common is we’re dissatisfied with the current ways of working, and we all know that how we work now is not going to help our organisations thrive and survive in the marketplace,” he said.
“It is my genuine belief that the people in this room are the people who are going to be pioneering the practices that in 10 years we’ll all be taking for granted.”
