When it comes to modernizing how software engineers write their code, sometimes you just have to light things on fire.
Just ask James Gagnon.
He’ll tell you that good engineering teams at Microsoft and in Microsoft Digital are driving towards using DevOps to do their work, and with good reason.
“DevOps is a foundational part of actually achieving digital transformation, of becoming truly agile,” says Gagnon, a software engineering lead on the Microsoft Digital team delivering finance applications inside Microsoft.
Moving to DevOps can be as simple as combining software engineering and support roles, then delivering smaller software increments. That seems easy enough, but here at Microsoft, Gagnon and others driving this kind of transformation often find that organizational boundaries, team culture, and business processes must also evolve to get people to buy into this modern approach to engineering.
Gagnon has seen this story play out over the last six years.
“There is no doubt that implementing DevOps will increase the speed and quality of software delivery, but it will cost you everything if done in isolation,” Gagnon says.
Leaders must burn old practices
“DevOps thrives with increased autonomy,” Gagnon says. “Empowering teams to measure, deliver, fail, learn, and improve internally will yield greater outcomes. Leaders that measure outcomes over execution governance will give their teams the freedom to innovate while taking greater accountability of their work.”
Gagnon says that team productivity measures and processes such as sprint burndowns, velocity, work in progress limits, and story pointing must be private to the sprint team. He says leaders should enable these basic practices and ensure they are in place, but they should also make sure the data stays with the team.
“This creates a safe environment to reflect and enables continuous improvement,” he says. “Leaders that create aggregate reports and force specific practices will create a culture of gamification. Reports and scorecards will become more important to the engineers than the outcomes and the customers.”
Leaders who control and over-centralize solutions will frustrate engineering teams and impact their ability to deliver working software with agility, Gagnon says.
“DevOps transformation starts with software transformation and culture transformation,” he says. “The sprint team knows best the debt that is impacting the team and is best suited to partner with business and leadership to define the roadmap.”
Delivery of schedule driven projects is a legacy success metric of the waterfall software delivery lifecycle. Gagnon has been part of this and learned the hard way that it doesn’t work.
“I’ve seen the perfect software engineering storm and it was a long nasty ride,” he says. “Leadership had set a date for achieving transformation. The engineering team changed structurally with new undocumented expectations. Engineers had to modernize monolithic legacy software, originally built on a tight budget, all while success was measured by new business feature delivery and quality. Unfortunately, leadership was the first and only thing to change.”
How it works—the UAT example
User Acceptance Testing (UAT) is one of the areas causing excessive toil when moving to an agile or modern engineering culture, Gagnon says. Many times, these legacy software engineering practices have been part of the release process for years and compliance validation or other factors required the business stakeholder to review and approve changes. When left in place, this conflicts with the goals of DevOps in general.
“The UAT process alone has huge implications in our ability to move towards DevOps, it’s simply not compatible. Engineering of the current feature sprint can’t stop while UAT happens for the previous feature sprint,” he says. “This is supposed to be faster, right?”
The changes required to move away from UAT need to happen both in engineering teams and the business teams.
“Engineers must take full accountability for quality and business stakeholders must be integral to the process,” Gagnon says. “Achieving the agreed upon success criteria defined at the beginning of a sprint must be demonstrated by a simple sprint review as part of closing the sprint.”
Not only does a UAT process slow down agile methodology, it creates code line management complexity, he says. This requires a separate integration step, an additional branch, and slows down development with increased cost.
“Modern engineering can’t be achieved when managing branches that map to legacy process,” Gagnon says. “We must deliver to production what was engineered within the same sprint.”
People make it happen
Just as culture needs to transform, the software engineers who make up that culture do as well. At Microsoft it started by combining the development and testing role, and now the company is merging the service engineering role into this DevOps role as well.
“The goal is to drive end to end accountability into the software engineering discipline,” Gagnon says. “This is crucial to the service view and achieving modern engineering practices.”
It’s a transition that’s harder than it looks.
“This isn’t simply a matter of learning a few new technical skills, but, also soft skills,” he says. “Writing code is just one expectation.”
Engineers need to work closely with business partners, manage more aspects of a project, and collaborate with other team members.
“I’d love to hire the mythical unicorn, someone who understands quality, who understands security, who can code, and who can communicate with others,” Gagnon says. “These candidates are far and few between. I hire for the basics across all required areas and seek diversity across the team for specialization. This approach has helped balance the team where everyone can contribute within the DevOps model.”
Putting it all together
Gagnon says proper implementation of DevOps has required multiple adaptations to culture and processes at all levels in Microsoft.
“It’s helping to drive our transformation,” he says. “It’s not only moving the organization forward, but the people as well. It’s driving a shift to centralize on outcomes over solutions, which enables DevOps engineers to truly own and take pride in their work.”
It’s the kind of thing that fires you up, and sends old, outdated practices up in smoke.
“I’ve never been more excited about what’s possible at Microsoft than right now,” Gagnon says “The transformation is occurring throughout leadership, which is making it easier for our engineering teams to realize their goals and have fun while doing it.”
