Is your team ready for devops?
Devops leads to a shared destination, requires investment, and ultimately, helps you reach your goals.
We want every software developer to think like operations, every software team to embrace automated testing and deployment, and every business to leverage continuous delivery. Our utopia is a world where every company made a journey like the fictional Parts Unlimited from The Phoenix Project. If you share this vision, let us explain three things you need to know about the journey: it leads to a shared destination, it requires investment, and, most of all, it is a way to achieve business value.
Planning a devops journey
People who talk about agile tend to lump all the benefits together. The reality is: there are some distinct stages, each with different benefits and different kinds of investment.
We keep hearing the success stories from Flickr, Netflix, and Etsy. We don't hear about how the transformation happened. We by no means are faulting those organizations. In many cases, they have described where they started, how long it took them, and how things changed. What we mean is we don't understand what they're saying because it's so unfamiliar. Fortunately, we can use the agile fluency model to help turn attention back to "how."
Mapping agile fluency to devops, we propose four stages:
- Orient to value for your organization
- Automate, as a team
- Measure business value
- Optimize for the whole
1: Orient towards value for your organization
Just do a Google search on "why devops" and you'll find all kinds of justifications. However, devops is not a goal in and of itself. It needs to fit the unique goals and direction of your organization. Even if your management desires the common justifications, these justifications become unique when applied to your products and services. Shipping your product faster is not the same as Etsy deploying 50 times a day. Shipping your service with better quality is not the same as Netflix maintaining sub-second MTTR.
To the extent that devops is an extension of agile, this first stage is the same for both.
It's about knowing what your company values, and planning in those terms.
We've seen teams where everyone is focused on only the part they play. It's frightening to accept responsibility for business results, when you're accustomed to just accepting assigned tasks. It takes a lot of trust in management and other team members. That's why this stage requires a team be ready to change the culture of itself. Scrum and kanban both provide good guidance and structure for making this kind of change.
Working on the right thing is a benefit to everyone. Working on assigned tasks makes me feel like I'm just a cog in a machine. When I have clear line-of-sight to what the business values, I know my purpose and feel like I'm truly part of a team. Benefits don't end at making me feel better. Managers call it "greater visibility." What it means is they have enough information to adjust their management style to fit the context. They can redirect people and teams when business priorities change.
The agile fluency model confirms this experience. It should take 2-6 months for a team to become fluent in aligning to and tracking in terms of business value. If you go longer, maybe you need to try something else. Maybe scrum isn't a fit for your team, try kanban. Maybe going it alone isn't enough for your team, try an agile coach. Or maybe the coach you have isn't focusing on the right things, try another.
There are many practices and tools that can help. For practices, We're quite fond of impact mapping, user stories, and specification by example. Each of those contribute turning big business goals into smaller pieces of work that a team can deliver in small chunks. In the Atlassian tool set, we reach for Confluence and JIRA Software to help. Confluence makes setting those business goals a collaborative process then keeps those goals ever-present as the team's target. Once oriented around business goals, JIRA Software helps track progress.
You don't have to build full fluency at this level before starting a competency in a higher one. That said, be realistic. Fluency is where we fall back when we get frustrated. If a team gets frustrated because it has taken on too much learning at once, it may regress to habits that obscure value and hide progress. Make sure the team has enough competence in "focus on value" that it can get back on track quickly.
2: Automate, as a team
The mismatch between individual knowledge and learning-as-a-team is a characteristic of the 2nd stage in agile fluency. It isn't about a team culture change like the first stage; it is about a team learning technical practices together. This is a different kind of investment. You may need to attend workshops together. You may need a different agile coach with technical chops. You may need to establish a "technical owner", as a parallel to product owner, to arbitrate decisions and prioritize automation options. Even in a team full of senior agile developers you might struggle with lowered productivity for a few months before reaching fluency as a team. This stage also yields a different kind of benefit. It is about focusing on delivery of value. Typically that is measured by shipping on market cadence and capturing value frequently. Frequency leads to revealing obstructions early. Management notices teams at this stage because they have low defects and high productivity.
Extreme programming has long provided a bundle of agile technical practices that apply to this stage: coding standards, unit tests built with test-driven development, automated acceptance tests, pair programming, serial and continuous integration, collective code ownership, and refactoring. It's difficult to keep practices and tools separate in this stage. Except for pair programming and collective code ownership, automated tools closely match the Extreme Programming practices. I use IDEs and static analysis tools to enforce coding standards and perform refactoring, and language-specific frameworks to make it easier to create unit tests and acceptance tests. Of course, both unit and acceptance tests are automation themselves. I use Bamboo for continuous integration to run all those automated tasks, as often as every commit. Good tools elevate the practice, making things more visible and aligned with business value. This is where software development turns in upon itself. Where economical, team practices become encoded as automation. This kind of automation is largely what developers bring to devops.
For some teams, those extreme programming practices might be enough to deliver value. Many open-source and shrink-wrapped software products are done when they are packaged and tested. But many more software products need to be put into production before the business realizes value. This has long been the case for software built in large organizations to support business operations. That's why Scott Ambler has advocated for thinking about operations in agile. His "Enterprise Unified Process" advocated for production and retirement phases and for a discipline of operations and support. Yet it seems only recently that devops has drawn mainstream attention to the role operations plays in delivering value.
Jez Humble and David Farley advocated for continuous delivery as a holistic view of delivering value. Their book extends automated testing and continuous integration into an automated deployment pipeline. In the Atlassian tool set, Bamboo supports this idea by having specialized deployment projects. These model a deployment pipeline with environments, like development, QA, staging, and production. Bamboo can track the deployment of an application into each of these environments. It is then aware that a release is made to production. Although the building blocks of automation are often the same as continuous integration, tracking the pipeline requires a richer information model. With each environment comes another audience that needs to understand how the automation works and when it is performed. In other words, the tool needs to be approachable for not only developers, but also testers and operations.
3: Measure business value
During the agile transition process, it's often difficult to understand what the business wants, let alone measure it. When the backlog gets filled out, the pain would turn toward leveraging automation to enable faster course corrections. Even at that point, measurement of business value always feels out of reach. This inward focus endures as long as the team is the constraint in overall delivery of value. Often times, when teams work their way out of being the critical constraint, strange things would follow. They fall into waiting for the business to make decisions. Or for operations to provide data about what they'd deployed. In 2011, Forrester coined this bookend experience, "water-scrum-fall." While software teams go agile, business and operations often continue to operate with big-batch planning and execution.
Bringing biz, dev, and ops together is not just about a team learning technical practices, it's a change in organizational structure. The change in organizational structure brings business thinking and operational insights into a development team. This stage has many of the benefits attributed to devops: better product decisions, higher value deliveries, and more reliability. Previous stages give many opportunities to practice measurement, such as measuring team velocity or the delivery pipeline. Measuring business value should feel a lot like automated acceptance testing except applied in production. Orienting to business value means you know what the business wants. It takes even tighter collaboration with the business to know how to measure the value of those things. Automating as a team means you know how to build "sensors" and aggregate data. It takes collaboration with operations to get to the right data. Fortunately, monitoring is a competency of most operations teams.
At Atlassian, our product teams measure Net Promoter Score and monthly active users. It has been a long journey to survey and monitor not only our cloud offerings but also the behind-the-firewall products. The transparency benefits should be readily apparent. The measurements are concrete business metrics that can be used to make business decisions. The objectivity and comparability of these metrics has replaced the politics of decision making with mutual trust. Product teams can negotiate rapidly to shift plans and align with what is best for the company as a whole. Data drives decisions at the speed they are needed by product teams.
Of course, this kind of change typically takes a long time. The agile fluency model estimates 1-5 years. The cost is often measured in social capital expended on incorporating business and operations expertise into the team. The team will need to avoid the negative connotations of "turf wars" or "fief building." If you want the rewards of this level, start cross-team collaboration early, even as you're working on earlier stages.
With organizational change, there may still be some culture change. The operations focus on "keeping the lights on" often means their work is more on demand than software developers. As a result, operations have a natural focus on flow. Some software teams may need to shift to even shorter batches to better fit with this reality of operations. This is why the devops community often favors kanban. That said, there are many practices and tools appropriate at this stage. Lean Startup provides many good ideas for running experiments and measuring results. Lean Software Development also provides good guidance to help teams take a disciplined approach to optimizing value delivered through measurement. At this stage, devops' CALMS acronym is fully realized, adding lean and measurement to culture, automation, and sharing.
The tools for measurement are evolving rapidly. The scale of some organizations demands big data. However, many organizations don't need the most expensive and cutting-edge big data tools. Instead, they need to focus on slightly different data gathering mechanisms. The key idea is logging. Everyone already has logs from running applications in production. In fact, they often have so many that steps have to be taken to regularly dump the overload of information. What makes logs so important is the time-centric structure. This time-centric nature is also reflected in business data warehouses, with time as a primary dimension. More and more, measurement for business, development, and operations is aligned on this time dimension. "If it changes, graph it." The trick is finding the tools that can aggregate over the right time scale for your organization.
4: Optimize as a whole
If you're standing at or before the first stage, it can be hard to imagine "optimize as a whole." The Phoenix Project concludes with Parts Unlimited at the apex of this journey. Getting to this stage requires the biggest kind of change: culture change for the whole organization. There are few non-fictional guiding examples in the industry. The organizations that achieve this stage invest in inventing new practices, instead of following existing ones. The idea of optimizing as a whole means cross-team decisions don't require top-down authority. Teams that can measure business value well provide objective insight to the whole organization. That means teams can use data from other teams to optimize their role in the system. Agile fluency recommends:
"For most organizations, four-star fluency is probably best left as an aspiration for the future, at least until three-star fluency is within reach."
As individuals, we hope people set this as a target and reach for excellence. However, organizations need to balance risk and reward. The rewards of this stage are hard enough to describe and, with so few examples to draw from, the risks are even more difficult to enumerate. Although we count Atlassian as one of the companies with competency at this stage, we'll leave it to those who have mastered optimizing for the whole to offer advice.
Are you ready?
Is what you hope to gain recognized as valuable by the business? If not, seek first to understand. Then try again in the language of your organization.
Is your organization willing to make the investment? If not, set a more modest target of a lower stage. Then try again with a more focused option. Success will win the ability to try for the higher stage later.
Once you get started, remember the power of conviction. Forget that you can fail. Build the bridge as you walk on it.
on a related note...
Devops and distributed teams
Four Atlassians in various roles talked about the particular challenges facing distributed teams who decide to embrace devops principles. Check it out.
Continuous delivery from code to customer
Bamboo is a continuous delivery tool that ties automated builds, tests and releases together in a single workflow. Take the tour to learn more.