DevOps culture - what are its fundaments

DevOps is one of the most popular buzzwords in the tech industry and for a good reason. Some consider it to be the root of all evil, while others praise it as a silver bullet. I like to call it “technical agile”. It’s simply a culture where we all put emphasis on getting features to the users as smoothly as possible, instead of playing constant blame game and finding a problem for every solution.

Meet waterfall

In the past, organizations were arranged in a certain way. Organizational silo, though fading into oblivion, is still present in some workplaces. In case you’re unaware of the concept: ‘’silo’’ is the key word here. Just as farmers use silos to separate different types of grains, employers may sometimes divide their employees into specialized teams, responsible for consecutive stages of software development. Such a practice builds walls between departments and a habit of throwing stuff over these walls. Not literal ones, but being separated from one another, people are basically encouraged to have the ‘’we’’ and ‘’they’’ approach. In this scenario, they don’t work as a team. What does devops have to do with all that? Can it change the way we think about software development?

DevOps 101

As the name suggests, DevOps is a combination of ‘’Development’’ and ‘’Operations’’, and describing it as “technical agile” seems surprisingly accurate. When we take a step back and try to determine what the purpose of all the things we are trying to do is, it boils down to giving users new stuff they want, and, at the same time, not breaking what they already have.

As engineers, we (sometimes) have a tendency of over-engineering things and that’s what I believe we did with waterfall. While the idea of Big-Design-Up-Front might have been just fine 50 years ago when writing a program took a couple of hours and compiling it took a couple of weeks, it simply doesn’t apply anymore.

In every difficult project, you eventually get to a point where you simply need to get together and discuss what needs to be done if you want to finish work and finally go home. After going through numerous hurdles, a bunch of guys probably did just that. Instead of declaring the code done but not available to users and blaming the code being not deployable on the developers, a few really smart gals and guys decided that’s not how they want to work. They coined up the term “DevOps” to push each other to work more closely together. It is a way of working devised from lean manufacturing initially implemented by Toyota. It is based on three principles (also called “three ways”).

The First Way – System Thinking

Image: itrevolution.com by Gene Kim

The Second Way – Amplify Feedback Loops

It’s about creating feedback loops and acting on them. We want to know about any problems the moment they arise to mitigate them instead of pretending everything’s fine or it’s someone else’s fault. It gives us the necessary information to know what should be fixed and how we might improve the way we work.

The Third Way – Culture of Continual Experimentation and Learning

You might ask, “Dude – but we’re building apps, not cars. How does it apply to us? Our work is so difficult/complicated/bizarre, etc”. Building car is complicated and, as much as I regret it, we still don’t have robots doing cool stuff in our offices as Toyota does in their factories. Let’s take a look at some problems that used to (or maybe still do in some places) occur in a ‘’silo mindset’’ company.

Constant blame game

Whose fault it is that the app isn’t working? Instead of simply cooperating to make it work again, the development team and operations were passing the buck back and forth fighting over who screwed up. The developers were thought to write crappy, unmaintainable code and the operations were known to take ages to do even the simplest task.

Staying in line

The development team were only responsible for a part of the development process (so-called “code complete”) and weren’t required to have the slightest idea of what happens later – that caused builds and deployments to fail. What’s more, as developers were not monitoring their apps in production, such software was usually flawed in terms of logging, monitoring, and overall telemetry; devs simply didn’t know what kind of tooling the ops team needed.

Fair share?

The operations team didn’t share knowledge of what was happening in production, restricting the developers’ role. Developers develop, operations operate. This may seem as if everyone’s got their fair share of work and with all the tasks divided, everything should go right. The problem is that working with different environments and being unaware of the next, post-development stages, fixing stuff could get unbearably difficult.

How DevOps changed everything

Ultimately, both devs and ops have the same goal: to make the organization as productive as possible. Though they may have different visions of how to achieve this state, they still play for the same team.

In traditional ops organizations, functional orientation was prevalent and teams were organized by their specialties. This does make sense to some extent, but it also slows down the process. Developers are focused on delivering new features as fast as possible. There are deadlines and the dates are often non-negotiable. In such situations, sometimes they have to cut the corners. Since they focus on delivering features, non-functional requirements such as security, reliability or scalability are the first in line to be cut out. And what happens when these things get overlooked? Technical debt starts to increase.

The operations, however, do care about those elements that were seen as irrelevant compared to features. Ops is mainly interested in delivering a reliable network experience and can’t really keep up with the demand from changed applications. This may result in large queues and long lead times. The developers’ urge to get the product out as soon as possible clashes with the reason of ops: they fight for stability and uptime. And that’s where DevOps steps in, bringing both teams together to work towards a common goal. It requires big-picture thinking from both sides, as they have to put aside their conflicting ideas and understand each other’s objectives in order to do what’s best for the organization as a whole.

Adopting DevOps in their organizations, however, many teams struggle to define what devops means to them and what benefits they may derive from this approach.

Gene Kim, commonly called a devops visionary, researcher and author of books such as “The DevOps Handbook” and “The Phoenix Project: A Novel about IT, DevOps, and helping your business win”, writes: Broadly speaking, to achieve DevOps outcomes, we need to reduce the effects of functional orientation (optimizing for cost) and enable market orientation (optimizing for speed).

This means having many small teams working safely and independently, quickly delivering value to the customer.

Taken to the extreme, market-oriented teams are responsible not only for feature development but also for testing, securing, deploying, and supporting their service in production, from idea conception to retirement.
These teams are designed to be cross-functional and independent — able to design and run user experiments, build and deliver new features, deploy and run their service in production, and fix any defects without manual dependencies on other teams, thus enabling them to move faster.

Devops requires from people to take responsibility for achieving common goals, which in turn leads to them ending the blame game as they can’t simply point a finger at someone whose fault it may be. When teams feel responsible for delivering results, there’s a significant increase in their productivity. It’s important to remember that devops in not only about merging two departments within the company, it’s a way to run your business. If you require the qualities provided by devops, the culture of it has a chance of surviving. It’s great if you encourage your team to work towards a shared objective but you must also keep in mind that it’s crucial to arm the team with the right tools.

Is it all that pretty?

Even though devops, if implemented correctly, may bring a company countless benefits, its reputation is still not that great, to say the least. What are the problems you might face when learning devops?

First of all, many will say that the sole concept of ‘’learning devops’’ is primarily flawed. Devops is a combination of a variety of concepts, practices, and techniques which may not be all that new, but combined provide a new approach towards organizational culture. In this interpretation, you can learn the tools or techniques, but you can’t just learn devops itself. It’s not an end state, it’s a process of continuous learning and improvement.

A problem that you may encounter when trying to introduce devops to your company is that your staff may not yet be totally on board with agile methodologies. As mentioned above, in the waterfall model, the stages are separated and roles are distinct. Devops maintains the core of Agile as it puts people above tools and processes. After all, the tools and processes are there to support people, and not the other way round. So first off, you need to make sure that Agile is what you do and that it is well understood by all team members. But even then problems may arise. In a breakdown situation, the team may step back into their ‘’not my fault’’ comfort zone.

By now, you’ve probably realized that devops is much more than what happens to the app. If your dream is to hire a ‘’devops guy’’ in hopes that he will solve the problem and bridge the gap between devs and ops, you’d better re-dream it. The transition to devops is much more a cultural move requiring organizational adjustments. This means that you actually need to have the right motivation if you want this undertaking to be a success. If you’re going for devops because it’s what the cool kids do, it’s likely you’ll fail. But if you understand that it’s done in order to reshape or improve business processes, you’re off to a good start. As always, you need to know your “Why”, and in this case, you must also be prepared to use your ‘’why’’ to convince people that the transition to devops pays off. Don’t jump into devops too hastily – start small and grow from each success along the way.

Why DevOps? Devs can nail operations and Ops can develop

When teams consist of both developers and admins and are collectively responsible for their apps from the very first line of code, through builds, and deployments up to running it smoothly in production, the process of diving into devops goes more smoothly and brings results, to name a few:

Implemented continuous delivery with loads of automation

This helps to get feedback fast in case of any problems. It’s also a form of executable, shared documentation on how apps should be built and deployed. The documentation should be accessible, used and actively maintained by every team member.

Rollbacks are automated

And that makes experimenting and learning painless as it’s much easier to try out new things if you know that in case of problems you just press a button and you go back to the previous version.

Infrastructure is provisioned via self-service tools

This prevents developers from waiting for the ops people, while ops have time to share the best practices considering handling specific (most difficult) scenarios.

This way, developers “learn how to do ops” simply from everyday practice and collaborating with operations. When encouraged to think beyond their area of expertise, developers will acquire new skills and be able to see the value that DevOps brings to the organization as a whole. Yes, documentation, configuration scripts, and automation are important, but we keep coming back to the culture. Encourage development, shared learning through transparency, and true collaboration, and if you do all that, implementing DevOps will not be a dreadful challenge for your organization.

DevOps: A Culture of Getting Things Done