A detailed look at what DevOps really is with the goal of clarifying all questions and doubts around it ✅
This is the written version of my youtube video ✍️ 🙂
DevOps has been gaining a lot of popularity and is taking over the traditional way of software development.
Difficulty to define DevOps
Many companies are adopting DevOps, however, the "DevOps" term itself is so broad. It includes so many things that it became difficult to define it exactly and clearly set the boundaries of DevOps compared to other IT fields. So this post tries to answer the question of "what DevOps is" in detail.
The simplest definition is that "DevOps is an intersection of Development and Operations."
But where do the boundaries of DevOps start and end? Which part of development is not DevOps? Or what part of operations is not DevOps? And why was there even a need for something between development and operations? 🙉
Development and Operations are two main components in the whole application release process:
So let's look in detail at this release process starting from the very beginning!
Application Release Process
Whenever we're developing an application, we always have the same process of delivering that application to the end users. So this is the main goal, no matter if you use waterfall or agile or whatever approach. At its core: 1) you create an application, 2) and you want to deliver it to your end users so that they can use it. 👩💻
So let's say you have a great idea about a cool application. You define its functionality or, in other words, what features it will have:
you code it
you test it
and now that you have a tested application, you want to actually deploy it on a public server and let users access it
For that, you build and package your application in some kind of executable form so that it can run. You configure the public server with all the needed stuff, like installing any tools the application needs and deploying your application there, you configure firewall rules to allow access to the application on the server, and you have launched, and users can start using it! 🚀
So that's the simplified basis of any application release, but that's not the end of the journey. While in use, you, of course, have to check in on your application:
Is everything running fine?
Are users experiencing any issues?
Maybe there are bugs in the application that you didn't catch when testing
Can the application handle high user loads?
etc
So after launching it, you have to actually make sure that your application is accessible and usable by end users, and if there are any issues for users, of course, you should fix them.
Process continues..
Now that was the initial launch of your application, but the application development is not done yet. If you see users like your application, you would want to make it even cooler, add new features, maybe optimize the performance by getting better servers or making your application faster, and so on. So you still have a lot of things to do. Every time you improve your application, either the code itself or the server configuration, you want to make this improvement accessible to the end users immediately.
So after the initial launch, you do multiple updates to your application, and to keep track of these updates, you version those changes.
And you do that over and over again:
you have an idea of improvement
you implement it in the code
you test it
build and package it
you deploy it
and once released, you observe it in the production to see whether there are any new improvement possibilities or any issues that need to be fixed right away
So this gives you a process of continuous delivery of changes and an endless cycle of improvements to your application.
And DevOps is about making this process of continuous delivery fast and with minimal errors and bugs:
So with DevOps, improvements get created and delivered to users fast, but also those improvements are of high quality and well tested. And that is a big challenge: quickly delivering high-quality code. 😳
Challenges DevOps tries to solve
Now let's see what are exactly the challenges that teams may face during this process and which DevOps tries to solve.
Because during this whole release process, we have roadblocks and frictions:
Now, what are the frictions and roadblocks in the release process?
1) Miscommunication and Lack of Collaboration ⛔️
The first and most important challenge is miscommunication and lack of collaboration between developers and operations.
So releasing the application has two main parts:
you code the application
you deploy and run the application
Developers are responsible for coding. Operations are responsible for running the application.
And between these two, there might be a gap between: "I wrote an application, but I can't run it" or "I'm running the application, but i don't know how it works" 🙇🏻♂️:
So developers would code without considering where or how the code will be deployed, while operations would try to deploy without really understanding what and why they are deploying or how the application even works. And this would result in miscommunications between these two.
For example: Developers finish coding, but the deployment guide for the operations team is not good enough or well-documented enough, so the operations team struggles to deploy it, so the release takes longer.
This kind of miscommunication could cause stretching the release periods for days and weeks and, in complex, badly maintained projects, maybe even months.
So between the developer is done with the feature and the operation starts deploying it, there is no clearly defined automated process of handover. It's based on a complex bureaucratic process of what checklists 📝 need to be completed and what needs to be documented and who needs to manually approve what for the release, and so on. So no streamlines or automated processes here.
2) Conflict of Interest ⛔️
Apart from miscommunications between development and operations, in a traditional setup, where one team is only responsible for the development and another team is only for operations, these two have seemingly different incentives that make it hard for them to work together.
Developers want to push out new features fast, while operations want to make sure those changes don't break anything because operations are incentivized to maintain stability in production. Their main focus is to make sure the application is available, doesn't crash, doesn't show 500 errors to the users, and so on.
This means that operations need to resist the speed of release and check all the aspects of a new release to make sure it's 100% safe, which again slows down the process, especially considering that operations don't really understand the code or the application. So it's even more effort for them to evaluate this new release.
So even though the main common goal of everyone in a company should be to deliver high-quality applications to the end users fast, in practice, the more immediate goals are for each role to do its job. And developer's job is to quickly create new features and push them out; the operations job is to maintain the system stability and resist new changes being pushed out:
This gives us a conflict of interest, and this kind of setup naturally makes it difficult for those two to collaborate. 🤷♀️
3) Security ⛔️
Another showstopper when releasing a feature is Security. Just like the operations team carefully evaluates changes to make sure they won't affect system stability, the security team will evaluate any changes to make sure they don't affect systems security:
And in a traditional setup, this is the same manual bureaucratic process as operations, which takes days or weeks and slows down the release process.
And as I mentioned, DevOps is about removing any roadblocks that slow down the process, so it includes this one as well.
However, even though this is part of the DevOps solution, there was a separate term created for that called "DevSecOps" in order to highlight and just remind the teams of the importance of security because it somehow got left out.
I actually have a separate dedicated video on DevSecOps, which you can also check out if you're interested: DevSecOps explained
4) Application Testing ⛔️
Now adding to the list of show stoppers is application testing. In many projects, there are separate teams or roles for testers, who test the application changes on different levels:
like testing just the feature
testing the whole application,
testing on multiple environments etc.
And often, these tests are done manually when teams cannot totally rely only on their automated tests, and only after manual testing is done can the change be released:
Even though this may not be done by a development or operations role but rather a separate tester role, this is an important part of the release process and may also slow it down considerably!
5) Manual Work ⛔️
As I mentioned, many of the tasks during the release process, like testing, security checks, deployment etc, used to be done manually. 🙇🏻♂️
For example, operations would do most of the operations tasks manually, either by directly executing commands on the servers to install tools, configure stuff, do patches, or have scripts or small programs they execute. But in both cases, this is manual work:
Drawbacks of manual work This manual work is slow and more error-prone because of human error. With manual work, you have a disadvantage in that knowledge sharing is very difficult because people who do the tasks would have to document it, and others would have to read it.
It's also very intransparent because it's hard to trace who executed what, when and finally when infrastructure configuration and so on was done manually; if something happens to the infrastructure, it may be really hard to recover and replicate the exact state fast. You would have to remember exactly what was done on the servers in which order to get to that previous infrastructure state. 🤦🏽♀️
DevOps tries to remove these roadblocks ✅
So you see, the main characteristic of all these issues is that they all slow down the release cycle and create roadblocks on the way. And you also see, in the case of security and testing, that DevOps may even go over only development or only operations responsibilities and tasks:
That's why to understand DevOps, instead of focusing on the name and what it means, we're focusing on what it tries to achieve: 💡
DevOps tries to remove all these roadblocks and things that slow down the release process, whatever that may be and instead of manual inefficient processes helps create fully automated streamlined processes for release cycles.
And this can be done step by step, removing one roadblock at a time until you have a fully optimized and automated DevOps process that makes your application release super easy. 🚀
DevOps Concept as a Solution
So how does DevOps help achieve this and solve all these challenges? 👀
Well, by the official definition, and this was the original idea of DevOps:
DevOps defines a combination of cultural philosophies, practices, and tools for doing that.
So DevOps is not just one set of tools or one specific concept; it's a combination of anything that creates the process of releasing the software fast and with high quality. And the main part of the concept was that developers and operations people should work together more often, talk to each other more often, and collaborate better to achieve that.
DevOps in Practice: DevOps as a separate Role
But actually, this definition is too broad and too high level and makes it hard to imagine how it works in practice. 🙉 So it's just not specific enough. So naturally, different companies implemented DevOps in different ways. So the actual implementation of DevOps looked pretty different from company to company.
But since companies started adopting it, gradually, it got a more concrete form with some of the common patterns across many companies. One of these patterns was that DevOps evolved into an actual role called a "Devops Engineer," where either developers are doing DevOps as a job next to development or operations are doing it, or someone is doing DevOps exclusively as their only job.
And a set of technologies that were used to implement the DevOps principles became DevOps technologies, which now DevOps engineers would need to learn:
And I understand that many people are resisting the idea of DevOps engineering, and the creators of the DevOps concept didn't see it used this way. Still, the reality is often different from the theory. We see that concept was adjusted and bent to meet the needs of the end goal, and the DevOps engineer role is what came out of it. 🤷♀️
And that DevOps role is responsible for creating a streamlined release process without any roadblocks slowing down the release, and that's why in the center of DevOps is the well-known Continuous Integration / Continuous Delivery process.
DevOps in Practice: How to become a DevOps Engineer
Check out the video to see how to become a DevOps Engineer:
After watching this, you may be thinking, these are a lot of things to learn, and it may be hard to know where to start or what to learn first, or what resources to use. 🤯
Well, there are many resources out there to learn individual DevOps technologies. 👍 I actually cover many of the DevOps technologies on my Youtube channel. 😊
But ideally, you want to follow a well-structured step-by-step roadmap and, more importantly, learn how to use these technologies together in combination because that's what DevOps engineers do. They use and integrate multiple technologies together to create DevOps processes, and of course, you want to learn all these with actual real-life project examples to know what it will look like in a real job. And very few courses and learning resources offer this, and that's exactly why we created a complete DevOps Bootcamp with a clear structure and lots of hands-on projects.
So if you're thinking about becoming a DevOps engineer or slowly transitioning into DevOps, you can definitely check out our DevOps bootcamp 🚀
DevOps vs SRE - How SRE fits into the whole DevOps process
To get a full picture of DevOps, I want to mention one more concept, which is SRE or Site Reliability Engineering, and how it fits into DevOps.
We just learned that there are two definitions of DevOps
the original definition, which is more high-level and broader and doesn't specify how exactly DevOps should be implemented
and a more practical one, which evolved over time with its own DevOps engineer role.
So when we compare DevOps with SRE, it's important to know which definition of DevOps we're using for this comparison.
1. First broader definition of DevOps vs. SRE DevOps is a more high-level concept that defines what needs to be done to achieve the automated, streamlined release process. At the same time, SRE is more specific about how to exactly implement this process and how to implement DevOps principles.
So many people would say that SRE is a specific implementation of the DevOps concepts:
2. Practical DevOps vs. SRE But as we saw, DevOps itself also became more practical with its own role and specific technologies and ways to implement it. So what's the comparison here? 🤔
Well, in many companies, this practical DevOps implementation became more focused. It concentrated on the speed of delivery for the application changes, and of course, even though it's part of the DevOps principles to not only release fast but release quality code, many DevOps teams in practice again seemed to optimize more for the speed than reliability.
So as a great complementary part of DevOps, SRE emerged with the same principles and goals in mind, which is to release quality code fast, but as the name suggests, more focused on reliability and keeping systems stable while allowing for fast changes:
So SRE is its own role with its own set of tools for making systems reliable. So these two were kind of parallel developments and are now often seen as two sides of the same coin, and it's not uncommon for teams to have both a DevOps engineer and SRE helping implement the DevOps principles.
What is SRE in detail
This was just a short look at SRE to understand it in comparison to DevOps. Still, since I have received many questions about what SRE is, I created a follow-up video on SRE to explain in more detail how SRE works in practice, what are the tasks and responsibilities of a Site Reliability engineer, and so on.
For DevOps, I hope I could clarify all your questions about it. If not, leave a comment in the video, and I will try to answer them 😊
Good luck on your DevOps journey! 🎉 💪
Be sure to check out our famous DevOps bootcamp, which is a complete educational program, where you learn how to build complete fully automated DevOps processes.
Other than that..
Like, share and follow me 😍 for more content: