How to transform Software Engineers into DevOps
As a Lead DevOps in a multinational insurance company, one of my responsibilities is to help developers build better applications by providing CICD (Jenkins), monitoring (Prometheus), access to AWS Services, a stable platform (RedHat OpenShift), … It also became evident that teaching all those technologies/tools was necessary to empower them.
So, the question is: How can I help them embrace the DevOps mindset, and, ultimately, handle development and operations activities?
Of course, the following might not apply to all situations and use-cases, but I think it’s generic enough to give the big picture and the general direction to follow.
1 — Jenkins Pipelines and CICD autonomy
Not all Software Engineers know about Jenkins, Groovy, Jenkinsfile, and CICD stages. Sometimes, they are developers: focusing on their code, on how to make a business requirement into a technical reality, and everything else is just noise. This is our job, as DevOps, to provide this CICD and to show them how it works, how they can manage it.
In order to provide developers the capacity to grow in autonomy, I’ve been using seed jobs (Job DSL) that creates pipeline jobs based on configuration files: this setup offers a solution where Software Engineers can slowly gain autonomy and knowledge, if they wish, to take ownership of their Jenkins pipeline.
- By default, projects will use the “standard” set of Jenkins pipelines. It will run the quality check, security check, build and deploy the application and provide promotion of Docker images from Dev to Prod environment. This is a way to provide CICD for ALL projects, even if the Software Engineers don’t have the required skill (or the time) to manage the associated Jenkinsfiles.
- When the project team feels more comfortable, they can “take over” the standard set by providing their own Jenkins pipelines and changing the configuration in the job DSL. This way, they can slowly learn how to integrate more stages in their CI, learn Groovy, familiarize themselves with Jenkins and understand the Continuous Release process.
Benefits: Software Engineers are slowly managing their CICD process and gaining Jenkins / Groovy knowledge. They are slowly empowered and become more familiar with DevOps tools and concepts (automation).
2 — Prometheus and Monitoring autonomy
Monitoring is a huge part of the DevOps mindset. That’s why it is crucial to give the ability to a developer to:
- understand the underlying concept of instrumenting / monitoring applications
- be able to monitor their own application
- create alerts specifically designed for their application
Additionally, DevOps should also have access to platform metrics and alerts to measure the stability and trigger the appropriate action if anything happens.
In order to provide such capabilities, I choose to deploy a Prometheus instance in each OpenShift project, and a Master Prometheus to gather all the metrics by pulling the “/federate” endpoint.
This situation allows Software Engineers and DevOps to benefits from monitoring / alerting of applications and platforms while offering a place to test Prometheus queries and alert rules.
Benefits: Software Engineers becomes aware of the ecosystem where their application is running. They can start monitoring their application (request duration, amount of request for a specific endpoint, …) and the platform where it runs (number of pods running, memory, CPU, …).
3 — Training and Q&A session
Knowledge is valuable only if it can be passed on others, which is why training sessions are so important. An effective format to maximize retention of information will look like this:
- 20–30 minutes of presentation
- 20–30 minutes of Hands-on or guided exercises
This way, Software Engineers (or other people that wants to join the training) can learn the basics of technology and start practicing it right away. Here are some of the main 2018-19 topics that Software Engineers should consider learning:
- Docker (https://medium.com/free-code-camp/a-beginner-friendly-introduction-to-containers-vms-and-docker-79a9e3e119b)
- Kubernetes (https://medium.com/free-code-camp/learn-kubernetes-in-under-3-hours-a-detailed-guide-to-orchestrating-containers-114ff420e882) / OpenShift (https://learn.openshift.com/)
- Jenkins (https://jenkins.io/)
- Prometheus (https://prometheus.io/)
On top of those training sessions, organizing “public Q&A”, open to everyone, 1h every week is a fantastic way of getting more specific use-cases. Here, I organize “AWS & OpenShift Q&A” sessions that target anyone (technical or not) with a specific question on AWS, OpenShift, or anything related. It’s a good “informal” way to spread the knowledge and to create curiosity.
Benefits: Software Engineers have the opportunity to learn new technologies, languages, and tools. They also have a dedicated time slot where they can ask their questions and dive into specific use-cases.
4 — Ownership of their application up until production
One of the most famous aspects of the DevOps mindset is to remove the metaphorical “wall” between developers and operations. One way of doing it is by providing privileges and granting access to developers up to the production environment.
This can be difficult in big companies though, because of something called “segregation of duty” that prevents developers of managing their own application production and pre-production environment, which strongly restricts their ownership.
To work around this situation, multiple actions are possible:
- Using Jenkins as the deployment tool and give access to developers to trigger the promotion of images up to pre-production.
- Shadow (as in “peer program”) with the developer while creating the production/preproduction environments
- Pair 2 Software Engineers and make them owners of each other’s application: this way, “segregation of duty” is left intact, and developers can still work in a “production” mode.
Benefits: Software Engineers start to learn how to run and operate an application from development to production, bringing valuable “ops” knowledge.
5 — Organizing security events (Capture the flag)
Since a couple of years, the new trend is “DevSecOps”: integrating security in the application development process, from start to finish. In a large company, the objective is to raise awareness about security issues related to the application (XSS, SQL Injection, …) and platform (managing Secrets, …).
The first thing will be to integrate security tools in the CICD process (OWASP Dependency check, Jfrog XRAY, CheckMarx, …). Fortunately, those tools come with very simple plugins or APIs to integrate with Jenkins and other CICD platforms.
Additionally, a very fun and interactive way of spreading knowledge about security is organizing a Capture the Flag (CTF) event. A CTF is a security competition where participants will have to find security flaws on a vulnerable application to earn points.
The first CTF that we ran was using the fantastic OWASP Juice-shop application: it’s an all-in-one vulnerable application that you can easily deploy as a container. More information can be found on their GitHub page: https://github.com/bkimminich/juice-shop
This event was a good way to learn security best practices, understand attack vectors commonly used by hackers to find security breaches in applications and raise awareness about security issues.
Benefits: Software Engineers see security reports in the CICD Pipeline and start thinking with a security-first mindset. They’re also able to see what the common security pitfalls and the most used attack vectors on web applications are.
The shift in mindset for developers to understand DevOps is not easy. Having an organization that promotes and helps Software Engineers to practice the related skills and usage of the common tools is essential. Of course, tools are not enough: mindset, processes, peoples are fundamental too to successfully operate a Digital Transformation.
I hope this article helped you in some way to propose ideas or activities that can drive your developers / Software Engineers towards DevOps.
So, what have you done today to empower Software Engineers and lead them to DevOps? Share your thoughts and ideas in the comments!
Cheers!
Johann B.
