Introducing DevSecOps
DevSecOps is a cultural change aiming to integrate security into the rapid-release cycles
typical of modern software application development and delivery, known as DevOps.
The ultimate goal of DevSecOps is to have development, security, and operations teams
working together to create business value through the fast delivery of secure software
using a process of continuous security.
This integration is a concept that the IT industry has long wrestled with but has become
possible only today due to the many evolutions the software engineering industry has
undergone in the last 20 years. The Agile and DevOps movements promoted the
necessary culture and tools needed to bring DevSecOps into life.
This chapter explores what DevSecOps is, what we secure, and the benefits of
DevSecOps adoption. It concludes with common misconceptions about the term. I hope
that by the end of the chapter, you will be able to understand the difference between
DevSecOps, continuous security, and security as code.
The Three Faces of DevSecOps
During the QCOn 2019, Guy Podjarny, CEO of Snyk, gave a talk titled “The Three Faces of
DevSecOps”, which I find helpful to illustrate and simplify the meaning of the term
DevSecOps. For him, if DevOps has the following three components:
1. Culture
2. Methodologies
3. Tools
Following on that, in practical terms, DevSecOps simply means the following:
1. To introduce security into DevOps culture
2. To secure DevOps methodologies
3. To secure DevOps tools
Let’s examine in detail what each of these components implies.
Introducing Security into DevOps Culture
The cultural aspect, especially the idea of shared ownership that brought Dev and Ops
teams to work together, is the keystone of any DevOps program. So, we just need to get
InfoSec to the room and wait for better results, right? Perhaps, but in reality, creating
this collaborative environment took almost a decade to gain traction.
In my view, it’s important to remember that technical transformation takes time so we
can allow our teams to adapt at their own pace. For example, in established
organizations with separated groups, I’ve found it risky to start a DevOps journey and
simultaneously put security into the mix. It can be much more costly and time-
consuming than it would be in a startup that is beginning from scratch.
To help understand how the IT industry got into this moment, Figure 1-1 illustrates how
we used to create software in the old times of the waterfall model, composed of a linear
workflow. First, during the Concept and Planning stage, system analysts gathered a list
of stakeholders’ requirements. Then developers would spend months working on the
software’s Architecture and Design. Only when the code was fully baked would the IT
operations team get involved to begin to prepare for Implementation (usually involving
late work or weekend journeys).
During the Testing and Bug Fixing phase, the company would conduct several
assessments, including application security audits. Then the security auditors would
provide a list of vulnerabilities along with a remediation plan, and, you guessed it, the
company could take even more valuable time to implement it. Or to release it anyway.
Figure 1-1. Waterfall model’s workflow
In 2001 the Agile movement acknowledged the fact that any software project is
constantly changing and urged retirement of the idea of linear workflow. Through the
fast and continuous release of software, the Agile development lifecycle allowed
developers to receive earlier feedback from their customers and identify problems
before the product reached implementation, as shown in Figure 1-2.
Figure 1-2. The Agile development lifecycle indicates the steady flow of feedback
In 2008 Patrick Debois presented the notion of Agile Infrastructure at the Agile
Conference in Toronto. He was looking for means to support the development team to
accelerate the push of new code into production, a concept also proposed by Andrew
Shafer. In 2009, at the O’Reilly Velocity Conference, John Allspaw and Paul Hammond
from Flickr gave their now-famous talk entitled “10 Deploys a Day,” consolidating the
notion that IT operators could be as agile as developers.
Allspaw and Hammond demonstrated that by stimulating a cultural change based on
communications and cooperation between the two teams, Flickr automated its
infrastructure and reached the process of continuous integration and deployment, as
shown in Figure 1-3. At the end of that year, Debois organized the first DevOpsDays
conference in Ghent, Belgium. Since then, other DevOpsDays events worldwide have
promoted the culture, methodologies, and tools, mainstreaming the term DevOps.
Figure 1-3. The DevOps infinity loop demonstrates how the collaboration between development and operations teams
resulted in continuous integration and continuous deployment
As DevOps is an expansion of Agile, so is DevSecOps an evolution of DevOps. Gene Kim
and Paul Love guided InfoSec professionals in strengthening relationships with IT
operations and development groups to advance IT objectives and business goals with
their book Visible Ops Security (IT Process Institute, 2008). However, the integration of
security into DevOps did not gain traction until 2012, when Shannon Lietz published the
DevSecOps manifesto. At the same time, James Wickett and Josh Corman advanced the
term Rugged DevOps, the combination of DevOps with the Rugged Manifesto.
Do we need a new term for the integration of security into DevOps culture? Should we
call it DevSecOps or Rugged DevOps? Perhaps we could continue to call it just plain and
simple DevOps. After all, the concept was never about just development and operations
teams working together -- it was about the conscious effort of breaking down all IT silos
to achieve business goals. While I agree with this definition, DevSecOps helps to
underline the teams’ synthesis and emphasizes the need to build up the knowledge and
skills so DevOps teams can perform security testing and fix it by themselves.
In Chapter 2, we’ll explore some of the essential capabilities to create a successful
DevSecOps program, but for now, let’s keep in mind that, as the term implies,
DevSecOps is an evolution of DevOps. It will depend highly on how good your
organization’s technical transformation maturity level is. Simply put: there is no
DevSecOps without DevOps and its culture.
Securing DevOps Methodologies
Along with Agile, the DevOps movement transformed software development and
delivery methodologies. It brought to the market a great set of new concepts such as
continuous integration and continuous delivery (CI/CD) pipelines. Designed to help
automate the steps between a developer’s submission of their code into the repository
and the release of that code into production, the workflow of a typical CI/CD pipeline
moves left to right, as illustrated in Figure 1-4.
DevSecOps is a cultural change aiming to integrate security into the rapid-release cycles
typical of modern software application development and delivery, known as DevOps.
The ultimate goal of DevSecOps is to have development, security, and operations teams
working together to create business value through the fast delivery of secure software
using a process of continuous security.
This integration is a concept that the IT industry has long wrestled with but has become
possible only today due to the many evolutions the software engineering industry has
undergone in the last 20 years. The Agile and DevOps movements promoted the
necessary culture and tools needed to bring DevSecOps into life.
This chapter explores what DevSecOps is, what we secure, and the benefits of
DevSecOps adoption. It concludes with common misconceptions about the term. I hope
that by the end of the chapter, you will be able to understand the difference between
DevSecOps, continuous security, and security as code.
DevSecOps is a cultural change aiming to integrate security into the rapid-release cycles
typical of modern software application development and delivery, known as DevOps.
The ultimate goal of DevSecOps is to have development, security, and operations teams
working together to create business value through the fast delivery of secure software
using a process of continuous security.
This integration is a concept that the IT industry has long wrestled with but has become
possible only today due to the many evolutions the software engineering industry has
undergone in the last 20 years. The Agile and DevOps movements promoted the
necessary culture and tools needed to bring DevSecOps into life.
This chapter explores what DevSecOps is, what we secure, and the benefits of
DevSecOps adoption. It concludes with common misconceptions about the term. I hope
that by the end of the chapter, you will be able to understand the difference between
DevSecOps, continuous security, and security as code.
DevSecOps is a cultural change aiming to integrate security into the rapid-release cycles
typical of modern software application development and delivery, known as DevOps.
The ultimate goal of DevSecOps is to have development, security, and operations teams
working together to create business value through the fast delivery of secure software
using a process of continuous security.
This integration is a concept that the IT industry has long wrestled with but has become
possible only today due to the many evolutions the software engineering industry has
undergone in the last 20 years. The Agile and DevOps movements promoted the
necessary culture and tools needed to bring DevSecOps into life.
This chapter explores what DevSecOps is, what we secure, and the benefits of
DevSecOps adoption. It concludes with common misconceptions about the term. I hope
that by the end of the chapter, you will be able to understand the difference between
DevSecOps, continuous security, and security as code.
Figure 1-4. The workflow of a typical CI/CD pipeline moves left to right.
Along with the expansion of infrastructure as code (IaC), an approach that emphasizes
consistent, repeatable routines for provisioning and changing systems and their
configuration, CI/CD pipelines enable the implementation of continuous security. In this
technical process, security activities are performed at every stage of the pipeline,
improving its quality and compliance adherence.
When one of these tests fails, we “shift left” along with the flow of information, giving
fast feedback to the developer or IT operator that ignited the action, allowing them to fix
their code. After the correction is created, a new push request is submitted, and the
process restarts.
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
A common misconception about DevSecOps is that InfoSec and DevOps have
irreconcilable goals due to the absence of conventional security controls and processes.
On the one hand, we have the zero-trust principle of never trust, always verify, while on
the other, trust is by far the most critical element of DevOps success. InfoSec
professionals are commonly divided into color-coded groups with well-defined
responsibilities such as red teams dedicated to offensive security, finding and exploiting
possible points of attacks, and blue teams, responsible for defensive security, protecting
the systems against real and perceived threats. Yet DevOps culture seeks to actively
break such barriers and silos. And in a DevSecOps setup, security is everyone’s
responsibility.
This doesn’t mean that organizations that use DevOps don’t have adequate controls and
processes. Instead, security activities are performed at every stage of software
development and delivery, improving its quality and compliance adherence, in a process
of continuous security.
This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps
cultures and processes, shifting from a reactive approach to a proactive one. It examines
key capabilities that are essential to creating a successful DevSecOps program and to
scale security as digital transformation becomes a reality for every business.
Accelerate: Capabilities to Drive Continuous
Security
Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting
research on measuring and improving software delivery. Their book Accelerate: The
Science of DevOps (IT Revolution Press, 2018) presents the findings and the science
behind their work: 24 key capabilities that influence the outcome of DevOps adoption
and that leaders should invest in to seek higher performance. This section explores ten
such capabilities that I believe every IT leader needs to promote continuous security.
Leadership
The first step for leaders heading the creation of a DevSecOps program is to understand
that its success depends highly on your organization’s technical transformation
maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be
introduced to the InfoSec team.
Next, be comfortable with the fact that it’s your role to lead the efforts to integrate
security into DevOps and generate a culture of high trust between the teams. Your
leadership skills can have powerful results, so invest the time to review lean
management practices on software delivery performance. IT leaders usually create a
dedicated transformation team, selecting some champions to accelerate change and
then scaling up their ideas and techniques for the entire team. Still, the challenge is
ultimately up to you.
As a leader, your message should be clear: security teams alone cannot avoid incidents.
Instead, their effort must be concentrated on improving the organization’s ability to
respond to these events. I understand that this is the most challenging part for InfoSec
professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a
distributed and software-defined infrastructure that makes it easy for anyone inside the
team to recreate entire IT environments in multiple regions turned our assets into
ephemeral objects. At the same time, the number of security attacks published daily
announces that this is a war we are clearly losing. We need more soldiers. As James
Wickett once pointed out, there are 100 developers available for every 10 operators,
and only one security professional. Once we do the math, we can see that only by
genuinely engaging development and operations will we take our defenses back.
Value Stream
In high-performance organizations, teams have a superior understanding of the flow of
work value, from the business all the way through to customers. This visibility creates a
sense of awareness about which products and features really matter to the company. If
you already have a DevOps program running, you can recognise the primary object that
we secure: products, platforms, and data that create value to the company goals.
Does it mean that other assets will be managed traditionally and organizations will have
a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing
organizations. Software-defined networking and its security are becoming popular
technologies, augmented by the use of artificial intelligence. Cloud-computing and
distributed offices are also breaking the perimeter concept, and for high-performance
companies, there are no more clearly defined lines between what is inside of the
network or somewhere in the cloud.
A good example for InfoSec professionals is endpoint security. Once upon a time, the
number of antivirus updates was one of the key metrics in our SOC dashboards. The
Covid-19 crisis and the sudden necessity of home office work globally reduced the level
of concerns high-performance organizations had with notebooks and who are using
them. Instead, the core of a DevSecOps strategy became how to make our products and
data resilient to any attack, be it by an internal or external device.
Collaboration
A successful DevSecOps program is all about better collaboration between teams.
Leaders must promote trust between the people involved, and a starting point is to
work on the InfoSec communication style. As a craft that in the mind of many is
associated with the military, InfoSec sees its share of manipulative and coercive
language that sometimes induces fear and blame. On the other hand, DevOps embraces
a more blameless culture, where everyone feels safe to make mistakes. In a
psychologically secure environment, professionals feel confident enough to express
their ideas and take risks, resulting in innovative changes.
Another DevOps best practice revolves around the idea of shared ownership and
common goals. Once it is established that security is now everyone’s responsibility, the
teams must work together to achieve such a goal. They can create and share
mechanisms or tools that help ensure the security of applications and platforms, such as
libraries and authentication and encryption services. With everything available and
easily searchable, it is easier for developers and operators to reuse secure code.
Post-mortem analysis, conducted after a security event happens, must also involve
developers and operators. After identifying the source of the attack, together they can
think about the necessary training and consider actions to prevent it from happening
again in the future, including the action plan on the team’s roadmap.
Shift Left
Shift left means more than making development and operations teams responsible for
securing what they built. It also means moving secure thinking to the earliest possible
point in the development and delivery process.
Security can be included in the product design phases, which can reduce later security
costs. Likewise, scans for insecure code and configuration mistakes can begin inside the
integrated development environment (IDE) software -- an application that provides
comprehensive facilities to computer programmers -- providing fast feedback and
improving code’s quality.
When a security issue is found, it should be centralized and tracked inside a unique
software issue platform. Some issues that may not need an InfoSec specialist to be fixed
can often be quickly solved by a senior DevOps professional, for example. This
lightweight change process accelerates and improves vulnerability fixes.
CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps
between a developer submitting their code and the release of that code into production,
run vulnerabilities assessments in the merge request so developers can fix them
immediately, as shown in Figure 2-1.
Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.
Empowered Teams
To give Dev and Ops ownership for security and thereby achieve better results, we need
to train them to make informed decisions and also make clear that the security
professionals are there to support them to make such choices effectively. Developers
and operations teams already are responsible for many tasks and may need to be
inspired to contribute to security. Otherwise, they may be tempted to merely plug a
scanner inside the CI/CD pipeline and wait for better results.
TIP
Are developers aware of secure development practices such as input validation and data cryptography? Are they able
to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online
community that produces freely available content and tools for web application security. It can be a valuable resource
for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure
Coding Practices" material and the Top 10 Web Application Security Risks list.
To show that security is there for DevOps, you can start by promoting an informal chat
between the two teams. While working at Etsy, Zane Lackey made t-shirts and other
swag and gave them to anyone who approached his team to ask a security question.
“Whether that was they reported a phishing email or a lead architect asking, Hey, can
you have a quick look at the service that we’re actually starting to think about doing?”
says Lackey. In the following weeks, the swag was seen inside the office, and people
kept asking security questions. This practice would basically do 90% of the branding
and cultural awareness work for a security team internally.
Test Automation
It is possible to bring more security to a DevOps pipeline by automating vulnerability
scanners. For code security, in the pre-build phase, we can use static application
security testing (SAST) tools: this is a test we perform in a non-production environment
that inspects an application’s code for coding flaws, back doors, and malware. Open
source tools such as Brakeman and SonarQube are good examples.
We can also scan dependencies at this stage. It is a matter of inventorying all
dependencies of binaries and executables and ensuring that these dependencies, over
which we often have no control, are free from vulnerabilities or malicious binaries. The
OWASP dependency check and the Snyk open source tool are the industry standard for
this test.
After deployment, we resort to dynamic application security testing (DAST) tools:
unlike static tests, DAST consists of assessments performed during execution,
monitoring items such as system memory, functional behavior, response time, and
overall system performance. This test is similar to tests that an attacker would do to
gain improper access to the application. Examples include OWASP ZAP, Arachini, and
AppScan. Some penetration tests can also be performed in an automated way using
tools like Metasploit, Burp Suite, and Nikto.
Infrastructure security tests can also be automated: from container to cloud security, a
tool can be included within the software delivery pipeline, as demonstrated in Figure 2-
2 and explored in more detail in Part 2 of this book.
Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.
Working in Small Batches
In the same way that development and operations slice their work into small pieces,
security tasks should be completed in a week or less. For example, instead of extensive
vulnerability assessments, teams must aim for rapid and effective security fixes. This is
precisely the idea behind the “start little and do often” principle of the Agile threat
modelling: a continuous, timeboxed process to help teams talk about risk and build
security within.
You can then create a user story in the team backlog. The report should be short and
include a description of the possible threats identified. Start small, identifying the
easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce
small victories and allow your team to continue to add new security stories at the
beginning of each new sprint.
TIP
The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for
the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an
architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor.
Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and
create a report with potential risks and mitigation advice.
Once the team has the muscle memory of threat modelling, you can move to find the
highest value security tasks, the ones that will improve the resilience of your systems. In
this way, we decentralize and automate such vulnerability assessments, giving our Dev
and Ops teams fast feedback and the power to take the necessary measures in their
daily routine.
Team Experimentation
Another important guiding principle while adopting DevSecOps is to move from a
culture of fear to one of readiness. Teams must accept two facts: first, their software is
likely to be used in ways they don’t expect and will be exploited by unrelenting bad
actors or automated threats. Second, the security controls they put in place will
eventually fail. Accepting that, DevSecOps teams can actively test their resilience against
attacks to ruggedize their practices and be prepared to respond promptly to any
security incident, improving their systems’ resilience.
This readiness is the main idea behind chaos engineering, a strategy made popular
several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s
infrastructure by continually – and randomly – terminating instances in production to
check how the shutdown would impact business. Chaos engineering has now evolved
into security chaos engineering, and the tools to automate such resilience tests are in
growing development, including commercial products such as Verica and Gremlin.
Chapter 8 discusses this subject in more detail and you can also check out Aaron
Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).
Other fun activities to promote team experimentation include game days, with
development and operations teams playing capture the flag (CTF)-style competitions.
These can be important learning opportunities, especially if scenarios involving the
actual business are outlined.
Finally, teams can experiment with automating incident response activities to reduce
time to mitigate a critical incident, preventing further damage. Instead of using manual
and procedural checklists, there are plenty of open-source tools available such as
TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and
Deep Instinct.
A common misconception about DevSecOps is that InfoSec and DevOps have
irreconcilable goals due to the absence of conventional security controls and processes.
On the one hand, we have the zero-trust principle of never trust, always verify, while on
the other, trust is by far the most critical element of DevOps success. InfoSec
professionals are commonly divided into color-coded groups with well-defined
responsibilities such as red teams dedicated to offensive security, finding and exploiting
possible points of attacks, and blue teams, responsible for defensive security, protecting
the systems against real and perceived threats. Yet DevOps culture seeks to actively
break such barriers and silos. And in a DevSecOps setup, security is everyone’s
responsibility.
This doesn’t mean that organizations that use DevOps don’t have adequate controls and
processes. Instead, security activities are performed at every stage of software
development and delivery, improving its quality and compliance adherence, in a process
of continuous security.
This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps
cultures and processes, shifting from a reactive approach to a proactive one. It examines
key capabilities that are essential to creating a successful DevSecOps program and to
scale security as digital transformation becomes a reality for every business.
Accelerate: Capabilities to Drive Continuous
Security
Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting
research on measuring and improving software delivery. Their book Accelerate: The
Science of DevOps (IT Revolution Press, 2018) presents the findings and the science
behind their work: 24 key capabilities that influence the outcome of DevOps adoption
and that leaders should invest in to seek higher performance. This section explores ten
such capabilities that I believe every IT leader needs to promote continuous security.
Leadership
The first step for leaders heading the creation of a DevSecOps program is to understand
that its success depends highly on your organization’s technical transformation
maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be
introduced to the InfoSec team.
Next, be comfortable with the fact that it’s your role to lead the efforts to integrate
security into DevOps and generate a culture of high trust between the teams. Your
leadership skills can have powerful results, so invest the time to review lean
management practices on software delivery performance. IT leaders usually create a
dedicated transformation team, selecting some champions to accelerate change and
then scaling up their ideas and techniques for the entire team. Still, the challenge is
ultimately up to you.
As a leader, your message should be clear: security teams alone cannot avoid incidents.
Instead, their effort must be concentrated on improving the organization’s ability to
respond to these events. I understand that this is the most challenging part for InfoSec
professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a
distributed and software-defined infrastructure that makes it easy for anyone inside the
team to recreate entire IT environments in multiple regions turned our assets into
ephemeral objects. At the same time, the number of security attacks published daily
announces that this is a war we are clearly losing. We need more soldiers. As James
Wickett once pointed out, there are 100 developers available for every 10 operators,
and only one security professional. Once we do the math, we can see that only by
genuinely engaging development and operations will we take our defenses back.
Value Stream
In high-performance organizations, teams have a superior understanding of the flow of
work value, from the business all the way through to customers. This visibility creates a
sense of awareness about which products and features really matter to the company. If
you already have a DevOps program running, you can recognise the primary object that
we secure: products, platforms, and data that create value to the company goals.
Does it mean that other assets will be managed traditionally and organizations will have
a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing
organizations. Software-defined networking and its security are becoming popular
technologies, augmented by the use of artificial intelligence. Cloud-computing and
distributed offices are also breaking the perimeter concept, and for high-performance
companies, there are no more clearly defined lines between what is inside of the
network or somewhere in the cloud.
A good example for InfoSec professionals is endpoint security. Once upon a time, the
number of antivirus updates was one of the key metrics in our SOC dashboards. The
Covid-19 crisis and the sudden necessity of home office work globally reduced the level
of concerns high-performance organizations had with notebooks and who are using
them. Instead, the core of a DevSecOps strategy became how to make our products and
data resilient to any attack, be it by an internal or external device.
Collaboration
A successful DevSecOps program is all about better collaboration between teams.
Leaders must promote trust between the people involved, and a starting point is to
work on the InfoSec communication style. As a craft that in the mind of many is
associated with the military, InfoSec sees its share of manipulative and coercive
language that sometimes induces fear and blame. On the other hand, DevOps embraces
a more blameless culture, where everyone feels safe to make mistakes. In a
psychologically secure environment, professionals feel confident enough to express
their ideas and take risks, resulting in innovative changes.
Another DevOps best practice revolves around the idea of shared ownership and
common goals. Once it is established that security is now everyone’s responsibility, the
teams must work together to achieve such a goal. They can create and share
mechanisms or tools that help ensure the security of applications and platforms, such as
libraries and authentication and encryption services. With everything available and
easily searchable, it is easier for developers and operators to reuse secure code.
Post-mortem analysis, conducted after a security event happens, must also involve
developers and operators. After identifying the source of the attack, together they can
think about the necessary training and consider actions to prevent it from happening
again in the future, including the action plan on the team’s roadmap.
Shift Left
Shift left means more than making development and operations teams responsible for
securing what they built. It also means moving secure thinking to the earliest possible
point in the development and delivery process.
Security can be included in the product design phases, which can reduce later security
costs. Likewise, scans for insecure code and configuration mistakes can begin inside the
integrated development environment (IDE) software -- an application that provides
comprehensive facilities to computer programmers -- providing fast feedback and
improving code’s quality.
When a security issue is found, it should be centralized and tracked inside a unique
software issue platform. Some issues that may not need an InfoSec specialist to be fixed
can often be quickly solved by a senior DevOps professional, for example. This
lightweight change process accelerates and improves vulnerability fixes.
CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps
between a developer submitting their code and the release of that code into production,
run vulnerabilities assessments in the merge request so developers can fix them
immediately, as shown in Figure 2-1.
Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.
Empowered Teams
To give Dev and Ops ownership for security and thereby achieve better results, we need
to train them to make informed decisions and also make clear that the security
professionals are there to support them to make such choices effectively. Developers
and operations teams already are responsible for many tasks and may need to be
inspired to contribute to security. Otherwise, they may be tempted to merely plug a
scanner inside the CI/CD pipeline and wait for better results.
TIP
Are developers aware of secure development practices such as input validation and data cryptography? Are they able
to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online
community that produces freely available content and tools for web application security. It can be a valuable resource
for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure
Coding Practices" material and the Top 10 Web Application Security Risks list.
To show that security is there for DevOps, you can start by promoting an informal chat
between the two teams. While working at Etsy, Zane Lackey made t-shirts and other
swag and gave them to anyone who approached his team to ask a security question.
“Whether that was they reported a phishing email or a lead architect asking, Hey, can
you have a quick look at the service that we’re actually starting to think about doing?”
says Lackey. In the following weeks, the swag was seen inside the office, and people
kept asking security questions. This practice would basically do 90% of the branding
and cultural awareness work for a security team internally.
Test Automation
It is possible to bring more security to a DevOps pipeline by automating vulnerability
scanners. For code security, in the pre-build phase, we can use static application
security testing (SAST) tools: this is a test we perform in a non-production environment
that inspects an application’s code for coding flaws, back doors, and malware. Open
source tools such as Brakeman and SonarQube are good examples.
We can also scan dependencies at this stage. It is a matter of inventorying all
dependencies of binaries and executables and ensuring that these dependencies, over
which we often have no control, are free from vulnerabilities or malicious binaries. The
OWASP dependency check and the Snyk open source tool are the industry standard for
this test.
After deployment, we resort to dynamic application security testing (DAST) tools:
unlike static tests, DAST consists of assessments performed during execution,
monitoring items such as system memory, functional behavior, response time, and
overall system performance. This test is similar to tests that an attacker would do to
gain improper access to the application. Examples include OWASP ZAP, Arachini, and
AppScan. Some penetration tests can also be performed in an automated way using
tools like Metasploit, Burp Suite, and Nikto.
Infrastructure security tests can also be automated: from container to cloud security, a
tool can be included within the software delivery pipeline, as demonstrated in Figure 2-
2 and explored in more detail in Part 2 of this book.
Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.
Working in Small Batches
In the same way that development and operations slice their work into small pieces,
security tasks should be completed in a week or less. For example, instead of extensive
vulnerability assessments, teams must aim for rapid and effective security fixes. This is
precisely the idea behind the “start little and do often” principle of the Agile threat
modelling: a continuous, timeboxed process to help teams talk about risk and build
security within.
You can then create a user story in the team backlog. The report should be short and
include a description of the possible threats identified. Start small, identifying the
easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce
small victories and allow your team to continue to add new security stories at the
beginning of each new sprint.
TIP
The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for
the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an
architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor.
Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and
create a report with potential risks and mitigation advice.
Once the team has the muscle memory of threat modelling, you can move to find the
highest value security tasks, the ones that will improve the resilience of your systems. In
this way, we decentralize and automate such vulnerability assessments, giving our Dev
and Ops teams fast feedback and the power to take the necessary measures in their
daily routine.
Team Experimentation
Another important guiding principle while adopting DevSecOps is to move from a
culture of fear to one of readiness. Teams must accept two facts: first, their software is
likely to be used in ways they don’t expect and will be exploited by unrelenting bad
actors or automated threats. Second, the security controls they put in place will
eventually fail. Accepting that, DevSecOps teams can actively test their resilience against
attacks to ruggedize their practices and be prepared to respond promptly to any
security incident, improving their systems’ resilience.
This readiness is the main idea behind chaos engineering, a strategy made popular
several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s
infrastructure by continually – and randomly – terminating instances in production to
check how the shutdown would impact business. Chaos engineering has now evolved
into security chaos engineering, and the tools to automate such resilience tests are in
growing development, including commercial products such as Verica and Gremlin.
Chapter 8 discusses this subject in more detail and you can also check out Aaron
Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).
Other fun activities to promote team experimentation include game days, with
development and operations teams playing capture the flag (CTF)-style competitions.
These can be important learning opportunities, especially if scenarios involving the
actual business are outlined.
Finally, teams can experiment with automating incident response activities to reduce
time to mitigate a critical incident, preventing further damage. Instead of using manual
and procedural checklists, there are plenty of open-source tools available such as
TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and
Deep Instinct.
A common misconception about DevSecOps is that InfoSec and DevOps have
irreconcilable goals due to the absence of conventional security controls and processes.
On the one hand, we have the zero-trust principle of never trust, always verify, while on
the other, trust is by far the most critical element of DevOps success. InfoSec
professionals are commonly divided into color-coded groups with well-defined
responsibilities such as red teams dedicated to offensive security, finding and exploiting
possible points of attacks, and blue teams, responsible for defensive security, protecting
the systems against real and perceived threats. Yet DevOps culture seeks to actively
break such barriers and silos. And in a DevSecOps setup, security is everyone’s
responsibility.
This doesn’t mean that organizations that use DevOps don’t have adequate controls and
processes. Instead, security activities are performed at every stage of software
development and delivery, improving its quality and compliance adherence, in a process
of continuous security.
This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps
cultures and processes, shifting from a reactive approach to a proactive one. It examines
key capabilities that are essential to creating a successful DevSecOps program and to
scale security as digital transformation becomes a reality for every business.
Accelerate: Capabilities to Drive Continuous
Security
Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting
research on measuring and improving software delivery. Their book Accelerate: The
Science of DevOps (IT Revolution Press, 2018) presents the findings and the science
behind their work: 24 key capabilities that influence the outcome of DevOps adoption
and that leaders should invest in to seek higher performance. This section explores ten
such capabilities that I believe every IT leader needs to promote continuous security.
Leadership
The first step for leaders heading the creation of a DevSecOps program is to understand
that its success depends highly on your organization’s technical transformation
maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be
introduced to the InfoSec team.
Next, be comfortable with the fact that it’s your role to lead the efforts to integrate
security into DevOps and generate a culture of high trust between the teams. Your
leadership skills can have powerful results, so invest the time to review lean
management practices on software delivery performance. IT leaders usually create a
dedicated transformation team, selecting some champions to accelerate change and
then scaling up their ideas and techniques for the entire team. Still, the challenge is
ultimately up to you.
As a leader, your message should be clear: security teams alone cannot avoid incidents.
Instead, their effort must be concentrated on improving the organization’s ability to
respond to these events. I understand that this is the most challenging part for InfoSec
professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a
distributed and software-defined infrastructure that makes it easy for anyone inside the
team to recreate entire IT environments in multiple regions turned our assets into
ephemeral objects. At the same time, the number of security attacks published daily
announces that this is a war we are clearly losing. We need more soldiers. As James
Wickett once pointed out, there are 100 developers available for every 10 operators,
and only one security professional. Once we do the math, we can see that only by
genuinely engaging development and operations will we take our defenses back.
Value Stream
In high-performance organizations, teams have a superior understanding of the flow of
work value, from the business all the way through to customers. This visibility creates a
sense of awareness about which products and features really matter to the company. If
you already have a DevOps program running, you can recognise the primary object that
we secure: products, platforms, and data that create value to the company goals.
Does it mean that other assets will be managed traditionally and organizations will have
a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing
organizations. Software-defined networking and its security are becoming popular
technologies, augmented by the use of artificial intelligence. Cloud-computing and
distributed offices are also breaking the perimeter concept, and for high-performance
companies, there are no more clearly defined lines between what is inside of the
network or somewhere in the cloud.
A good example for InfoSec professionals is endpoint security. Once upon a time, the
number of antivirus updates was one of the key metrics in our SOC dashboards. The
Covid-19 crisis and the sudden necessity of home office work globally reduced the level
of concerns high-performance organizations had with notebooks and who are using
them. Instead, the core of a DevSecOps strategy became how to make our products and
data resilient to any attack, be it by an internal or external device.
Collaboration
A successful DevSecOps program is all about better collaboration between teams.
Leaders must promote trust between the people involved, and a starting point is to
work on the InfoSec communication style. As a craft that in the mind of many is
associated with the military, InfoSec sees its share of manipulative and coercive
language that sometimes induces fear and blame. On the other hand, DevOps embraces
a more blameless culture, where everyone feels safe to make mistakes. In a
psychologically secure environment, professionals feel confident enough to express
their ideas and take risks, resulting in innovative changes.
Another DevOps best practice revolves around the idea of shared ownership and
common goals. Once it is established that security is now everyone’s responsibility, the
teams must work together to achieve such a goal. They can create and share
mechanisms or tools that help ensure the security of applications and platforms, such as
libraries and authentication and encryption services. With everything available and
easily searchable, it is easier for developers and operators to reuse secure code.
Post-mortem analysis, conducted after a security event happens, must also involve
developers and operators. After identifying the source of the attack, together they can
think about the necessary training and consider actions to prevent it from happening
again in the future, including the action plan on the team’s roadmap.
Shift Left
Shift left means more than making development and operations teams responsible for
securing what they built. It also means moving secure thinking to the earliest possible
point in the development and delivery process.
Security can be included in the product design phases, which can reduce later security
costs. Likewise, scans for insecure code and configuration mistakes can begin inside the
integrated development environment (IDE) software -- an application that provides
comprehensive facilities to computer programmers -- providing fast feedback and
improving code’s quality.
When a security issue is found, it should be centralized and tracked inside a unique
software issue platform. Some issues that may not need an InfoSec specialist to be fixed
can often be quickly solved by a senior DevOps professional, for example. This
lightweight change process accelerates and improves vulnerability fixes.
CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps
between a developer submitting their code and the release of that code into production,
run vulnerabilities assessments in the merge request so developers can fix them
immediately, as shown in Figure 2-1.
Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.
Empowered Teams
To give Dev and Ops ownership for security and thereby achieve better results, we need
to train them to make informed decisions and also make clear that the security
professionals are there to support them to make such choices effectively. Developers
and operations teams already are responsible for many tasks and may need to be
inspired to contribute to security. Otherwise, they may be tempted to merely plug a
scanner inside the CI/CD pipeline and wait for better results.
TIP
Are developers aware of secure development practices such as input validation and data cryptography? Are they able
to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online
community that produces freely available content and tools for web application security. It can be a valuable resource
for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure
Coding Practices" material and the Top 10 Web Application Security Risks list.
To show that security is there for DevOps, you can start by promoting an informal chat
between the two teams. While working at Etsy, Zane Lackey made t-shirts and other
swag and gave them to anyone who approached his team to ask a security question.
“Whether that was they reported a phishing email or a lead architect asking, Hey, can
you have a quick look at the service that we’re actually starting to think about doing?”
says Lackey. In the following weeks, the swag was seen inside the office, and people
kept asking security questions. This practice would basically do 90% of the branding
and cultural awareness work for a security team internally.
Test Automation
It is possible to bring more security to a DevOps pipeline by automating vulnerability
scanners. For code security, in the pre-build phase, we can use static application
security testing (SAST) tools: this is a test we perform in a non-production environment
that inspects an application’s code for coding flaws, back doors, and malware. Open
source tools such as Brakeman and SonarQube are good examples.
We can also scan dependencies at this stage. It is a matter of inventorying all
dependencies of binaries and executables and ensuring that these dependencies, over
which we often have no control, are free from vulnerabilities or malicious binaries. The
OWASP dependency check and the Snyk open source tool are the industry standard for
this test.
After deployment, we resort to dynamic application security testing (DAST) tools:
unlike static tests, DAST consists of assessments performed during execution,
monitoring items such as system memory, functional behavior, response time, and
overall system performance. This test is similar to tests that an attacker would do to
gain improper access to the application. Examples include OWASP ZAP, Arachini, and
AppScan. Some penetration tests can also be performed in an automated way using
tools like Metasploit, Burp Suite, and Nikto.
Infrastructure security tests can also be automated: from container to cloud security, a
tool can be included within the software delivery pipeline, as demonstrated in Figure 2-
2 and explored in more detail in Part 2 of this book.
Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.
Working in Small Batches
In the same way that development and operations slice their work into small pieces,
security tasks should be completed in a week or less. For example, instead of extensive
vulnerability assessments, teams must aim for rapid and effective security fixes. This is
precisely the idea behind the “start little and do often” principle of the Agile threat
modelling: a continuous, timeboxed process to help teams talk about risk and build
security within.
You can then create a user story in the team backlog. The report should be short and
include a description of the possible threats identified. Start small, identifying the
easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce
small victories and allow your team to continue to add new security stories at the
beginning of each new sprint.
TIP
The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for
the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an
architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor.
Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and
create a report with potential risks and mitigation advice.
Once the team has the muscle memory of threat modelling, you can move to find the
highest value security tasks, the ones that will improve the resilience of your systems. In
this way, we decentralize and automate such vulnerability assessments, giving our Dev
and Ops teams fast feedback and the power to take the necessary measures in their
daily routine.
Team Experimentation
Another important guiding principle while adopting DevSecOps is to move from a
culture of fear to one of readiness. Teams must accept two facts: first, their software is
likely to be used in ways they don’t expect and will be exploited by unrelenting bad
actors or automated threats. Second, the security controls they put in place will
eventually fail. Accepting that, DevSecOps teams can actively test their resilience against
attacks to ruggedize their practices and be prepared to respond promptly to any
security incident, improving their systems’ resilience.
This readiness is the main idea behind chaos engineering, a strategy made popular
several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s
infrastructure by continually – and randomly – terminating instances in production to
check how the shutdown would impact business. Chaos engineering has now evolved
into security chaos engineering, and the tools to automate such resilience tests are in
growing development, including commercial products such as Verica and Gremlin.
Chapter 8 discusses this subject in more detail and you can also check out Aaron
Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).
Other fun activities to promote team experimentation include game days, with
development and operations teams playing capture the flag (CTF)-style competitions.
These can be important learning opportunities, especially if scenarios involving the
actual business are outlined.
Finally, teams can experiment with automating incident response activities to reduce
time to mitigate a critical incident, preventing further damage. Instead of using manual
and procedural checklists, there are plenty of open-source tools available such as
TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and
Deep Instinct.
A common misconception about DevSecOps is that InfoSec and DevOps have
irreconcilable goals due to the absence of conventional security controls and processes.
On the one hand, we have the zero-trust principle of never trust, always verify, while on
the other, trust is by far the most critical element of DevOps success. InfoSec
professionals are commonly divided into color-coded groups with well-defined
responsibilities such as red teams dedicated to offensive security, finding and exploiting
possible points of attacks, and blue teams, responsible for defensive security, protecting
the systems against real and perceived threats. Yet DevOps culture seeks to actively
break such barriers and silos. And in a DevSecOps setup, security is everyone’s
responsibility.
This doesn’t mean that organizations that use DevOps don’t have adequate controls and
processes. Instead, security activities are performed at every stage of software
development and delivery, improving its quality and compliance adherence, in a process
of continuous security.
This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps
cultures and processes, shifting from a reactive approach to a proactive one. It examines
key capabilities that are essential to creating a successful DevSecOps program and to
scale security as digital transformation becomes a reality for every business.
Accelerate: Capabilities to Drive Continuous
Security
Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting
research on measuring and improving software delivery. Their book Accelerate: The
Science of DevOps (IT Revolution Press, 2018) presents the findings and the science
behind their work: 24 key capabilities that influence the outcome of DevOps adoption
and that leaders should invest in to seek higher performance. This section explores ten
such capabilities that I believe every IT leader needs to promote continuous security.
Leadership
The first step for leaders heading the creation of a DevSecOps program is to understand
that its success depends highly on your organization’s technical transformation
maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be
introduced to the InfoSec team.
Next, be comfortable with the fact that it’s your role to lead the efforts to integrate
security into DevOps and generate a culture of high trust between the teams. Your
leadership skills can have powerful results, so invest the time to review lean
management practices on software delivery performance. IT leaders usually create a
dedicated transformation team, selecting some champions to accelerate change and
then scaling up their ideas and techniques for the entire team. Still, the challenge is
ultimately up to you.
As a leader, your message should be clear: security teams alone cannot avoid incidents.
Instead, their effort must be concentrated on improving the organization’s ability to
respond to these events. I understand that this is the most challenging part for InfoSec
professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a
distributed and software-defined infrastructure that makes it easy for anyone inside the
team to recreate entire IT environments in multiple regions turned our assets into
ephemeral objects. At the same time, the number of security attacks published daily
announces that this is a war we are clearly losing. We need more soldiers. As James
Wickett once pointed out, there are 100 developers available for every 10 operators,
and only one security professional. Once we do the math, we can see that only by
genuinely engaging development and operations will we take our defenses back.
Value Stream
In high-performance organizations, teams have a superior understanding of the flow of
work value, from the business all the way through to customers. This visibility creates a
sense of awareness about which products and features really matter to the company. If
you already have a DevOps program running, you can recognise the primary object that
we secure: products, platforms, and data that create value to the company goals.
Does it mean that other assets will be managed traditionally and organizations will have
a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing
organizations. Software-defined networking and its security are becoming popular
technologies, augmented by the use of artificial intelligence. Cloud-computing and
distributed offices are also breaking the perimeter concept, and for high-performance
companies, there are no more clearly defined lines between what is inside of the
network or somewhere in the cloud.
A good example for InfoSec professionals is endpoint security. Once upon a time, the
number of antivirus updates was one of the key metrics in our SOC dashboards. The
Covid-19 crisis and the sudden necessity of home office work globally reduced the level
of concerns high-performance organizations had with notebooks and who are using
them. Instead, the core of a DevSecOps strategy became how to make our products and
data resilient to any attack, be it by an internal or external device.
Collaboration
A successful DevSecOps program is all about better collaboration between teams.
Leaders must promote trust between the people involved, and a starting point is to
work on the InfoSec communication style. As a craft that in the mind of many is
associated with the military, InfoSec sees its share of manipulative and coercive
language that sometimes induces fear and blame. On the other hand, DevOps embraces
a more blameless culture, where everyone feels safe to make mistakes. In a
psychologically secure environment, professionals feel confident enough to express
their ideas and take risks, resulting in innovative changes.
Another DevOps best practice revolves around the idea of shared ownership and
common goals. Once it is established that security is now everyone’s responsibility, the
teams must work together to achieve such a goal. They can create and share
mechanisms or tools that help ensure the security of applications and platforms, such as
libraries and authentication and encryption services. With everything available and
easily searchable, it is easier for developers and operators to reuse secure code.
Post-mortem analysis, conducted after a security event happens, must also involve
developers and operators. After identifying the source of the attack, together they can
think about the necessary training and consider actions to prevent it from happening
again in the future, including the action plan on the team’s roadmap.
Shift Left
Shift left means more than making development and operations teams responsible for
securing what they built. It also means moving secure thinking to the earliest possible
point in the development and delivery process.
Security can be included in the product design phases, which can reduce later security
costs. Likewise, scans for insecure code and configuration mistakes can begin inside the
integrated development environment (IDE) software -- an application that provides
comprehensive facilities to computer programmers -- providing fast feedback and
improving code’s quality.
When a security issue is found, it should be centralized and tracked inside a unique
software issue platform. Some issues that may not need an InfoSec specialist to be fixed
can often be quickly solved by a senior DevOps professional, for example. This
lightweight change process accelerates and improves vulnerability fixes.
CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps
between a developer submitting their code and the release of that code into production,
run vulnerabilities assessments in the merge request so developers can fix them
immediately, as shown in Figure 2-1.
Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.
Empowered Teams
To give Dev and Ops ownership for security and thereby achieve better results, we need
to train them to make informed decisions and also make clear that the security
professionals are there to support them to make such choices effectively. Developers
and operations teams already are responsible for many tasks and may need to be
inspired to contribute to security. Otherwise, they may be tempted to merely plug a
scanner inside the CI/CD pipeline and wait for better results.
TIP
Are developers aware of secure development practices such as input validation and data cryptography? Are they able
to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online
community that produces freely available content and tools for web application security. It can be a valuable resource
for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure
Coding Practices" material and the Top 10 Web Application Security Risks list.
To show that security is there for DevOps, you can start by promoting an informal chat
between the two teams. While working at Etsy, Zane Lackey made t-shirts and other
swag and gave them to anyone who approached his team to ask a security question.
“Whether that was they reported a phishing email or a lead architect asking, Hey, can
you have a quick look at the service that we’re actually starting to think about doing?”
says Lackey. In the following weeks, the swag was seen inside the office, and people
kept asking security questions. This practice would basically do 90% of the branding
and cultural awareness work for a security team internally.
Test Automation
It is possible to bring more security to a DevOps pipeline by automating vulnerability
scanners. For code security, in the pre-build phase, we can use static application
security testing (SAST) tools: this is a test we perform in a non-production environment
that inspects an application’s code for coding flaws, back doors, and malware. Open
source tools such as Brakeman and SonarQube are good examples.
We can also scan dependencies at this stage. It is a matter of inventorying all
dependencies of binaries and executables and ensuring that these dependencies, over
which we often have no control, are free from vulnerabilities or malicious binaries. The
OWASP dependency check and the Snyk open source tool are the industry standard for
this test.
After deployment, we resort to dynamic application security testing (DAST) tools:
unlike static tests, DAST consists of assessments performed during execution,
monitoring items such as system memory, functional behavior, response time, and
overall system performance. This test is similar to tests that an attacker would do to
gain improper access to the application. Examples include OWASP ZAP, Arachini, and
AppScan. Some penetration tests can also be performed in an automated way using
tools like Metasploit, Burp Suite, and Nikto.
Infrastructure security tests can also be automated: from container to cloud security, a
tool can be included within the software delivery pipeline, as demonstrated in Figure 2-
2 and explored in more detail in Part 2 of this book.
Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.
Working in Small Batches
In the same way that development and operations slice their work into small pieces,
security tasks should be completed in a week or less. For example, instead of extensive
vulnerability assessments, teams must aim for rapid and effective security fixes. This is
precisely the idea behind the “start little and do often” principle of the Agile threat
modelling: a continuous, timeboxed process to help teams talk about risk and build
security within.
You can then create a user story in the team backlog. The report should be short and
include a description of the possible threats identified. Start small, identifying the
easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce
small victories and allow your team to continue to add new security stories at the
beginning of each new sprint.
TIP
The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for
the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an
architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor.
Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and
create a report with potential risks and mitigation advice.
Once the team has the muscle memory of threat modelling, you can move to find the
highest value security tasks, the ones that will improve the resilience of your systems. In
this way, we decentralize and automate such vulnerability assessments, giving our Dev
and Ops teams fast feedback and the power to take the necessary measures in their
daily routine.
Team Experimentation
Another important guiding principle while adopting DevSecOps is to move from a
culture of fear to one of readiness. Teams must accept two facts: first, their software is
likely to be used in ways they don’t expect and will be exploited by unrelenting bad
actors or automated threats. Second, the security controls they put in place will
eventually fail. Accepting that, DevSecOps teams can actively test their resilience against
attacks to ruggedize their practices and be prepared to respond promptly to any
security incident, improving their systems’ resilience.
This readiness is the main idea behind chaos engineering, a strategy made popular
several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s
infrastructure by continually – and randomly – terminating instances in production to
check how the shutdown would impact business. Chaos engineering has now evolved
into security chaos engineering, and the tools to automate such resilience tests are in
growing development, including commercial products such as Verica and Gremlin.
Chapter 8 discusses this subject in more detail and you can also check out Aaron
Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).
Other fun activities to promote team experimentation include game days, with
development and operations teams playing capture the flag (CTF)-style competitions.
These can be important learning opportunities, especially if scenarios involving the
actual business are outlined.
Finally, teams can experiment with automating incident response activities to reduce
time to mitigate a critical incident, preventing further damage. Instead of using manual
and procedural checklists, there are plenty of open-source tools available such as
TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and
Deep Instinct.
A common misconception about DevSecOps is that InfoSec and DevOps have
irreconcilable goals due to the absence of conventional security controls and processes.
On the one hand, we have the zero-trust principle of never trust, always verify, while on
the other, trust is by far the most critical element of DevOps success. InfoSec
professionals are commonly divided into color-coded groups with well-defined
responsibilities such as red teams dedicated to offensive security, finding and exploiting
possible points of attacks, and blue teams, responsible for defensive security, protecting
the systems against real and perceived threats. Yet DevOps culture seeks to actively
break such barriers and silos. And in a DevSecOps setup, security is everyone’s
responsibility.
This doesn’t mean that organizations that use DevOps don’t have adequate controls and
processes. Instead, security activities are performed at every stage of software
development and delivery, improving its quality and compliance adherence, in a process
of continuous security.
This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps
cultures and processes, shifting from a reactive approach to a proactive one. It examines
key capabilities that are essential to creating a successful DevSecOps program and to
scale security as digital transformation becomes a reality for every business.
Accelerate: Capabilities to Drive Continuous
Security
Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting
research on measuring and improving software delivery. Their book Accelerate: The
Science of DevOps (IT Revolution Press, 2018) presents the findings and the science
behind their work: 24 key capabilities that influence the outcome of DevOps adoption
and that leaders should invest in to seek higher performance. This section explores ten
such capabilities that I believe every IT leader needs to promote continuous security.
Leadership
The first step for leaders heading the creation of a DevSecOps program is to understand
that its success depends highly on your organization’s technical transformation
maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be
introduced to the InfoSec team.
Next, be comfortable with the fact that it’s your role to lead the efforts to integrate
security into DevOps and generate a culture of high trust between the teams. Your
leadership skills can have powerful results, so invest the time to review lean
management practices on software delivery performance. IT leaders usually create a
dedicated transformation team, selecting some champions to accelerate change and
then scaling up their ideas and techniques for the entire team. Still, the challenge is
ultimately up to you.
As a leader, your message should be clear: security teams alone cannot avoid incidents.
Instead, their effort must be concentrated on improving the organization’s ability to
respond to these events. I understand that this is the most challenging part for InfoSec
professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a
distributed and software-defined infrastructure that makes it easy for anyone inside the
team to recreate entire IT environments in multiple regions turned our assets into
ephemeral objects. At the same time, the number of security attacks published daily
announces that this is a war we are clearly losing. We need more soldiers. As James
Wickett once pointed out, there are 100 developers available for every 10 operators,
and only one security professional. Once we do the math, we can see that only by
genuinely engaging development and operations will we take our defenses back.
Value Stream
In high-performance organizations, teams have a superior understanding of the flow of
work value, from the business all the way through to customers. This visibility creates a
sense of awareness about which products and features really matter to the company. If
you already have a DevOps program running, you can recognise the primary object that
we secure: products, platforms, and data that create value to the company goals.
Does it mean that other assets will be managed traditionally and organizations will have
a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing
organizations. Software-defined networking and its security are becoming popular
technologies, augmented by the use of artificial intelligence. Cloud-computing and
distributed offices are also breaking the perimeter concept, and for high-performance
companies, there are no more clearly defined lines between what is inside of the
network or somewhere in the cloud.
A good example for InfoSec professionals is endpoint security. Once upon a time, the
number of antivirus updates was one of the key metrics in our SOC dashboards. The
Covid-19 crisis and the sudden necessity of home office work globally reduced the level
of concerns high-performance organizations had with notebooks and who are using
them. Instead, the core of a DevSecOps strategy became how to make our products and
data resilient to any attack, be it by an internal or external device.
Collaboration
A successful DevSecOps program is all about better collaboration between teams.
Leaders must promote trust between the people involved, and a starting point is to
work on the InfoSec communication style. As a craft that in the mind of many is
associated with the military, InfoSec sees its share of manipulative and coercive
language that sometimes induces fear and blame. On the other hand, DevOps embraces
a more blameless culture, where everyone feels safe to make mistakes. In a
psychologically secure environment, professionals feel confident enough to express
their ideas and take risks, resulting in innovative changes.
Another DevOps best practice revolves around the idea of shared ownership and
common goals. Once it is established that security is now everyone’s responsibility, the
teams must work together to achieve such a goal. They can create and share
mechanisms or tools that help ensure the security of applications and platforms, such as
libraries and authentication and encryption services. With everything available and
easily searchable, it is easier for developers and operators to reuse secure code.
Post-mortem analysis, conducted after a security event happens, must also involve
developers and operators. After identifying the source of the attack, together they can
think about the necessary training and consider actions to prevent it from happening
again in the future, including the action plan on the team’s roadmap.
Shift Left
Shift left means more than making development and operations teams responsible for
securing what they built. It also means moving secure thinking to the earliest possible
point in the development and delivery process.
Security can be included in the product design phases, which can reduce later security
costs. Likewise, scans for insecure code and configuration mistakes can begin inside the
integrated development environment (IDE) software -- an application that provides
comprehensive facilities to computer programmers -- providing fast feedback and
improving code’s quality.
When a security issue is found, it should be centralized and tracked inside a unique
software issue platform. Some issues that may not need an InfoSec specialist to be fixed
can often be quickly solved by a senior DevOps professional, for example. This
lightweight change process accelerates and improves vulnerability fixes.
CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps
between a developer submitting their code and the release of that code into production,
run vulnerabilities assessments in the merge request so developers can fix them
immediately, as shown in Figure 2-1.
Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.
Empowered Teams
To give Dev and Ops ownership for security and thereby achieve better results, we need
to train them to make informed decisions and also make clear that the security
professionals are there to support them to make such choices effectively. Developers
and operations teams already are responsible for many tasks and may need to be
inspired to contribute to security. Otherwise, they may be tempted to merely plug a
scanner inside the CI/CD pipeline and wait for better results.
TIP
Are developers aware of secure development practices such as input validation and data cryptography? Are they able
to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online
community that produces freely available content and tools for web application security. It can be a valuable resource
for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure
Coding Practices" material and the Top 10 Web Application Security Risks list.
To show that security is there for DevOps, you can start by promoting an informal chat
between the two teams. While working at Etsy, Zane Lackey made t-shirts and other
swag and gave them to anyone who approached his team to ask a security question.
“Whether that was they reported a phishing email or a lead architect asking, Hey, can
you have a quick look at the service that we’re actually starting to think about doing?”
says Lackey. In the following weeks, the swag was seen inside the office, and people
kept asking security questions. This practice would basically do 90% of the branding
and cultural awareness work for a security team internally.
Test Automation
It is possible to bring more security to a DevOps pipeline by automating vulnerability
scanners. For code security, in the pre-build phase, we can use static application
security testing (SAST) tools: this is a test we perform in a non-production environment
that inspects an application’s code for coding flaws, back doors, and malware. Open
source tools such as Brakeman and SonarQube are good examples.
We can also scan dependencies at this stage. It is a matter of inventorying all
dependencies of binaries and executables and ensuring that these dependencies, over
which we often have no control, are free from vulnerabilities or malicious binaries. The
OWASP dependency check and the Snyk open source tool are the industry standard for
this test.
After deployment, we resort to dynamic application security testing (DAST) tools:
unlike static tests, DAST consists of assessments performed during execution,
monitoring items such as system memory, functional behavior, response time, and
overall system performance. This test is similar to tests that an attacker would do to
gain improper access to the application. Examples include OWASP ZAP, Arachini, and
AppScan. Some penetration tests can also be performed in an automated way using
tools like Metasploit, Burp Suite, and Nikto.
Infrastructure security tests can also be automated: from container to cloud security, a
tool can be included within the software delivery pipeline, as demonstrated in Figure 2-
2 and explored in more detail in Part 2 of this book.
Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.
Working in Small Batches
In the same way that development and operations slice their work into small pieces,
security tasks should be completed in a week or less. For example, instead of extensive
vulnerability assessments, teams must aim for rapid and effective security fixes. This is
precisely the idea behind the “start little and do often” principle of the Agile threat
modelling: a continuous, timeboxed process to help teams talk about risk and build
security within.
You can then create a user story in the team backlog. The report should be short and
include a description of the possible threats identified. Start small, identifying the
easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce
small victories and allow your team to continue to add new security stories at the
beginning of each new sprint.
TIP
The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for
the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an
architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor.
Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and
create a report with potential risks and mitigation advice.
Once the team has the muscle memory of threat modelling, you can move to find the
highest value security tasks, the ones that will improve the resilience of your systems. In
this way, we decentralize and automate such vulnerability assessments, giving our Dev
and Ops teams fast feedback and the power to take the necessary measures in their
daily routine.
Team Experimentation
Another important guiding principle while adopting DevSecOps is to move from a
culture of fear to one of readiness. Teams must accept two facts: first, their software is
likely to be used in ways they don’t expect and will be exploited by unrelenting bad
actors or automated threats. Second, the security controls they put in place will
eventually fail. Accepting that, DevSecOps teams can actively test their resilience against
attacks to ruggedize their practices and be prepared to respond promptly to any
security incident, improving their systems’ resilience.
This readiness is the main idea behind chaos engineering, a strategy made popular
several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s
infrastructure by continually – and randomly – terminating instances in production to
check how the shutdown would impact business. Chaos engineering has now evolved
into security chaos engineering, and the tools to automate such resilience tests are in
growing development, including commercial products such as Verica and Gremlin.
Chapter 8 discusses this subject in more detail and you can also check out Aaron
Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).
Other fun activities to promote team experimentation include game days, with
development and operations teams playing capture the flag (CTF)-style competitions.
These can be important learning opportunities, especially if scenarios involving the
actual business are outlined.
Finally, teams can experiment with automating incident response activities to reduce
time to mitigate a critical incident, preventing further damage. Instead of using manual
and procedural checklists, there are plenty of open-source tools available such as
TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and
Deep Instinct.
A common misconception about DevSecOps is that InfoSec and DevOps have
irreconcilable goals due to the absence of conventional security controls and processes.
On the one hand, we have the zero-trust principle of never trust, always verify, while on
the other, trust is by far the most critical element of DevOps success. InfoSec
professionals are commonly divided into color-coded groups with well-defined
responsibilities such as red teams dedicated to offensive security, finding and exploiting
possible points of attacks, and blue teams, responsible for defensive security, protecting
the systems against real and perceived threats. Yet DevOps culture seeks to actively
break such barriers and silos. And in a DevSecOps setup, security is everyone’s
responsibility.
This doesn’t mean that organizations that use DevOps don’t have adequate controls and
processes. Instead, security activities are performed at every stage of software
development and delivery, improving its quality and compliance adherence, in a process
of continuous security.
This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps
cultures and processes, shifting from a reactive approach to a proactive one. It examines
key capabilities that are essential to creating a successful DevSecOps program and to
scale security as digital transformation becomes a reality for every business.
Accelerate: Capabilities to Drive Continuous
Security
Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting
research on measuring and improving software delivery. Their book Accelerate: The
Science of DevOps (IT Revolution Press, 2018) presents the findings and the science
behind their work: 24 key capabilities that influence the outcome of DevOps adoption
and that leaders should invest in to seek higher performance. This section explores ten
such capabilities that I believe every IT leader needs to promote continuous security.
Leadership
The first step for leaders heading the creation of a DevSecOps program is to understand
that its success depends highly on your organization’s technical transformation
maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be
introduced to the InfoSec team.
Next, be comfortable with the fact that it’s your role to lead the efforts to integrate
security into DevOps and generate a culture of high trust between the teams. Your
leadership skills can have powerful results, so invest the time to review lean
management practices on software delivery performance. IT leaders usually create a
dedicated transformation team, selecting some champions to accelerate change and
then scaling up their ideas and techniques for the entire team. Still, the challenge is
ultimately up to you.
As a leader, your message should be clear: security teams alone cannot avoid incidents.
Instead, their effort must be concentrated on improving the organization’s ability to
respond to these events. I understand that this is the most challenging part for InfoSec
professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a
distributed and software-defined infrastructure that makes it easy for anyone inside the
team to recreate entire IT environments in multiple regions turned our assets into
ephemeral objects. At the same time, the number of security attacks published daily
announces that this is a war we are clearly losing. We need more soldiers. As James
Wickett once pointed out, there are 100 developers available for every 10 operators,
and only one security professional. Once we do the math, we can see that only by
genuinely engaging development and operations will we take our defenses back.
Value Stream
In high-performance organizations, teams have a superior understanding of the flow of
work value, from the business all the way through to customers. This visibility creates a
sense of awareness about which products and features really matter to the company. If
you already have a DevOps program running, you can recognise the primary object that
we secure: products, platforms, and data that create value to the company goals.
Does it mean that other assets will be managed traditionally and organizations will have
a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing
organizations. Software-defined networking and its security are becoming popular
technologies, augmented by the use of artificial intelligence. Cloud-computing and
distributed offices are also breaking the perimeter concept, and for high-performance
companies, there are no more clearly defined lines between what is inside of the
network or somewhere in the cloud.
A good example for InfoSec professionals is endpoint security. Once upon a time, the
number of antivirus updates was one of the key metrics in our SOC dashboards. The
Covid-19 crisis and the sudden necessity of home office work globally reduced the level
of concerns high-performance organizations had with notebooks and who are using
them. Instead, the core of a DevSecOps strategy became how to make our products and
data resilient to any attack, be it by an internal or external device.
Collaboration
A successful DevSecOps program is all about better collaboration between teams.
Leaders must promote trust between the people involved, and a starting point is to
work on the InfoSec communication style. As a craft that in the mind of many is
associated with the military, InfoSec sees its share of manipulative and coercive
language that sometimes induces fear and blame. On the other hand, DevOps embraces
a more blameless culture, where everyone feels safe to make mistakes. In a
psychologically secure environment, professionals feel confident enough to express
their ideas and take risks, resulting in innovative changes.
Another DevOps best practice revolves around the idea of shared ownership and
common goals. Once it is established that security is now everyone’s responsibility, the
teams must work together to achieve such a goal. They can create and share
mechanisms or tools that help ensure the security of applications and platforms, such as
libraries and authentication and encryption services. With everything available and
easily searchable, it is easier for developers and operators to reuse secure code.
Post-mortem analysis, conducted after a security event happens, must also involve
developers and operators. After identifying the source of the attack, together they can
think about the necessary training and consider actions to prevent it from happening
again in the future, including the action plan on the team’s roadmap.
Shift Left
Shift left means more than making development and operations teams responsible for
securing what they built. It also means moving secure thinking to the earliest possible
point in the development and delivery process.
Security can be included in the product design phases, which can reduce later security
costs. Likewise, scans for insecure code and configuration mistakes can begin inside the
integrated development environment (IDE) software -- an application that provides
comprehensive facilities to computer programmers -- providing fast feedback and
improving code’s quality.
When a security issue is found, it should be centralized and tracked inside a unique
software issue platform. Some issues that may not need an InfoSec specialist to be fixed
can often be quickly solved by a senior DevOps professional, for example. This
lightweight change process accelerates and improves vulnerability fixes.
CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps
between a developer submitting their code and the release of that code into production,
run vulnerabilities assessments in the merge request so developers can fix them
immediately, as shown in Figure 2-1.
Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.
Empowered Teams
To give Dev and Ops ownership for security and thereby achieve better results, we need
to train them to make informed decisions and also make clear that the security
professionals are there to support them to make such choices effectively. Developers
and operations teams already are responsible for many tasks and may need to be
inspired to contribute to security. Otherwise, they may be tempted to merely plug a
scanner inside the CI/CD pipeline and wait for better results.
TIP
Are developers aware of secure development practices such as input validation and data cryptography? Are they able
to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online
community that produces freely available content and tools for web application security. It can be a valuable resource
for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure
Coding Practices" material and the Top 10 Web Application Security Risks list.
To show that security is there for DevOps, you can start by promoting an informal chat
between the two teams. While working at Etsy, Zane Lackey made t-shirts and other
swag and gave them to anyone who approached his team to ask a security question.
“Whether that was they reported a phishing email or a lead architect asking, Hey, can
you have a quick look at the service that we’re actually starting to think about doing?”
says Lackey. In the following weeks, the swag was seen inside the office, and people
kept asking security questions. This practice would basically do 90% of the branding
and cultural awareness work for a security team internally.
Test Automation
It is possible to bring more security to a DevOps pipeline by automating vulnerability
scanners. For code security, in the pre-build phase, we can use static application
security testing (SAST) tools: this is a test we perform in a non-production environment
that inspects an application’s code for coding flaws, back doors, and malware. Open
source tools such as Brakeman and SonarQube are good examples.
We can also scan dependencies at this stage. It is a matter of inventorying all
dependencies of binaries and executables and ensuring that these dependencies, over
which we often have no control, are free from vulnerabilities or malicious binaries. The
OWASP dependency check and the Snyk open source tool are the industry standard for
this test.
After deployment, we resort to dynamic application security testing (DAST) tools:
unlike static tests, DAST consists of assessments performed during execution,
monitoring items such as system memory, functional behavior, response time, and
overall system performance. This test is similar to tests that an attacker would do to
gain improper access to the application. Examples include OWASP ZAP, Arachini, and
AppScan. Some penetration tests can also be performed in an automated way using
tools like Metasploit, Burp Suite, and Nikto.
Infrastructure security tests can also be automated: from container to cloud security, a
tool can be included within the software delivery pipeline, as demonstrated in Figure 2-
2 and explored in more detail in Part 2 of this book.
Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.
Working in Small Batches
In the same way that development and operations slice their work into small pieces,
security tasks should be completed in a week or less. For example, instead of extensive
vulnerability assessments, teams must aim for rapid and effective security fixes. This is
precisely the idea behind the “start little and do often” principle of the Agile threat
modelling: a continuous, timeboxed process to help teams talk about risk and build
security within.
You can then create a user story in the team backlog. The report should be short and
include a description of the possible threats identified. Start small, identifying the
easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce
small victories and allow your team to continue to add new security stories at the
beginning of each new sprint.
TIP
The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for
the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an
architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor.
Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and
create a report with potential risks and mitigation advice.
Once the team has the muscle memory of threat modelling, you can move to find the
highest value security tasks, the ones that will improve the resilience of your systems. In
this way, we decentralize and automate such vulnerability assessments, giving our Dev
and Ops teams fast feedback and the power to take the necessary measures in their
daily routine.
Team Experimentation
Another important guiding principle while adopting DevSecOps is to move from a
culture of fear to one of readiness. Teams must accept two facts: first, their software is
likely to be used in ways they don’t expect and will be exploited by unrelenting bad
actors or automated threats. Second, the security controls they put in place will
eventually fail. Accepting that, DevSecOps teams can actively test their resilience against
attacks to ruggedize their practices and be prepared to respond promptly to any
security incident, improving their systems’ resilience.
This readiness is the main idea behind chaos engineering, a strategy made popular
several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s
infrastructure by continually – and randomly – terminating instances in production to
check how the shutdown would impact business. Chaos engineering has now evolved
into security chaos engineering, and the tools to automate such resilience tests are in
growing development, including commercial products such as Verica and Gremlin.
Chapter 8 discusses this subject in more detail and you can also check out Aaron
Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).
Other fun activities to promote team experimentation include game days, with
development and operations teams playing capture the flag (CTF)-style competitions.
These can be important learning opportunities, especially if scenarios involving the
actual business are outlined.
Finally, teams can experiment with automating incident response activities to reduce
time to mitigate a critical incident, preventing further damage. Instead of using manual
and procedural checklists, there are plenty of open-source tools available such as
TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and
Deep Instinct.
A common misconception about DevSecOps is that InfoSec and DevOps have
irreconcilable goals due to the absence of conventional security controls and processes.
On the one hand, we have the zero-trust principle of never trust, always verify, while on
the other, trust is by far the most critical element of DevOps success. InfoSec
professionals are commonly divided into color-coded groups with well-defined
responsibilities such as red teams dedicated to offensive security, finding and exploiting
possible points of attacks, and blue teams, responsible for defensive security, protecting
the systems against real and perceived threats. Yet DevOps culture seeks to actively
break such barriers and silos. And in a DevSecOps setup, security is everyone’s
responsibility.
This doesn’t mean that organizations that use DevOps don’t have adequate controls and
processes. Instead, security activities are performed at every stage of software
development and delivery, improving its quality and compliance adherence, in a process
of continuous security.
This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps
cultures and processes, shifting from a reactive approach to a proactive one. It examines
key capabilities that are essential to creating a successful DevSecOps program and to
scale security as digital transformation becomes a reality for every business.
Accelerate: Capabilities to Drive Continuous
Security
Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting
research on measuring and improving software delivery. Their book Accelerate: The
Science of DevOps (IT Revolution Press, 2018) presents the findings and the science
behind their work: 24 key capabilities that influence the outcome of DevOps adoption
and that leaders should invest in to seek higher performance. This section explores ten
such capabilities that I believe every IT leader needs to promote continuous security.
Leadership
The first step for leaders heading the creation of a DevSecOps program is to understand
that its success depends highly on your organization’s technical transformation
maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be
introduced to the InfoSec team.
Next, be comfortable with the fact that it’s your role to lead the efforts to integrate
security into DevOps and generate a culture of high trust between the teams. Your
leadership skills can have powerful results, so invest the time to review lean
management practices on software delivery performance. IT leaders usually create a
dedicated transformation team, selecting some champions to accelerate change and
then scaling up their ideas and techniques for the entire team. Still, the challenge is
ultimately up to you.
As a leader, your message should be clear: security teams alone cannot avoid incidents.
Instead, their effort must be concentrated on improving the organization’s ability to
respond to these events. I understand that this is the most challenging part for InfoSec
professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a
distributed and software-defined infrastructure that makes it easy for anyone inside the
team to recreate entire IT environments in multiple regions turned our assets into
ephemeral objects. At the same time, the number of security attacks published daily
announces that this is a war we are clearly losing. We need more soldiers. As James
Wickett once pointed out, there are 100 developers available for every 10 operators,
and only one security professional. Once we do the math, we can see that only by
genuinely engaging development and operations will we take our defenses back.
Value Stream
In high-performance organizations, teams have a superior understanding of the flow of
work value, from the business all the way through to customers. This visibility creates a
sense of awareness about which products and features really matter to the company. If
you already have a DevOps program running, you can recognise the primary object that
we secure: products, platforms, and data that create value to the company goals.
Does it mean that other assets will be managed traditionally and organizations will have
a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing
organizations. Software-defined networking and its security are becoming popular
technologies, augmented by the use of artificial intelligence. Cloud-computing and
distributed offices are also breaking the perimeter concept, and for high-performance
companies, there are no more clearly defined lines between what is inside of the
network or somewhere in the cloud.
A good example for InfoSec professionals is endpoint security. Once upon a time, the
number of antivirus updates was one of the key metrics in our SOC dashboards. The
Covid-19 crisis and the sudden necessity of home office work globally reduced the level
of concerns high-performance organizations had with notebooks and who are using
them. Instead, the core of a DevSecOps strategy became how to make our products and
data resilient to any attack, be it by an internal or external device.
Collaboration
A successful DevSecOps program is all about better collaboration between teams.
Leaders must promote trust between the people involved, and a starting point is to
work on the InfoSec communication style. As a craft that in the mind of many is
associated with the military, InfoSec sees its share of manipulative and coercive
language that sometimes induces fear and blame. On the other hand, DevOps embraces
a more blameless culture, where everyone feels safe to make mistakes. In a
psychologically secure environment, professionals feel confident enough to express
their ideas and take risks, resulting in innovative changes.
Another DevOps best practice revolves around the idea of shared ownership and
common goals. Once it is established that security is now everyone’s responsibility, the
teams must work together to achieve such a goal. They can create and share
mechanisms or tools that help ensure the security of applications and platforms, such as
libraries and authentication and encryption services. With everything available and
easily searchable, it is easier for developers and operators to reuse secure code.
Post-mortem analysis, conducted after a security event happens, must also involve
developers and operators. After identifying the source of the attack, together they can
think about the necessary training and consider actions to prevent it from happening
again in the future, including the action plan on the team’s roadmap.
Shift Left
Shift left means more than making development and operations teams responsible for
securing what they built. It also means moving secure thinking to the earliest possible
point in the development and delivery process.
Security can be included in the product design phases, which can reduce later security
costs. Likewise, scans for insecure code and configuration mistakes can begin inside the
integrated development environment (IDE) software -- an application that provides
comprehensive facilities to computer programmers -- providing fast feedback and
improving code’s quality.
When a security issue is found, it should be centralized and tracked inside a unique
software issue platform. Some issues that may not need an InfoSec specialist to be fixed
can often be quickly solved by a senior DevOps professional, for example. This
lightweight change process accelerates and improves vulnerability fixes.
CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps
between a developer submitting their code and the release of that code into production,
run vulnerabilities assessments in the merge request so developers can fix them
immediately, as shown in Figure 2-1.
Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.
Empowered Teams
To give Dev and Ops ownership for security and thereby achieve better results, we need
to train them to make informed decisions and also make clear that the security
professionals are there to support them to make such choices effectively. Developers
and operations teams already are responsible for many tasks and may need to be
inspired to contribute to security. Otherwise, they may be tempted to merely plug a
scanner inside the CI/CD pipeline and wait for better results.
TIP
Are developers aware of secure development practices such as input validation and data cryptography? Are they able
to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online
community that produces freely available content and tools for web application security. It can be a valuable resource
for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure
Coding Practices" material and the Top 10 Web Application Security Risks list.
To show that security is there for DevOps, you can start by promoting an informal chat
between the two teams. While working at Etsy, Zane Lackey made t-shirts and other
swag and gave them to anyone who approached his team to ask a security question.
“Whether that was they reported a phishing email or a lead architect asking, Hey, can
you have a quick look at the service that we’re actually starting to think about doing?”
says Lackey. In the following weeks, the swag was seen inside the office, and people
kept asking security questions. This practice would basically do 90% of the branding
and cultural awareness work for a security team internally.
Test Automation
It is possible to bring more security to a DevOps pipeline by automating vulnerability
scanners. For code security, in the pre-build phase, we can use static application
security testing (SAST) tools: this is a test we perform in a non-production environment
that inspects an application’s code for coding flaws, back doors, and malware. Open
source tools such as Brakeman and SonarQube are good examples.
We can also scan dependencies at this stage. It is a matter of inventorying all
dependencies of binaries and executables and ensuring that these dependencies, over
which we often have no control, are free from vulnerabilities or malicious binaries. The
OWASP dependency check and the Snyk open source tool are the industry standard for
this test.
After deployment, we resort to dynamic application security testing (DAST) tools:
unlike static tests, DAST consists of assessments performed during execution,
monitoring items such as system memory, functional behavior, response time, and
overall system performance. This test is similar to tests that an attacker would do to
gain improper access to the application. Examples include OWASP ZAP, Arachini, and
AppScan. Some penetration tests can also be performed in an automated way using
tools like Metasploit, Burp Suite, and Nikto.
Infrastructure security tests can also be automated: from container to cloud security, a
tool can be included within the software delivery pipeline, as demonstrated in Figure 2-
2 and explored in more detail in Part 2 of this book.
Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.
Working in Small Batches
In the same way that development and operations slice their work into small pieces,
security tasks should be completed in a week or less. For example, instead of extensive
vulnerability assessments, teams must aim for rapid and effective security fixes. This is
precisely the idea behind the “start little and do often” principle of the Agile threat
modelling: a continuous, timeboxed process to help teams talk about risk and build
security within.
You can then create a user story in the team backlog. The report should be short and
include a description of the possible threats identified. Start small, identifying the
easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce
small victories and allow your team to continue to add new security stories at the
beginning of each new sprint.
TIP
The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for
the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an
architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor.
Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and
create a report with potential risks and mitigation advice.
Once the team has the muscle memory of threat modelling, you can move to find the
highest value security tasks, the ones that will improve the resilience of your systems. In
this way, we decentralize and automate such vulnerability assessments, giving our Dev
and Ops teams fast feedback and the power to take the necessary measures in their
daily routine.
Team Experimentation
Another important guiding principle while adopting DevSecOps is to move from a
culture of fear to one of readiness. Teams must accept two facts: first, their software is
likely to be used in ways they don’t expect and will be exploited by unrelenting bad
actors or automated threats. Second, the security controls they put in place will
eventually fail. Accepting that, DevSecOps teams can actively test their resilience against
attacks to ruggedize their practices and be prepared to respond promptly to any
security incident, improving their systems’ resilience.
This readiness is the main idea behind chaos engineering, a strategy made popular
several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s
infrastructure by continually – and randomly – terminating instances in production to
check how the shutdown would impact business. Chaos engineering has now evolved
into security chaos engineering, and the tools to automate such resilience tests are in
growing development, including commercial products such as Verica and Gremlin.
Chapter 8 discusses this subject in more detail and you can also check out Aaron
Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).
Other fun activities to promote team experimentation include game days, with
development and operations teams playing capture the flag (CTF)-style competitions.
These can be important learning opportunities, especially if scenarios involving the
actual business are outlined.
Finally, teams can experiment with automating incident response activities to reduce
time to mitigate a critical incident, preventing further damage. Instead of using manual
and procedural checklists, there are plenty of open-source tools available such as
TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and
Deep Instinct.
A common misconception about DevSecOps is that InfoSec and DevOps have
irreconcilable goals due to the absence of conventional security controls and processes.
On the one hand, we have the zero-trust principle of never trust, always verify, while on
the other, trust is by far the most critical element of DevOps success. InfoSec
professionals are commonly divided into color-coded groups with well-defined
responsibilities such as red teams dedicated to offensive security, finding and exploiting
possible points of attacks, and blue teams, responsible for defensive security, protecting
the systems against real and perceived threats. Yet DevOps culture seeks to actively
break such barriers and silos. And in a DevSecOps setup, security is everyone’s
responsibility.
This doesn’t mean that organizations that use DevOps don’t have adequate controls and
processes. Instead, security activities are performed at every stage of software
development and delivery, improving its quality and compliance adherence, in a process
of continuous security.
This chapter explores how DevSecOps can bridge the gap between InfoSec and DevOps
cultures and processes, shifting from a reactive approach to a proactive one. It examines
key capabilities that are essential to creating a successful DevSecOps program and to
scale security as digital transformation becomes a reality for every business.
Accelerate: Capabilities to Drive Continuous
Security
Since late 2013, Dr Nicole Forsgren, Jez Humble, and Gene Kim have been conducting
research on measuring and improving software delivery. Their book Accelerate: The
Science of DevOps (IT Revolution Press, 2018) presents the findings and the science
behind their work: 24 key capabilities that influence the outcome of DevOps adoption
and that leaders should invest in to seek higher performance. This section explores ten
such capabilities that I believe every IT leader needs to promote continuous security.
Leadership
The first step for leaders heading the creation of a DevSecOps program is to understand
that its success depends highly on your organization’s technical transformation
maturity level, as we saw in Chapter 1. The DevOps culture in place will merely be
introduced to the InfoSec team.
Next, be comfortable with the fact that it’s your role to lead the efforts to integrate
security into DevOps and generate a culture of high trust between the teams. Your
leadership skills can have powerful results, so invest the time to review lean
management practices on software delivery performance. IT leaders usually create a
dedicated transformation team, selecting some champions to accelerate change and
then scaling up their ideas and techniques for the entire team. Still, the challenge is
ultimately up to you.
As a leader, your message should be clear: security teams alone cannot avoid incidents.
Instead, their effort must be concentrated on improving the organization’s ability to
respond to these events. I understand that this is the most challenging part for InfoSec
professionals since our craft was built upon zero-trust and reduced surface attack. Yet, a
distributed and software-defined infrastructure that makes it easy for anyone inside the
team to recreate entire IT environments in multiple regions turned our assets into
ephemeral objects. At the same time, the number of security attacks published daily
announces that this is a war we are clearly losing. We need more soldiers. As James
Wickett once pointed out, there are 100 developers available for every 10 operators,
and only one security professional. Once we do the math, we can see that only by
genuinely engaging development and operations will we take our defenses back.
Value Stream
In high-performance organizations, teams have a superior understanding of the flow of
work value, from the business all the way through to customers. This visibility creates a
sense of awareness about which products and features really matter to the company. If
you already have a DevOps program running, you can recognise the primary object that
we secure: products, platforms, and data that create value to the company goals.
Does it mean that other assets will be managed traditionally and organizations will have
a dual-path strategy for IT? Perhaps for some time, for medium and low IT performing
organizations. Software-defined networking and its security are becoming popular
technologies, augmented by the use of artificial intelligence. Cloud-computing and
distributed offices are also breaking the perimeter concept, and for high-performance
companies, there are no more clearly defined lines between what is inside of the
network or somewhere in the cloud.
A good example for InfoSec professionals is endpoint security. Once upon a time, the
number of antivirus updates was one of the key metrics in our SOC dashboards. The
Covid-19 crisis and the sudden necessity of home office work globally reduced the level
of concerns high-performance organizations had with notebooks and who are using
them. Instead, the core of a DevSecOps strategy became how to make our products and
data resilient to any attack, be it by an internal or external device.
Collaboration
A successful DevSecOps program is all about better collaboration between teams.
Leaders must promote trust between the people involved, and a starting point is to
work on the InfoSec communication style. As a craft that in the mind of many is
associated with the military, InfoSec sees its share of manipulative and coercive
language that sometimes induces fear and blame. On the other hand, DevOps embraces
a more blameless culture, where everyone feels safe to make mistakes. In a
psychologically secure environment, professionals feel confident enough to express
their ideas and take risks, resulting in innovative changes.
Another DevOps best practice revolves around the idea of shared ownership and
common goals. Once it is established that security is now everyone’s responsibility, the
teams must work together to achieve such a goal. They can create and share
mechanisms or tools that help ensure the security of applications and platforms, such as
libraries and authentication and encryption services. With everything available and
easily searchable, it is easier for developers and operators to reuse secure code.
Post-mortem analysis, conducted after a security event happens, must also involve
developers and operators. After identifying the source of the attack, together they can
think about the necessary training and consider actions to prevent it from happening
again in the future, including the action plan on the team’s roadmap.
Shift Left
Shift left means more than making development and operations teams responsible for
securing what they built. It also means moving secure thinking to the earliest possible
point in the development and delivery process.
Security can be included in the product design phases, which can reduce later security
costs. Likewise, scans for insecure code and configuration mistakes can begin inside the
integrated development environment (IDE) software -- an application that provides
comprehensive facilities to computer programmers -- providing fast feedback and
improving code’s quality.
When a security issue is found, it should be centralized and tracked inside a unique
software issue platform. Some issues that may not need an InfoSec specialist to be fixed
can often be quickly solved by a senior DevOps professional, for example. This
lightweight change process accelerates and improves vulnerability fixes.
CI/CD platforms, such as Gitlab and GitHub, designed to help automate the steps
between a developer submitting their code and the release of that code into production,
run vulnerabilities assessments in the merge request so developers can fix them
immediately, as shown in Figure 2-1.
Figure 2-1. A vulnerability found prompts a notification inside the CI/CD platform.
Empowered Teams
To give Dev and Ops ownership for security and thereby achieve better results, we need
to train them to make informed decisions and also make clear that the security
professionals are there to support them to make such choices effectively. Developers
and operations teams already are responsible for many tasks and may need to be
inspired to contribute to security. Otherwise, they may be tempted to merely plug a
scanner inside the CI/CD pipeline and wait for better results.
TIP
Are developers aware of secure development practices such as input validation and data cryptography? Are they able
to make a threat assessment by themselves? The Open Web Application Security Project (OWASP) is an online
community that produces freely available content and tools for web application security. It can be a valuable resource
for internal training materials, and I highly recommend that everyone on the team become familiar with their "Secure
Coding Practices" material and the Top 10 Web Application Security Risks list.
To show that security is there for DevOps, you can start by promoting an informal chat
between the two teams. While working at Etsy, Zane Lackey made t-shirts and other
swag and gave them to anyone who approached his team to ask a security question.
“Whether that was they reported a phishing email or a lead architect asking, Hey, can
you have a quick look at the service that we’re actually starting to think about doing?”
says Lackey. In the following weeks, the swag was seen inside the office, and people
kept asking security questions. This practice would basically do 90% of the branding
and cultural awareness work for a security team internally.
Test Automation
It is possible to bring more security to a DevOps pipeline by automating vulnerability
scanners. For code security, in the pre-build phase, we can use static application
security testing (SAST) tools: this is a test we perform in a non-production environment
that inspects an application’s code for coding flaws, back doors, and malware. Open
source tools such as Brakeman and SonarQube are good examples.
We can also scan dependencies at this stage. It is a matter of inventorying all
dependencies of binaries and executables and ensuring that these dependencies, over
which we often have no control, are free from vulnerabilities or malicious binaries. The
OWASP dependency check and the Snyk open source tool are the industry standard for
this test.
After deployment, we resort to dynamic application security testing (DAST) tools:
unlike static tests, DAST consists of assessments performed during execution,
monitoring items such as system memory, functional behavior, response time, and
overall system performance. This test is similar to tests that an attacker would do to
gain improper access to the application. Examples include OWASP ZAP, Arachini, and
AppScan. Some penetration tests can also be performed in an automated way using
tools like Metasploit, Burp Suite, and Nikto.
Infrastructure security tests can also be automated: from container to cloud security, a
tool can be included within the software delivery pipeline, as demonstrated in Figure 2-
2 and explored in more detail in Part 2 of this book.
Figure 2-2. A security dashboard provides a list of vulnerabilities found inside the code and infrastructure of the branch.
Working in Small Batches
In the same way that development and operations slice their work into small pieces,
security tasks should be completed in a week or less. For example, instead of extensive
vulnerability assessments, teams must aim for rapid and effective security fixes. This is
precisely the idea behind the “start little and do often” principle of the Agile threat
modelling: a continuous, timeboxed process to help teams talk about risk and build
security within.
You can then create a user story in the team backlog. The report should be short and
include a description of the possible threats identified. Start small, identifying the
easiest attacker paths, such as the lack of two-factor authentication (2FA), to reinforce
small victories and allow your team to continue to add new security stories at the
beginning of each new sprint.
TIP
The company ThoughtWorks created a set of Agile threat modelling resources that includes a workshop slide deck for
the facilitator and printable STRIDE cue cards. The Threagile open-source toolkit allows the modelling of an
architecture with its assets in an agile declarative fashion as a YAML file directly inside the IDE or any YAML editor.
Upon executing the Threagile toolkit, a set of risk rules execute security checks against the architecture model and
create a report with potential risks and mitigation advice.
Once the team has the muscle memory of threat modelling, you can move to find the
highest value security tasks, the ones that will improve the resilience of your systems. In
this way, we decentralize and automate such vulnerability assessments, giving our Dev
and Ops teams fast feedback and the power to take the necessary measures in their
daily routine.
Team Experimentation
Another important guiding principle while adopting DevSecOps is to move from a
culture of fear to one of readiness. Teams must accept two facts: first, their software is
likely to be used in ways they don’t expect and will be exploited by unrelenting bad
actors or automated threats. Second, the security controls they put in place will
eventually fail. Accepting that, DevSecOps teams can actively test their resilience against
attacks to ruggedize their practices and be prepared to respond promptly to any
security incident, improving their systems’ resilience.
This readiness is the main idea behind chaos engineering, a strategy made popular
several years ago by Netflix. It designed the Chaos Monkey tool to test the company’s
infrastructure by continually – and randomly – terminating instances in production to
check how the shutdown would impact business. Chaos engineering has now evolved
into security chaos engineering, and the tools to automate such resilience tests are in
growing development, including commercial products such as Verica and Gremlin.
Chapter 8 discusses this subject in more detail and you can also check out Aaron
Rinehart and Kelly Shortridge’s book, Security Chaos Engineering (O’Reilly, 2020).
Other fun activities to promote team experimentation include game days, with
development and operations teams playing capture the flag (CTF)-style competitions.
These can be important learning opportunities, especially if scenarios involving the
actual business are outlined.
Finally, teams can experiment with automating incident response activities to reduce
time to mitigate a critical incident, preventing further damage. Instead of using manual
and procedural checklists, there are plenty of open-source tools available such as
TheHive and Cyphon, and commercial tools like Darktrace Autonomous AI Antigena and
Deep Instinct.
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
Securing DevOps Tools
Elite and high-performance organizations using DevOps are more likely to take
advantage of cloud computing due to benefits such as cost visibility, fast auto-scaling,
and reliability. Since these organizations are also leading the adoption of DevSecOps, the
two ideas travel together in this book.
Cloud-native security was the most challenging part for me to understand, as the
average software stack has gotten way too complex and distributed to comprehend. If
you are coming to DevSecOps from an InfoSec background like me, in-depth security
modeling can make complicated things more straightforward. Think about cloud-native
security as four layers, also known as the 4Cs:
1. Code security
2. Container security
3. Cluster security
4. Cloud security
NOTE
If you decide to adopt DevSecOps in on-premise infrastructures, just exchange the last C from cloud to computer --
after all, the cloud is just someone else’s computer. You can then apply host security techniques.
In practice, cloud-native tools and environments are way more complex because they
also include microservices, service meshes, and declarative APIs. Thus, practitioners
also model cloud-native security into distinct phases that constitute the application
lifecycle:
Develop:
In this phase, we secure code development and ensure the integrity of the workload
delivered through continual automated scanning using static application security
testing (SAST) and dynamic application security testing (DAST) tools at the merge
requests. Infrastructure as code (IaC) and orchestration manifest controls and
integrations are also tested at this stage.
Deploy :
Here, container images are scanned for vulnerabilities, malware, and other insecure
practices. Upon completion, artifacts are cryptographically signed to ensure integrity
and enforce non-repudiation. Finally, observability and logging are enabled.
Distribute:
Application containers here are deployed into production. A cluster (such as
Kubernetes) orchestrates these containers to assure high availability and extend the
application to hybrid clouds and multi-regions.
Runtime:
Once we’ve secured each component of our application pipeline, we must ensure that
they remain protected while running. Linux kernel technologies such as eBFP allow us
to trace both systems and applications at runtime to detect and prevent attacks.
Why Adopt DevSecOps?
Here are four good reasons to adopt DevSecOps:
1. The current wave of digital transformation made every business a software-
centric company: Therefore application and data security have a critical role
in avoiding breaches. Additionally, data protection laws like the European
Union’s General Data Protection Regulation (GDPR) and the California
Consumer Privacy Act (CCPA) increased the pressure upon companies to
protect their digital assets, which is impossible to guarantee with occasional
vulnerability assessments.
2. It can help reduce costs while improving software delivery quality: To fix
vulnerabilities before an application is placed into production costs a
hundred times less. The price tag also is diminished when your team is
steadily improving security controls rather than responding to incidents that
can presumably impact the business brand and ability to make new deals.
3. DevSecOps reduces cloud-computing complexibility: Along with Agile and
DevOps, the past ten years have seen the rise of the microservice and
containers model where monolithic applications are broken down into
smaller parts that run independently. This breakdown has also impacted
how software is distributed, leading to the increased adoption of public
clouds. And cloud-native applications and data don’t lend themselves to
static perimeters, security policies, and checklists, making the job of InfoSec
professionals even more complex. This is why we need to make our
applications and infrastructures resilient and integrate security at every
stage of their life cycle.
4. DevSecOps is the natural next step: Companies that have reached the highest
level (stage 5) of the DevOps Evolution Model, proposed by Puppet, are also
the ones with the greatest rank of security integration. As Jason Chan, VP of
Security at Netflix, has pointed out: “We’re really trying to minimize the
situations where somebody has to come to ask us to do something. It’s
freedom but also responsibility. I would say a general management
philosophy for Netflix -- and you can even see our CEO talking about it
publicly -- is you really want to distribute decision making as much as you
can. As leading the security team, I want to be making as few decisions as
possible, and the best way to facilitate that is to make sure that people have
context about what’s important to the company. However it is a security
process, mechanism, or tool we’re going to build to support developers.”
Common Misconceptions About DevSecOps
To make what DevSecOps means in this book even clearer, let’s examine some of the
common misunderstandings about the concept:
DevSecOps is just another name for application security:
The automation of application security tests inside a development pipeline is just a part
of a DevSecOps strategy. The process of continuous security inside a CI/CD pipeline also
covers infrastructure security, for example.
DevSecOps is just another name for compliance as code:
Compliance as code seeks to enclose regulatory and security compliance requirements
into configuration files. It is an extension of IaC and, if used standalone, it’s just
automation. For example, take an InfoSec team that adopts compliance as code for its
infrastructure, accelerating some processes. They are not using continuous security, the
automation of protection inside a CI/CD pipeline, nor adopting cultural elements such
as alignment with business objectives or shared responsibilities with developers and
operations.
DevSecOps is just another name for cloud-native security:
Not exactly. You can adopt DevSecOps using on-premise infrastructure, but most top-
level organizations extensively use the cloud, which confuses. I highly recommend you
to give cloud-native environments a chance not only because of their elasticity but also
because multi-region distributed infrastructure reduces security risks and improves
resilience. Chapter 7 further discusses these ideas.
DevSecOps is just another buzzword:
As we’ve seen, DevSecOps results from a set of improvements on software engineering
performance. Perhaps there is no need to label such advancements as Agile, continuous
delivery, or DevOps, but there is value in the use of such buzzwords. If we treat software
engineering as a scientific discipline, these terms can be seen as our field’s scientific
modeling. Models are simplified theoretical constructions of complex concepts and aim
to make it easier to define, understand, and visualize the essentials of scientific
principles. And because all models are reduced reflections of reality, by definition, they
are wrong. Just as the supply and demand economic model can’t forecast every market
activity, we can’t expect DevOps maturity models to mirror all companies’ realities.
However, as the British statistician George Box once put it, although all models are
wrong, some are useful.
Modeling is also a way to amplify such scientific principles. By making it easier to
understand and discuss, complex concepts gradually become commonly accepted
knowledge. Could we think about InfoSec giving some security controls to developers
without the restless series of DevOps events promoting the importance of trust, for
example? Such talks, videos, books, and even memes pave the way to further
advancement. Let’s make peace with buzzwords and evolution models. They are not
intended to be the ultimate truth but simple learning mechanisms.
Elite and high-performance organizations using DevOps are more likely to take
advantage of cloud computing due to benefits such as cost visibility, fast auto-scaling,
and reliability. Since these organizations are also leading the adoption of DevSecOps, the
two ideas travel together in this book.
Cloud-native security was the most challenging part for me to understand, as the
average software stack has gotten way too complex and distributed to comprehend. If
you are coming to DevSecOps from an InfoSec background like me, in-depth security
modeling can make complicated things more straightforward. Think about cloud-native
security as four layers, also known as the 4Cs:
1. Code security
2. Container security
3. Cluster security
4. Cloud security
NOTE
If you decide to adopt DevSecOps in on-premise infrastructures, just exchange the last C from cloud to computer --
after all, the cloud is just someone else’s computer. You can then apply host security techniques.
In practice, cloud-native tools and environments are way more complex because they
also include microservices, service meshes, and declarative APIs. Thus, practitioners
also model cloud-native security into distinct phases that constitute the application
lifecycle:
Develop:
In this phase, we secure code development and ensure the integrity of the workload
delivered through continual automated scanning using static application security
testing (SAST) and dynamic application security testing (DAST) tools at the merge
requests. Infrastructure as code (IaC) and orchestration manifest controls and
integrations are also tested at this stage.
Deploy :
Here, container images are scanned for vulnerabilities, malware, and other insecure
practices. Upon completion, artifacts are cryptographically signed to ensure integrity
and enforce non-repudiation. Finally, observability and logging are enabled.
Distribute:
Application containers here are deployed into production. A cluster (such as
Kubernetes) orchestrates these containers to assure high availability and extend the
application to hybrid clouds and multi-regions.
Runtime:
Once we’ve secured each component of our application pipeline, we must ensure that
they remain protected while running. Linux kernel technologies such as eBFP allow us
to trace both systems and applications at runtime to detect and prevent attacks.
Summary
Agile and DevOps were decisive milestones that paved the way to the maturity of
software engineering, and DevSecOps reflects this evolution. DevSecOps underscores the
importance of a cultural change aiming to integrate security into the rapid-release
cycles typical of modern software application development and delivery.
Continuous security is the technical methodology where security activities are
performed at every stage of the CI/CD pipeline, improving its quality and compliance.
On the other hand, compliance as code seeks to enclose regulatory and security
compliance requirements into configuration files. It is an extension of infrastructure as
code (IaC).
The companies leading the adoption of DevSecOps are also the ones that make high use
of cloud computing, so the basic tools we need to secure usually are code, containers,
clusters, and cloud, also known as the 4Cs.
