Improving DevOps Implementations with Greater Ops Visibility

Ops VisibilityDevOps creates an environment which may be systemically flawed, in part due to a lack of ops visibility into application post-deployment. Ironically, the fundamental, underlying goal of DevOps is to create an environment in which both developers and operators are brought into a collaborative, transparent environment. Cooperative transparency is what gives DevOps its incredible speed and responsiveness. Not only can the code be initially written for optimal performance on the targeted deployment environments, but the deployment environment – which is increasingly an infrastructure-as-code world – can be optimized to support the application based on the app’s architecture. From a get-functional-products-out-the-door-quickly perspective, DevOps is a beautiful solution. However, as often realized, attempting to implement DevOps usually creates a new, albeit larger, functional silo from the two smaller silos of development and operations.

The business case for moving to a DevOps culture is hard to ignore. DevOps practices have repeatedly been shown to deliver improved productivity, allowing organizations to capture greater market share through which they can sustain the increased profitability that DevOps delivers with each project.[1] However, there is more to improving profitability, production, and market share than pumping out more products than the competitor.

Throughout this seven-part series on improving your DevOps implementation, we have looked at how organizations can left-shift security all the way into the design phase and automatically produce immediately actionable threat modeling outputs for fast-paced developers. We have considered how enterprise threat modeling can produce genuine cross-functional collaboration at the speed and scale needed by DevOps. In our most recent article, we examined the benefits of moving QA considerations up-front. Implementing the development-side benefits and outputs of architecturally-based threat modeling through a self-serve model leverages the expertise and experience of security subject matter experts across the entire DevOps project portfolio and keeps pace with every fast-moving sprint. Moreover, it also removes the need for transforming developers into security experts as architecturally-based threat modeling naturally puts security and QA considerations into terms well-understood by developers and easily integrates into their existing workflow and toolset. The next thing needed to improve DevOps is improved ops visibility into the threats of deployed applications.

DevOps Needs Increased Ops Visibility

DevOps is a powerful digital transformation that is here to stay, not because it is fresh or avant-garde. It is not drawing the attention of organizations of all sizes and across multiple sectors because of academic approval or a well-greased PR machine. DevOps is the digital transformation because provides developers and ops visibility into the each other’s bailiwick. The increased visibility it works to produce sustainable business outcomes which yield a competitive advantage with tangible ROI. This success is the direct result of aligning the development and operational IT functions.

Typical DevOps Process

However, much of the discussion, tools, and implementation of DevOps often focuses on producing applications faster, not on promoting after-deployment development or ops visibility into security concerns. The CI/CD pipeline exists explicitly to get the latest product iteration through the deployment finish line. Without disparaging this excellent and valuable outcome of the DevOps transformation, it should be pointed out that stopping here would be shortsighted, hamstringing the full-throttle potential that DevOps could offer.

The “deployment finish line” really is not the end of the SDLC any more than increased production representative of the entire business “big picture.” Once a piece of code completes its rounds through the CI/CD pipeline and reaches the gates of deployment, it may no longer be the concern of developers, but it is just beginning its journey into the full world of the organization’s cyber environment and business life.

Here’s where we can see the shortsightedness of much of the current DevOps efforts: a deployed application must function securely in the deployment environment, not just make it to deployment. This is precisely why ops visibility into the security of deployed applications is crucial to the whole DevOps process. Developers get to move on to their next iteration or project. Primary responsibility for the application is now handed off to administrators and security personnel. The operational environment is the stage where the proverbial rubber finally meets the road, both from a QA and a security perspective – for rarely do threat agents act upon applications in production. Threat agents act upon applications in the operational deployment environment to attack the various business assets. When DevOps is implemented from a get-it-out-the-door-faster perspective, it creates a functional silo that focuses on production which may not we well-aligned with the bigger operational picture or the broader business goals. [2]

Organizations need Greater Ops Visibility into Deployed Applications

While under development, code is tested for QA and security while in isolation. Understandably, it is necessary to make sure the application works in a controlled environment before releasing it into the wild. However, once deployed, the application becomes subject to interactions with other applications, vulnerabilities introduced by 3rd party systems, and threats associated with shared components. It is even possible that the operational deployment environment may have idiosyncrasies which are different from the production test environment. Furthermore, without a doubt, the operational deployment environment is subject to all the creativity, ingenuity and tireless persistence which the organization’s attacker population can bring to bear. Developers get to experience and test the application in a relatively clean, sterile environment. Administrators, on the other hand, are like field surgeons performing open surgery in foxhole conditions during a raging battle. Their world is much more messy and dynamic than the happy place in which the newly deployed application was formed. Without providing ops visibility into each application as handed off from production, it will be difficult, if not impossible, for them to ensure the application functions securely in the operational environment.

When it comes to securing an operational environment, automated tools to detect attacker activity are absolutely necessary – but they are not nearly enough, especially if attackers manage to acquire admin credentials.[3] Proactive management is required.[4] Administrators need more visibility into the security of their systems than what can be provided by logs, monitors, and network scanners. Ops visibility into both what potential attackers could do given the deployment environment, as well as real-time insight as to whether any new threats are relevant – and if so, precisely what applications or components are vulnerable – is what is missing from many DevOps implementations.

Architecturally-Based Threat Modeling provides Ops Visibility

Ops visibility that yields proactive insight and analysis is best produced by the same enterprise threat modeling that enhances developers’ CI/CD pipeline and left-shifts security into the application design phase. The same threat models which application architects can build during the project whiteboarding stage and which provides seamless integration of security into the CI/CD pipeline, as living documents, can follow the applications as they enter the operational deployment environment.

DevSecOps vs DevOps v2 e

Undoubtedly, receiving hundreds of disparate application threat models would prove to be a useless, cumbersome burden for system administrators already stretched thin and would not constructively promote ops visibility. Consider, however, the benefit of having each of those application threat models available as just another architectural component of the deployment environment threat model. The application threat models could then be added to the cyber system big picture with drag-and-drop ease. The disparate collection of individual application threat models could then be combined into a comprehensive threat portfolio or visualized through a “big picture” process flow diagram, which would provide ops visibility into the whole operational deployment environment – whatever that deployment environment might be.

For example, suppose we just readied for deployment – with great proficiency no less – the latest and greatest iteration of our online banking application. The process flow diagram for this application might look something like:

Banking Application Threat Model

Here each use case of the application is shown from a high-level perspective, highlighting the process by which a user would enter the application and move through the application. Handing off the application threat model used by developers will not meet the needs of ops visibility.

The ops personnel are not concerned with the intricacies of each deployed application, per se. They are involved with the secure and functional operations of the IT system from a big picture perspective. The process flow diagram of an on-premises operational deployment environment used by the bank operational personnel might look something like:

Banking OTM wth Nested Endpoint and ATM v2

 

Ops visibility is needed for how the deployed application interacts with and within the deployment environment. In the above process flow diagram, the web application threat model for the recently deployed application was added as a simple architectural component within the Server Cluster group. Note also, though, that any existing threat model could be nested. As shown above, threat models for a standard computing endpoint and a shared SiteMinder component were added with drag-and-drop ease to the bank’s operational threat model. By allowing the nested threat model’s threats to be imported into the operational threat model, ops visibility into the entire attack surface of their on-premises system is increased significantly – thereby enabling them to prioritize and adjust their defensive measures accordingly.

It should be noted, too, that a “big picture.” architecturally-based threat model easily allows administrators and security executives to conduct dynamic “what-if” analysis of planned or deployed compensating controls. By simply adding and configuring a compensating control to the operational process flow diagram – shown above by a couple of on-diagram configurable WAF components – stakeholders can determine the defensive configuration that best balances their security needs with budgetary restrictions.

Gain Ops Visibility into Different Deployment Environments

However, it is conceivable that the same web application could be readied for deployment to a simple cloud-based environment, in which case the deployment environment might be visualized like:

AWS Basic Threat Model v2

In this operational threat model, the same banking web application threat model that was placed in the above on-premises threat model is now placed in the Web Server container running on an AWS EC2 instance.

If the bank application requires greater scalability, the DevOps team may reconfigure it for deployment in a microservices architecture. In that case, the application nested in the operational deployment environment may be visualized as:

AWS Microservices Architecture Threat Model v3

in which each of the functions contained in the VPC are separate cloud-based applications.

Regardless what the operational deployment environment for which applications are prepared, the architecturally-based threat models uses to improve the development-side of the DevOps implementation from design through the CI/CD pipeline continue to serve the interests of the organization by providing ops visibility after application deployment. With clarity into the application within its operational deployment environment, security and operational personnel can understand how unmitigated threats to the application can put the greater cyber ecosystem at risk. Furthermore, threats that may not have been a concern during the application development or which are newly introduced from the wild can be understood – including any downstream impact – across the entire IT system, providing invaluable feedback for the next application iteration.

If DevOps is genuinely to create an IT environment without functional silos, the digital transformation cannot stop at merely removing the dividing wall between developers and deployment. The Ops in DevOps does not stop at deployment. Operational visibility into applications after deployment is needed if an organization’s DevOps digital transformation is to realize its full-throttle potential.

When you want to learn more about how architecturally-based enterprise threat modeling improves your DevOps implementation, click here and schedule a live ThreatModeler demo.


[1] Forsgren, Nicole. “2017 State of DevOps Report.” DevOps Research and Assessment (DORA). Puppet: Portland. 2017.

[2] Koch, Alan S. “DevOps in Action: Ops on Dev Teams.” Project Connections. Emprend, Inc: Redwood City. June 2017.

[3] Thompson, Melissa. “You had an Ongoing Data Breach for Months. How Could you not Know?” Business.com. Purch: Ogden. February 22, 2017.

[4] den Hartog, Marcel. “Boosting the Ops in DevOps for More Application Success.” Wired. CA Technologies: New York. April 6, 2015.