The next most popular conversation around DevOps has to do with where the bottleneck is currently at its worst, DevOps security. Throughout this series, How to Improve your DevOps Implementation, we have examined, how to improve the throughput by integrating security throughout the CI/CD pipeline and shifting security all the way left into the design whiteboarding stage.
Furthermore, we have seen that by doing so with the right tooling, automation extends well into functions and tasks that are heavily dependent upon a manual application of human intelligence. However, DevOps is much more than just faster application development – hence improving a DevOps implementation also requires shifting security right as far as possible.
We have previously hinted at shifting security right by providing greater Ops visibility into deployed applications and developing data-driven transformational leadership. While these are crucial aspects of shifting security right, there is more to it – we can right shift still further and provide the necessary automation to highly manual processes at the same time.
DevOps security discussions invariably center on automation – especially those areas that are deemed bottlenecks to getting products out the door. 47% of DevOps organizations currently automate their deployment, followed closely by the build process (44%) and releasing (41%).[1]
Currently, the most oft-cited bottleneck in the DevOps security environment is testing / QA [1]. While this function is increasingly being automated, particularly with SAST and DAST tools, it still relies heavily on manual processes. However, we saw that by shifting security all the way left it is possible to generate up-front DevOps quality assurance before any coding takes place. Doing so allows quality to be initially “baked in” as QA personnel will be able to assist developers to understand and code to the project’s acceptance criteria. Furthermore, up-front QA allows QA engineers to ensure that the necessary tests are appropriately set up, further reducing testing lag and improving product throughput.
The exciting thing is this same concept can be applied to shifting security right as far as possible within the DevOps environment.
Shifting DevOps Security Right Incorporates the DBA
If project whiteboarding is the far left of the DevOps environment, then a secure database defines the far right. The database is where the rubber meets the road within the environment.
The application may provide the user interface and functionality demanded by end users. The infrastructure may provide the underlying horsepower to ensure the application runs smoothly for an exceptional end-user experience. However, in a CI/CD environment, these are stateless assets – they are, ideally, continuously undergoing change and improvement. The database, on the other hand – and in particular, the users’ data stored, accessed, and managed therein – is stateful. It must be maintained and kept secure[2] – the goal of shifting security right.
DBAs expend tremendous effort on shifting security right to align the database with development and production.[3] Much of this effort may go into manual code reviews and automation script writing. Frequently, the situation arises when changes to applications require a modification to the database. Most developers do not have a solid understanding of what goes on in a relational database.[4] Thus the DBA may also be called upon to resolve coding issues, post-deployment, without access to the original development team.
For this reason, eight out of ten recognized DevOps thought leaders believe the database is the next bottleneck for DevOps security to overcome[5]. It is also proof that the database is an integral part of the DevOps implementation environment. [6] Iterative application changes must synchronize with database changes if the application is to function correctly in the deployment environment. The application might not fail in the production and testing environment; that database may have limited generic data, and the production DBA can quickly modify it. The deployment environment is a whole different story. That database necessarily contains real data for which the organization must provide all diligence and care – thus the need for shifting security right to include the database as part of securing the rest of the DevOps environment.
Securing the Database cannot be Overlooked or Siloed
Fully integrating DevOps security improves operational performance and positively impacts the organization’s bottom line.[7] However, if the database is part of the DevOps environment, then it, too, needs to be given the same security considerations through shifting security right. Otherwise, all the previous security integration and left-shifting will, ultimately, be for naught for any number of reasons. For example,
- Developers may implement an undocumented “hotfix” as a temporary measure to complete some other coding objective, but later fail to replace it with secure coding as the pressure to deliver the next release takes precedence;
- DBAs may reach “resource exhaustion” and begin to expand their service level agreement, for example, from 48 hours per submitted ticket to 72 hours or taking the full 48 hours to resolve an issue that should only take an hour, thus significantly delaying application deployment;
- DBAs, pressured to keep pace with the rest of the DevOps team, may find they have little or no time for “less pressing” tasks or tasks not directly related to product deployment, such as performing a code review for each change of the database.
Of course, shifting security right to include the database is not easy in fast-paced, iterative, incremental manner. Testing a database requires loading it with data. It is possible – particularly within a cloud environment – to create multiple copies of a database and implement version control, thereby eliminating some of the challenges to integrating the database with the rest of the DevOps CI/CD pipeline. However, this may raise its own set of issues – especially if real data is used in the production testing environment, or where it becomes unwieldy to sync the deployment database with each iteration due to its size and complexity.
Threat Modeling Reduces the Issues with DevOps Databases
In traditional DevOps thinking, automation is a key driver in digital transformation. However useful automation may be, it is not the cure-all for the challenges of DevOps implementation.
Consider QA and testing, currently the primary bottleneck pain point for most organizations with 91% of survey respondents reporting that quality is being sacrificed to some extent for the sake of speed [1]. Automation of the QA and testing function is appreciably ranked with automation of building, releasing, and deploying.
As we discussed in our previous article, though, automating QA and testing is fraught with false positives, the relative slowness of bug fixes, and the challenges associated with identifying and isolating more substantial errors. Furthermore, automated testing cannot efficiently deal with new bugs and vulnerabilities. The tests cannot keep themselves current with changing projects. Nor can they mitigate every issue.
In the same way, automating the database administration and security has some advantages for shifting security right, but it cannot fully address the relevant issues. Intelligent human involvement is required to fully integrate the database administration with the rest of the DevOps environment. It may be possible to create a specific database threat model – just as is traditionally done to improve the security of an application.[8]
However, there is a better way which will more fully integrate with the rest of the DevOps security workflow while entirely shifting security right to cover the deployment database. Just as enterprise threat modeling made a paradigm shift from after-coding QA to up-front QA, the concerns and expertise of the DBA can be incorporated into the application whiteboarding process. The net result provides several outstanding opportunities for improving the DevOps implementation:
- Developers gain much-needed visibility into the database security, configuration, and design considerations before they start coding, thus allowing them to “bake in” proper and secure database interactions and significantly reducing the DBA’s need to review every line of code;
- Potential threats to the database – particularly new relevant threats and data regulatory compliance issues – may be quickly identified and made part of the DevOps teams’ existing issue tracking workflow, thereby reducing or eliminating potential security misconfigurations as the database undergoes modifications relative to application changes;
- DBA’s gain visibility into the application modifications with each CI/CD iteration, allowing the DBA to make timely changes to the database as required, thereby further eliminating causes of deployment bottlenecks.
Shifting Security Right with Enterprise Threat Modeling
Including the DBA’s considerations in the original application threat model during the architect’s whiteboarding exercise is clearly a security left-shift. Enterprise threat modeling, however, also allows for shifting security right as far as possible.
Enterprise threat modeling provides the organization with the ability to see and understand the development environment “big picture.” As an individual application and operational environment, a threat model is completed and continuously modified through each CI/CD iteration, the organization can automatically combine and collate the threat models into a comprehensive, enterprise-wide, portfolio. The automated outputs from this collection allow the organization to conduct in-depth attack surface analysis and provides security executives and other leaders with a high-level dashboard from which they can quantify the organization’s level of security.
DBAs, Ops personnel, and security leaders gain real-time insight into the threats, attackers, compensating controls, data exposure, and probable attack paths relevant to the live, deployed environment. Furthermore, enterprise threat modeling is not a snapshot in time of the deployment environment. It is, instead, a functional window into the cyber ecosystem as it changes. Within this function window, stakeholders can perform dynamic “what-if” scenarios, analyze new potential threats, or understand how changing regulatory considerations impact the deployment environment, each application therein, and the associated databases.
Shifting security right with enterprise threat modeling empowers organizations to fully integrate their databases into the rest of the DevOps security environment.
If you are ready to learn more about shifting security right with enterprise threat modeling, click here to schedule a ThreatModeler demo.
[1] “DevOps Review 2017: Culture, methods, and tools.” Computing. Incisive Media: London. March 2017.
[2] Fal, Mike. “DevOps and the DBA.” Redgate Hub. Red Gate Software Ltd: Cambridge. May 8, 2015.
[3] Yehuda, Yaniv. “Automating the Database: A Win-Win for DBAs and DevOps.” InfoQ. C4Media Inc: New York. December 19. 2016.
[4] Yehuda, Yaniv. “The Role of the Database within DevOps and What it Means of the DBA.” ITProPortal. Future Publishing Limited Quay House: Bath. March 2, 2017.
[5] “The Future of DevOps – 10 Thought Leaders Weigh In.” Datical. Datial, Inc: Austin. October 16, 2017.
[6] Hilbert, Matt. “Where Does the Database Fit into Continuous Delivery?” DevOps.com Mediaops, LLC: Boca Raton. April 3, 2017.
[7] Forsgren, Nicole. “2017 State of DevOps Report.” DevOps Research and Assessment (DORA). Puppet: Portland. 2017.
[8] Williams, Timothy D. “The Value of Threat Models in Enterprise Security Testing of Databse Systems and Services.” Technical Report. Information Security Group. University of London: Surrey. March 4, 2015.