Seeing Red
Why BAS is the Wrong Answer for Control Validation
You are trapped in a simulation.
Security controls are the backbone of your defense—non-negotiable, expensive. We rely on and expect these tools to protect our business. Because of this, knowing if they’re actually working should be as easy as checking your mail.
It isn't.
That’s why many organizations turn to Breach and Attack Simulation (BAS) tools. These tools claim to mimic real-world threats, exposing vulnerabilities and gaps in your defenses.
But, BAS tools merely scratch the surface. Requiring significant resources and relying on pre-defined scenarios—these tools provide a narrow snapshot that fails to answer the critical question: "Are my controls working as expected?" Driven by magic quadrants, buying guides, and marketing speak, we've been living in a simulation that aims to convince you that when you need to know with certainty, BAS is good enough.
It isn't.
It's time to wake up.
.webp)
BAS tools claim to validate controls by simulating attack scenarios, but this approach falls short of true control validation. Rather than assessing the deployment, configuration, and real-world efficacy of controls, BAS tools focus narrowly on point-in-time, predefined attack paths and checking off framework bingo cards like CIS and MITRE.
When it comes time to truly understand if your security investments are working as expected, your answer lies not with BAS, but with continuous security validation.
Why BAS has you
seeing red
.svg){kind=icon}
Painting an incomplete picture of control validation
Above all else, BAS tools fall short in answering the critical question: "Are my controls working as expected?"
In the case of BAS, validation is performed through predefined simulations, which only mimic specific attack scenarios. They are primarily focused on specific controls, namely EDR and reliant on agent installation; ensuring these tools are only deployed on small subsets of machines. Predefined, specific, and subsets, are not representative terms of a dynamic security environment.
Despite their claims of "continuous testing," BAS tools typically offer only periodic snapshots of security performance rather than real-time monitoring. These snapshots quickly become outdated, leaving organizations with an inaccurate understanding of their security posture.
While these tools can provide valuable insights on potential vulnerabilities, their functionality is heavily reliant on the nature of their test library versus the nature of the outcomes or value they create. They focus on ticking boxes on frameworks like MITRE ATT&CK and CIS as if they were bingo cards. While helpful for understanding known attack patterns and tactics, it creates significant limitations when it comes to adapting to the constantly shifting and evolving threat landscape.
Take for example, an output that has you seeing red across the MITRE matrix. This is an incomplete representation of your security environment. How many of those 300+ techniques are actually relevant? How much time will your team invest in turning those reds to green for your next simulation, without knowing if these are representative of real threats to your business?
The outcome then is not just continued exposure to emerging threats, unconventional attack methods, and novel vulnerabilities, but to foundational security gaps: misconfigured policies and missing controls.
Requiring heavy resources despite limited scope
BAS tools often require extensive manual oversight and an agent-based installation, which makes them resource-intensive, time-consuming, and prone to human error. A SOC has enough to deal with when it comes to real threats than to be wondering whether their simulated ones are configured appropriately and providing actionable outcomes.
When it comes to outcomes, teams are further plagued by frequent false positives because their BAS tool relies on simulated attacks. This disconnect between simulation and reality means teams waste valuable time investigating alerts that don’t represent real risks, reducing overall efficiency and trust in their tools and their security posture.
Take our MITRE matrix example, with hundreds of red boxes across the bingo card. The team works tirelessly without clarity into prioritization, to see green across the board in what is likely a testing environment and think: safe.
More likely?
Your team knows these gaps are not a priority for your organization, leading to wasted time and wasted dollars on a tool that got you no closer to an answer of whether your controls are working. When teams cannot rely on their tools to truly automate or augment their actions, then the tool is nothing more than a constraint.
Lacking alignment to the needs and goals of the business
If the goal of your business, not just the security team, is to understand whether expensive security investments are working as expected, the answer should not come with its own significant resource constraints. These challenges can prevent over-capacity teams from effectively addressing the findings of their simulations, leaving critical vulnerabilities unaddressed and reducing the overall value these tools are meant to provide.
At its core, the breach and attack space was built to inject automation and scalability into traditionally manual and high-floor activities like red teaming and purple teaming. Organizations that couldn't spring for a services team or build in-house, could look to a BAS tool to solve for control validation. What they find is additional complexity and limited ability to create the high-value outcomes of red and purple teaming that they aspire to.
All told, BAS tools often fail to integrate with broader business objectives, such as compliance requirements, operational goals, and long-term threat management strategies. This misalignment means organizations lack the strategic insights needed to prioritize resources effectively, address vulnerabilities proactively, and ensure their security tools and efforts support overall business success.
What if there was
a better way?
If the goal of your business, not just the security team, is to understand whether expensive security investments are working as expected, the answer should not come with its own significant resource constraints. These challenges can prevent over-capacity teams from effectively addressing the findings of their simulations, leaving critical vulnerabilities unaddressed and reducing the overall value these tools are meant to provide.
At its core, the breach and attack space was built to inject automation and scalability into traditionally manual and high-floor activities like red teaming and purple teaming. Organizations that couldn't spring for a services team or build in-house, could look to a BAS tool to solve for control validation. What they find is additional complexity and limited ability to create the high-value outcomes of red and purple teaming that they aspire to.
All told, BAS tools often fail to integrate with broader business objectives, such as compliance requirements, operational goals, and long-term threat management strategies. This misalignment means organizations lack the strategic insights needed to prioritize resources effectively, address vulnerabilities proactively, and ensure their security tools and efforts support overall business success.
Maximizing your existing security controls environment
You're investing heavily in best-of-breed security controls. But many function like opaque black boxes, requiring deep, technical expertise to understand if those tools are performing optimally. To answer whether your control are working as expected, you must understand three things:
.svg){kind=icon}
Deployment
Health
.svg){kind=icon}
Configuration
For many teams, the answer of whether controls are working as expected is the right question, being answered by the wrong tool. Where BAS is equipped to emulate techniques and expose response shortcomings, they cannot ascertain whether the controls you’ve purchased are appropriately deployed and configured in such a way to either prevent at scale or be tested to begin with.
Continuous security validation (CSV) tools integrate across your entire security stack (email, network, identity, etc.) and run autonomously, alerting your team as soon as gaps are identified. By capturing insights from your entire environment, your team is more equipped to identify common trends. For instance, the regular OS update pushed by IT that puts hundreds of devices in their EDR's passive mode due to an outdated sensor.
This proactive approach surfaces foundational gaps—such as policy misconfigurations, suboptimal performance, or missing controls—allowing teams to prioritize and address critical vulnerabilities before leaping into advanced testing. These capabilities empower security teams to focus on targeted improvements, ensuring defenses are as functional as they can be before building and iterating on detection and response capabilities.
In many ways, these outcomes feel like some of the easiest things you can do—and they are. You cannot run before you walk, and this level of integration and automation is exactly what modern security strategies require for baseline resilience.
Visualize compliance in-between the audits
Beyond the continuity of business, one of the primary reasons for security decisions is compliance. Ensuring your organization is aligned to regulatory guidelines and can hold up under audit is simply the cost of doing business. And failure to do so can be just as damaging as a breach.
While BAS tools prioritize mapping simulations to specific frameworks, namely MITRE ATT&CK, they often prioritize attack detection versus proper control configuration. Alternatively, CSV tools are built to continuously verify these requirements, like firewall rules for ISO 27001 or that privileged access management (PAM) solutions are correctly configured to prevent unauthorized access to critical systems, a key provision of NIST.
Auditors aren't asking for the latest report from your BAS tool. They want to know if the technical controls you're expected to have are present and operational. You want to understand if those tools actually protect you, which makes testing and BAS a natural progression, but you must first understand the deployment, health, and configuration of your controls.
The argument is not whether to test, but to ensure that your controls are in a position to be tested, while ensuring the right business outcomes are met. Many teams turn to BAS and similar tools to validate controls are working, but this is the right question with the wrong tool. Why test without first understanding how your controls are deployed and how they are configured to respond to these techniques in the first place?
Leveraging CSV, security teams can map their entire control environment to pertinent techniques and threats, operationalizing their threat intelligence to effectively understand and manage their threat exposure. All of this without ever installing an agent or running a test. By first putting controls in fighting shape and addressing priority gaps, you allow for a stronger environment against likely threats and can enable more robust testing operations.
Get the answers every CISO needs
All of this boils down to helping CISOs get the answers they need. Where traditional validation exercises and BAS tools inject workload and complexity into a security environment, continuous security validation clears the way. With easily accessible data on control coverage and configuration, security teams can easily understand whether their controls are working as expected before embarking on more robust exercises.
Such an answer has a natural progression to "Is my security budget being spent effectively?" which is top of mind for any security leader—and the board of directors they report to.
With millions of dollars being spent on controls expected to protect you against the latest threats, security leaders need outcomes that help their team's action to quickly and regularly augment their security. That way, when the board inquires whether their organization is protected against the latest threat on the wire, the CISO can answer with a degree of confidence.
This justifies investments in the people, the processes, and the platforms—all of which go on to build trust in a security team. So when the time comes to run a red or purple team exercise or invest in additional staff, stakeholders have the confidence that resources are well spent. On the contrary, when your CISO is left with another report from your BAS tool seeing red across the board (again)—how do they know where to focus their energy? Their money?
The market agrees, CSV is the future
.png)
Many organizations conduct periodic security configuration reviews against vendor best practices. However, these often fail to account for the organization’s specific exposure to the threat landscape. By 2028, investments in technologies that reduce threat exposure will grow twice as fast as investments in technologies that detect and respond to incidents.
.png)
We deployed Prelude in our environment in roughly 5 minutes (Via API) and within 10 minutes of deploying it and running tests, we were diagnosing configuration drift across our security tools and identifying where we needed to focus both our technical and strategic efforts.
.png)
The authoring agencies recommend continually testing your security program, at scale, in a production environment to ensure optimal performance.
It's time for
real outcomes
not simulated ones
Security teams must evolve beyond the limitations of traditional BAS tools. Rather than relying on predefined threats and narrow tests that leave teams with more questions than answers, it’s time to empower security leaders with actionable insights. Insights that help prioritize their teams and investments, demonstrate alignment with compliance standards, and mitigate organizational risk in a real and meaningful way—not just a simulated one.
.svg){kind=icon}