Four takeaways from my fireside chat with Matt Hand

As we closed out 2024, I had the opportunity to sit down and chat with Prelude’s Director of Security Research Matt Hand. Matt has over 10 years of experience in red and purple team operations, and wrote the book on endpoint detection and response (EDR) tools, Evading EDR. We discussed his career, the driving forces behind his book, and his work here at Prelude and one thing was clear: the nature of control validation has changed.

The industry has evolved to the point where organizations cannot rely on traditional, expensive, and point-in-time exercises to understand if their controls are working as expected. Our full chat is available here, but for now, here are a few takeaways from our conversation.

1. Purple teaming is an advantageous, but challenging, endeavor

Red teaming has long been a cornerstone of security practices, offering organizations valuable insight into their vulnerabilities. Matt spent 10 years on service teams at SpecterOps, Rapid 7, and Booz Allen running regular exercises. These exercises proved expensive and repetitive, though, often finding more of the same with each regular engagement. As Matt points out, it falls short of preparing organizations for the equally critical tasks of regular detection and response.

Working to fill this gap, Matt co-led the creation of a purple team offering—seeing it as a natural evolution of control validation. It shifts focus from standalone penetration and response exercises to a more integrated approach, testing how well existing and refined detection capabilities function against realistic threats in realistic environments.

While this offering was more productive for clients, it was also found to be far from simple and not something most organizations could feasibly conduct with a services team, let alone in-house. Matt pointed out purple team exercises typically require:

Significant expertise: Teams need to be deeply knowledgeable of both a wide spectrum of offensive tradecraft and defensive tooling. Skill gaps are prohibitive to effective engagements.
Intensive resources: The time and technical infrastructure needed to simulate a more focused scenario can be prohibitive, and often lead to similar gaps of expensive, point-in-time engagements with a services team.
Continuous investment: Unlike red counterparts, purple team exercises aren’t meant to be one-and-done activities. Long-term value requires iterative testing and fine-tuning, which are held up by resource requirements.

With the ultimate goal of his work being control validation, Matt was able to see the superior alternative in purple teaming, but deficiencies remained. Many testing exercises both in services roles and at Prelude revealed similar gaps across organizations—the “easy” things that should be done without question.

2. Foundational security is still non-negotiable

Breach and attack simulations, pentests, or purple team engagements often take the spotlight (and budget) for security teams. There’s very little curb appeal for the basics like making sure your EDR is fully deployed and essential policies are turned on. And yet, Matt points out, these are critical gaps for many teams.

In building control testing and monitoring capabilities at Prelude, Matt and his team uncovered countless organizations with EDR tools running in a passive or “zombie” mode, such as CrowdStrike’s Reduced Functionality Mode. Others with whole suites of devices lacking the control entirely. How do you evade EDR? You go where it isn’t.

There is little point in embarking on expensive testing exercises, (especially those that often test only golden image devices) when the foundation of your security environment is not properly configured. These are where security teams should be making their investments, Matt says.

It’s not a one-time effort, either. Misconfigurations, outdated sensors, new devices, all of these things happen regularly and can go unnoticed in large, dynamic environments. They’re another blind spot that adversaries can exploit without continuous monitoring. As Matt puts it, controls need to be in “fighting shape” before you can effectively evaluate defenses with actionable results.

3. The future of control validation doesn’t include BAS

Traditional breach and attack simulation (BAS) tools, once heralded as the solution for validating security controls, are foundering in relevance. As Matt explains, these tools often prioritize breadth over depth, attempting to tick as many boxes in frameworks like MITRE ATT&CK rather than addressing real-world threats. They are attractive to teams given their ability to effectively expose risk against specific techniques, but are often used to answer whether your controls are simply working, of which they fall short.

This “bingo card” approach creates a false sense of security. Even when tests show coverage for techniques like credential dumping, misconfigurations or control gaps often leave organizations exposed—which is precisely what BAS is not built to identify. The problem isn’t with the tools themselves, but rather that they are the wrong tool to answer the right question: are my controls working as expected?

The future of control validation lies in continuous, automated processes that blend practicality with precision. Rather than focusing solely on expansive testing libraries, modern approaches must prioritize real-world applicability:

Continuous monitoring: Ensuring controls are active, properly configured, and effective over time as environments change—not just during one-off tests.
Integrated workflows: Correlating test results with detection and prevention workflows and automation tools, reducing the need for manual intervention.
Scalable testing: BAS might not be the future, but testing certainly is. The future of testing requires lightweight tools that validate controls at scale, without disrupting operations.

By covering the bases of control validation in regular, routine operations, organizations can lay a much stronger foundation before leaping into testing exercises and platforms that they're not ready to address properly. This shift not only improves defensive readiness but also minimizes resource strain, enabling teams to focus on higher-value activities.

4. Adversaries are evolving with AI, defenders need to as well

It wouldn’t have been a retrospective in 2024 if we didn’t have the chance to talk about artificial intelligence. It’s obviously no secret: if every vendor is leveraging AI to enhance their capabilities, adversaries are doing the same.

AI has reduced the barrier to entry for less-skilled attackers, enabling them to perform more sophisticated actions with less effort. Further, it accelerates the capabilities of advanced threat actors, allowing them to execute attacks faster and obfuscate more complex defenses with greater efficiency. This creates a future where dwell time—once measured in days or hours—could be seconds.

Security teams need to be proactive about their adoption of AI to improve their own capabilities including detection speed and the adaptability and resilience of their defenses. It all tracks back, though, to doing the basic things first and well.

Organizations need to prepare for an era of better equipped adversaries, where volume and sophistication of attacks can and will grow. Preparation begins with a commitment to foundational security. Only when critical controls are deployed fully and optimally can teams commit to AI in an effective and proactive way.

Wrapping up

After 10 years in the industry testing dynamic environments, Matt is confident that control validation has shifted. Defending against improved adversaries starts with continuously validating that the basics—your security baseline—is set. Are your tools deployed and configured correctly?

Only when you can confidently answer these questions can you move on to iterative, robust testing of your detection and response capabilities.

If you want to follow more of the work Matt and his team are doing here at Prelude to help organizations do just that, you can explore some of their recent publications: