Bart Farrell: First things first, who are you, what's your role, and where do you work?
Evan Powell: Hi, I'm Evan Powell. I'm founding CEO of DeepTempo, and I'm a many-time open source technical founder. At DeepTempo, we're building a leader in using AI to protect from AI-powered attacks. I'm looking forward to the conversation.
Bart Farrell: So Evan, banks already have SIEMs, EDR, detection rules, and huge volumes of telemetry. What are they still fundamentally unable to detect that requires a model trained specifically on telemetry?
Evan Powell: Great question. It seems the European Central Bank asked itself a similar question because as many banks above a certain size in the European Union are finding, they have to tell the ECB, what are we doing about AI-powered attacks specifically? And it turns out that when an attacker uses AI, of course they move faster, but they also are more innovative. They have novel attacks, which used to be, well, quite rare, are now becoming quite common. So in short, you have to be able to see today's attacks, which are increasingly hard to see because they are novel. And you've got to be able to respond at machine speed or much more quickly than you did before. That's the short of it. And regulators are noticing, including the ECB, as I mentioned.
Bart Farrell: Now, AI introduces probabilistic behavior into environments where banks want deterministic controls and auditability. How do you let AI accelerate detection and incident response without creating a new source of operational risk?
Evan Powell: It's a great question. What we do is we try to have the generative AI, which is perhaps the least deterministic piece of AI, do what it's good at, which in short is not detecting attacks. You don't want to put your logs into Claude or your locally hosted version of Claude. It will give you only 15 to 20% false positives. You can do a lot better than that with a little bit of Python code. It's going to burn a lot of tokens. It's going to cause a lot of chaos. You want to have a purpose-built model or models or traditional rules, maybe authored by LLMs to detect what is happening. That's upstream. To get to your question, once you're actually operating the reasoning models, they're great at lots of things, but it's your job to control those guardrails and to update those guardrails and to track that. This is where we think an open source approach but some sort of approach to a learning or updated harness that keeps an eye on these reasoning models over time is absolutely crucial that gives you the footprints the logs the OTel telemetry, what is happening here and maybe also just as importantly, it enables you to build trust on the part of your teams and your regulators that this threat hunt was actually done in very similar ways, 64 times over the last week. And we can see that behavior happening. So we can start to trust, document, and really scale and learn from these workflows.
Bart Farrell: And what evidence should a regulated Kubernetes or cloud platform team demand before trusting AI-driven security automation in production? False positive rates, drift testing, explainability, human approval, something else?
Evan Powell: Yes. Those are all great. Very self-serving. I would ask them if they helped automate in a prior life, Netflix or what have you, because obviously at DeepTempo, my background includes having built StackStorm, which is used by kind of the prior generation of folks to truly automate themselves at massive scale. So do you have a background in helping folks get towards high degrees of automation without loss of control? That's really crucial. Transparency. The other is scale. It may not be immediately intuitive, although we're getting the question a lot, but there's essentially a tokenomics way to do a denial of service, right? So let's say you do use a capable reasoning model in your SOC, but you use it somewhat inefficiently. But who really cares? Well, if the attackers continue to increase the volume of their attacks by 10 or 20 or by one measure, 40-fold this year, it's a regulated entity, not a bank, a telco, well, you're going to burn a lot of tokens, right? So I think the other thing in addition to the things you mentioned, Bart, might be in addition to maybe a history of having worked with large organizations as they automate themselves. But another thing might be, give me the token burn, right? So we look at confidence as sort of a Bayesian prior that is then updated during operations. So can you look into the system and have it tell you how confident is it in its efficacy right now? Kind of a basic point, but you'd be surprised. And then secondly, can it project how much it's going to cost and of course, update those priors to do the next thing? And you may find when you do a what-if analysis, what if we get a 10X spike in the attacks, you may find you're going to blow through your economics, right? Which can be concerning. You should have visibility both into how well the system thinks it's going to perform and how much it's going to cost.
Bart Farrell: And with all the things that are going on right now with AI and security, what's the thing that you think people are getting wrong most frequently? What's the biggest mistake people are making when it comes to AI and security right now?
Evan Powell: Maybe there's two things, if you'll allow me. One is the grain of salt when you hear the pronouncements from, if I can say them, call them by their first name, Sam and Dario, you know who we're talking about here, are not taking a sufficiently large grain of salt there. I mean, the reality is these models that they're building are better for attackers than defenders, full stop. They're not good at detecting advanced attacks. And we should just keep that in mind that what is being done by the AI labs is in part marketing. They're trying to insert themselves into your workflows. They can be useful, but for detection, not so much. So that's one thing. And secondly, related to that is the pedal faster sort of thought, which is traditionally in security, there's all sorts of processes that made perfect sense when the attack is perceived in the wild, then I better search for it, then I better detect for it, etc. You can't wait for the attack to be seen in the wild. In fact, there's studies from CrowdStrike and others suggesting the attacks are appearing before they're even reported, right? So you have to, that's a big change, right? You have to flip things. And if you simply pedal faster down the old routines you've had, you will always be behind. So instead, you've got to think about how to be proactive, how to emulate the attacker, how to see attacks you've never seen before, which is Five Eyes, the ECB, NIS2. They all say this, but again and again, security is dominated by a lot of craft, a lot of depth of experience. We need systems thinking and really need to re-examine your priors and maybe figure out a way to get ahead of the attackers. And that's tricky.
Bart Farrell: If people want to get in touch with you, what's the best way to do that?
Evan Powell: Go to deeptempo.ai or epowell101 all over the place, including LinkedIn. I'd love to have the conversations. You can also go to vigilsoc.org, which is a leading open source AI SOC that we started and try it out today and try out our software factory there as well. Get in touch. We're in it together. I look forward to chatting with people.
