AGI-Pilled Cyber Defense: Automating Digital Forensics w/ Asymmetric Security CEO Alexis Carlier

Hello and welcome back to the Cognitive Revolution. Before getting started today, I want to take a moment to introduce our newest sponsor, Granola. You probably know Granola as a leading AI note taker, but it's in fact much more than that because it works at the operating system level. It can capture all audio in and out of your computer, allowing it to take notes not just on meetings, every podcast you listen to, every video you watch, and if you choose, everything you say right now to help new users make the most of the platform, Granola is featuring AI recipes from entrepreneurial thought leaders, including several past guests of this show. There's a repl.recipe that converts discussion notes to a repl.app build brief, a Bentossel recipe that creates content production plans, and a Dan Shipper recipe that looks across multiple sessions to build an unspoken company culture handbook. My own recipe, which you can try now on Granola in is a blind spot finder. It looks back at recent conversations and attempts to identify things that I am totally missing. This has already proven useful in the context of contingency planning for my son's.

0:00

Cancer treatment, though I am very happy.

1:07

To report that he is still doing extremely well and over time I expect it will become invaluable for suggesting AI topic areas that I've neglected and really ought to explore. For today, my guest is Alexi Carlier, founder and CEO of Asymmetric Security, which just recently came out of Stealth. In response to the launch announcement, Logan Graham, who leads the Red Team at Anthropic, described Alexei as one of the most AGI pilled founders in the space. So naturally I had to find out what that means for the future of cybersecurity. As you'll hear, Alexei's motivation is increasingly familiar, but nevertheless profound. If we assume that AGI is coming and that it represents a near infinite supply of intelligent labor, the question becomes how should we redesign our cyber defenses from the ground up? His answer is to move from a paradigm of reactive emergency triage to one of proactive continuous digital forensics. We begin the conversation by describing the current threat landscape, distinguishing between the spray and pray tactics of financially motivated criminals, the more sophisticated ransomware attacks of cybercrime gangs, and the patient high stakes IP theft operations concern conducted by nation states like China. Alexei also shares fascinating details on the North Korean remote worker phenomenon where state backed actors infiltrate Western tech companies not just to steal secrets, but also to earn salaries that actually fund the regime. From there we turn to how Asymmetric is building AI agents capable of performing the deep investigative work that was previously only available from a very limited number of expensive human experts. We discussed the jagged frontier of current model capabilities in the security domain and why, though off the shelf models can already achieve 90% accuracy on many investigative tasks, Asymmetric is going to market with help from insurance companies with a Services first business model focused on business email compromises, both to ensure that they deliver consistently for customers and to build the proprietary dataset needed to close that final gap, and in performance and reliability. Importantly, Alexei also makes the argument that while most AI technology is inherently dual use, the fact that people who specialize in these investigations don't tend to become particularly outstanding hackers suggests that digital forensics could be a domain that allows us to differentially accelerate defensive AI capabilities. And arguably it's this strategy of intentionally shaping the AI capability frontier that is Alexei's most important contribution, and I would love to see this approach ported to other domains. The opportunity to build specialized data sets, evaluation methods, and training environments that can help harden society's defenses seems both increasingly tractable and urgent. So if you're thinking this way and building, whether in cybersecurity, biosecurity, mental health, or anything else where a DAC strategy could make a big difference, please reach.

1:08

Out and tell me about it.

4:06

With that, I hope you enjoyed this overview of the cybersecurity landscape and and preview of the future of automated Digital forensics with Alexi Carlier of Asymmetric Security.

4:08

Alexi Carlier, Founder at Asymmetric Security welcome to the cognitive Revolution.

4:20

Thanks for having me.

4:25

So yeah, I'm excited about this conversation. An interesting little endorsement of you and your point of view that I saw online on Twitter from Logan Graham at Anthropic, who leads the Red Team efforts there and really kind of is at the intersection of national security concerns and what can models do and you know, has been deep down the rabbit hole of how weird the world might get in the not too distant future. He said about you in response to the announcement of the new company that few people are more AGI pilled than you. So I wanted to start off by just getting a little bit of your worldview. You can tell maybe a little bit of your background too and kind of how you came to be one of the most AGI filled people around. But how did that happen? And even more so, what does it mean to be so AGI built?

4:27

Yeah, I saw this. I suspect what he's getting at is I've taken throughout my career professionally done a lot of stuff that's been really premised on the idea that AI and AGI is going to be a big deal. So a few years ago I was on the early team of the center for the Governance of AI, which is a big AI governance think tank, especially prominent in the UK and US and elsewhere. And this was like in the pre. ChatGPT, I don't know, 2020, 2021 days when AI people were talking about it, but it wasn't really this big thing it is today. And back then we were trying to think about what would happen as we saw that it was going to be a bigger and bigger deal. And there were just very few people doing that at the time. And today, sort of continuing that with asymmetric security, where we have quite a distinctive thing we're doing in the space of cyber defense. And I think because we're taking AGI very seriously, like one frame and what we're doing is we're assuming something like AGI will exist in the world and then be like, okay, what makes most sense from a cyber defense perspective? Assuming that. And that takes you to quite a different place than lots of other security companies. And if we're wrong, the company is. Maybe it's still valuable. It's way less valuable than it would otherwise be. And so these big costly bets on like, yeah, this is a thing that's happening.

5:13

And what does AGI mean to you? Because we've had multiple rounds now, like Tyler Cowan at, at the O3 launch said, okay, I'm calling it, this is AGI. Friend of the show, Dean Ball called it with Claude Code with Opus. And you know, I have to say it's, it's certainly getting there. Right. Where do you think we are on the. Is AGI here or not? If not, what is missing in your perspective that you think is kind of going to tip us into a stranger future?

6:37

Yeah, this is how I think about AGI. It's not here yet. I have in mind more the drop it, remote worker style version of things where you're really fully substituting for remote work that humans do, including for long horizon types of tasks. And you can really fully sub in for humans at the moment. We have this, this weird jagged frontier of capabilities where the models are geniuses at some things and then just cannot string together tasks over long enough horizons. And I think that sort of explains why we have this weird paradox of on some dimensions, clearly they're superhuman, but then they clearly are not having that much economic output just yet. And so I think for me, we will see it in the economic output and GDP Statistics and so forth when we really have this like you're actually subbing it for human work.

7:08

I find the GDP question so confusing because so many of the most impressive moments that I've experienced with AI have been GDP destroying because it has, it's when it's effectively substituted for something that I would have otherwise had to go out and hire done that I'm like this is, you know, this is where I'm really feeling the AGI. And so I have kind of a strange confusion I guess around like more output.

7:54

Sure.

8:21

You know, I would expect more services rendered. Right. More legal services, more you know, more medical second opinions obviously lots more code but potentially at a lot smaller prices than previously. So that's kind of a weird. I don't know if you have a point of view on that, but I still kind of find myself just like, I don't know, the frontier model developers revenue is exploding but we don't quite see it in the GDP statistics. But I'm not even sure what I would expect to see. Like in some ways I might expect to see GDP shrink.

8:21

That's interesting. Yeah. I suspect probably the more relevant thing to be tracking actually is just output and there I think very clearly we should expect a lot more than we're seeing today.

8:53

Yeah, obviously becomes a little tricky how do you measure output? Because we have dollars are the standard way and it's yeah gonna be tricky I guess just cause I've been so deep down this rally like so many. I don't know where your Claude code or open code or Multbot journey is at the moment but I have been you know, like many trying to figure out how I can create leverage for myself. I've done over time like tons of task automation type things where I break down a task and make a little eval set and build a workflow. But that doesn't feel like AGI, that feels like task automation. Now I'm kind of like we're getting definitely a little closer with this Claude code setup. Like it can figure things out in really remarkable diversity of different situations. The one thing I have noted and I'd be interested in your reflections on this is and it feels like it's probably going to be pretty easily fixed. But the one thing I've noticed recently is that the Opus 4.5 in Claude Code has a strong tendency to try to write code with heuristics for things that I really just wanted to spend tokens on to understand with its own fluid intelligence. So a random task was backfilling Transcripts for the podcast onto the website because we didn't do that in the early days of the podcast. And I was like, well, it'd be really be nice to go back and have transcripts for all those episodes. But so many times in the backfilling process I see it write a little Python script with guesses about how things are going to be and then that doesn't quite work. There's too many edge cases, whatever, and I'm just all capping it. Use your fluid intelligence. Just read the file. If you just read the file, you'll know what to do. Don't write these kind of roundabout Python scripts. I definitely think that speaks to jaggedness and it also speaks to in my mind how subtle and kind of honestly minor some of the major weaknesses still are. Like it just has this strong impulse to try to code up heuristic based guesses about how things should work when it's like if you just read it, you're definitely smart enough Claude to know what to do. But it just has kind of some wrong impulses in a few different ways. I mean, maybe we should save your response to that and contextualize it in what you're actually building. But if you have any immediate reactions, I'd be interested to hear them as. Because I do. I mean, Logan commenting that you're one of those AGI filled people out there, I think that is, that is a strong statement, you know, knowing who he spends his time with. So I am interested in kind of your take on how you understand jaggedness and how soon it might be resolved.

9:02

Yeah, I think my, my take here is roughly that RL is a big part of what's going on with the pre training paradigm. You were getting a lot of generality kind of for free because you were reading all the Internet and the models were just improving in this very predictable way. And I think a lot of what's happening now, and I think Karpathi and others have said this as much publicly, is because you're sort of relying on verifiable rewards. The areas where they're getting good are really kind of predictably things like coding and math. And so I actually suspect it'll be until we have some big other paradigm. I don't think it's going to be to get that great at writing poetry, for example, for quite a while. That's generally high level.

11:44

We'll put a pin in that. We'll come back to some of those related questions as we get deeper into the problems you're solving and the way you're solving them. I'd love to take a little survey of cybersecurity broadly and how AI is beginning to impact that and how you expect it to impact it more and more over time. Again, this is an area that really confuses me. So this is a great learning opportunity. On the one hand, I kind of work from the assumption that like everything is hacked all the time. And I've gotten so used to it that I would confess to not my like key I've, you know, privileged passwords or you know, passwords I don't reuse that are like, for key accounts like my Gmail and so on. And then I just have kind of throwaway passwords that I use for like long tail stuff. I'm like, if anybody, you know, compromises any of these services, I don't. I could probably live with that. And I've just gotten to the point where I like click ignore when Chrome pops up that one of my passwords has been found in a breach. So that's the level of security I'm operating with. And I would have to assume that like, you know, it puts me in pretty good company as being certainly exposed in some ways. And yet life mostly goes on and like, not only does like critical infrastructure continue to run and, but even I, with my terrible practices mostly don't have any problems. So I'm a little bit confused about the landscape today, including who are the attackers, what do they want, who are they attacking, how are they attacking as kind of a baseline? And then how is AI starting to change all that? I'd love to just get kind of a survey rundown from you.

12:27

Sure, yeah. I think broadly the majority of attacks are by relatively unsophisticated amateurs or criminals. Maybe this is like roughly 80%, something like that. And often these are financially motivated. And then on the other end you have these very sophisticated attacks from nation states, a small minority from a volume perspective, but often they're going after really critical IP or things like this. And so some of these low sophistication groups will do things like spray and pray attacks, sending out phishing emails to as many targets as possible. There'll be like Script Kitty, so using malicious programs that others more sophisticated folks have come up with, but just sending them that en masse and hoping it works out, that's sort of one end. If you were being targeted yourself, that's probably what you would be seeing. Probably because you don't have a ton of super sensitive IP or stuff that really sophisticated folks are going to come try steal and probably you're just like, you're not going to collect the phishing email and you're fine. And then the other end is, you know, the nation states. And here it's a very different ball game, like a few important actors. Here you have China, a big one, obviously, the ccp, a big focus of theirs is IP theft from R and D heavy industries in the West. AI is almost definitely like a huge focus. Now Russia is another big player. They're interestingly, a bit more focused on political disruption. So famously there was the 2016 US election disruption. And another big one here is actually North Korea, which has very different motivations, actually largely about funding the regime. And a big thing they do here is they have this North Korean remote worker program where they send literal North Korean operators to just get tech jobs as remote workers in foreign companies just to fund the regime. That's a pretty unusual one, but happens a lot.

14:01

I've heard a little bit about that, but I didn't realize it was.

15:45

Yeah, it's a thing. We just brought on someone from, from CrowdStrike who worked some of these cases where the companies in question realized that something was up with their employee and they were both sending back money and selling back various bits of ip.

15:50

So the play is to get the job, get access to either information or some sort of financial tooling that they can then siphon money off.

16:06

Exactly.

16:16

Get the BREX card, get the ramp card asap. Yeah, that's fascinating. Okay, so we got China, we got North Korea, Russia, obviously a player in this as well. Is there a middle? There's also kind of like, who is it? When a hospital, this recently happened to a medical system locally here. I went to see my primary care physician and he was like, ah, it's been a nightmare. You know, our stuff is totally inaccessible and there's some cyber gang somewhere that is demanding ransom or whatever and they don't want to pay it. And so I'm like taking notes on paper. Who's that?

16:17

Yeah, that's one of the most common types of attack, actually is ransomwares. And these are often organized criminals who are moderately sophisticated and act in this coordinated way. In particular, what they tend to do is they find ways of getting access to systems, encrypt it all, and then just put up a notice on the screen being like, hey, if you want your data back, send us a bunch of money. There was a huge pop up during COVID actually, and this economically is one of the most damaging things. Relatively recently, Jaguar had a big ransomware case and if I recall correctly, they needed a $2 billion loan from the UK government after this happened to avoid going under and their operations were down for a month, two months, something like that. So very significant economically. Interestingly, the type of attack they're doing means they operate very differently than nation states where the motivation of the ransomware folks is to be discovered as soon as possible. Once they've encrypted the data, the nation state, more sophisticated guys on the other end are just trying to stay sleuthy for as long as possible. And so you end up with these very different modes of operation.

16:57

Yeah, okay, that's quite interesting. When you talk about like a $2 billion loan, that makes me wonder how much of the, you know, in like business there's always this question of like how much consumer surplus are you creating? How much of the value that you're creating? Are you, you know, as a business able to capture? How much of the damage, I guess, that the cybercriminals are causing do they manage to capture for themselves? Like if Jaguar, if the damage to Jaguar was x billion dollars, that doesn't mean they paid x billion dollars to the hackers, right? Is it like a 10 to 1 ratio, 100 to 1 ratio? Do you have any sense for how much they're actually able to get from the damage they do?

18:07

Yeah, I don't know the specific numbers, but it's incredibly negative. Some, I think order of 10 to 1 sounds right. It's just incredibly chaotically damaging. And I've heard, I've had some economists claim that if you sort out this sort of cybercrime, it's actually like a huge effect on growth because you're just, it's so value destroying.

18:50

Hey. We'll continue our interview in a moment after a word from our sponsors.

19:06

Are you interested in a career in AI policy research? If so, you should know that Gov AI is hiring. Ten years ago, a small group of researchers made a bet that AI was going to change the world. That bet became govai, which is now one of the world's leading organizations studying how to manage the transition to advanced AI AI systems. GOVAI advises governments and companies on how to address tough AI policy questions and produces groundbreaking AI research. GOVAI is now hiring its next cohort of researchers to tackle hard problems that will define AI's role in society. The research scholar position is a one year appointment for talented, ambitious individuals looking to transition into the field. And they're also hiring for research fellows, experienced researchers doing high impact AI policy work. Past scholars and fellows have defined new research directions, published in leading media outlets and journals, done government secondments, gone on to work in leading AI labs, government agencies and research groups, and even launched new organizations. Applications close on February 15th so hurry to Governance AI opportunities. That's Governance AI opportunities or see the link in our show Notes Want to accelerate Software development by 500% meet Blitzy, the only autonomous code generation platform with infinite code context purpose built for large complex enterprise scale code bases. While other AI coding tools provide snippets of code and struggle with context, Blitzy ingests millions of lines of code and orchestrates thousands of agents that reason for hours to map every line level dependency With a complete contextual understanding of your code base. Blitzy is ready to be deployed at the beginning of every sprint, creating a bespoke agent plan and then autonomously generating enterprise grade premium quality code grounded in a deep understanding of your existing code base, services and standards. Blitzy's orchestration layer of cooperative agents thinks for hours to days autonomously planning, building, improving and validating code. It executes spec and test driven development done at the speed of compute. The platform completes more than 80% of the work autonomously, typically weeks to months of work, while providing a clear action plan for the remaining human development used for both large scale feature additions and modernization work. Blitzi is the secret weapon for Fortune 500 companies globally, unlocking 5x engineering velocity and delivering months of engineering work in a matter of days. You can hear directly about blitzi from other Fortune 500 ctos on the modern CTO or CIO classified podcasts, or meet directly with the Blitzi team by visiting blitzi.com that's B L I T Z Y.com schedule a meeting with their AI Solutions consultants to discuss enabling an AI native SDLC in your organization today in.

19:10

Terms of the baseline situation of the attacks that are happening. And maybe you could. I don't know if it makes more sense to segment this by like volume. Probably not. Maybe more like by damages to the degree that we can. How many of these attacks are of the social engineering variety versus finding purely technical exploits where somebody left, I don't know, a port open or you know, whatever the kind of common things are. And how many of them are like these real cinematic type of things where people are really figuring out exploits that nobody knows about? Obviously that's got to be relatively rare, but how should we understand how much damage each of those kinds of things is doing?

22:17

Yeah, I think a bit hard to say in terms of damages, a bit more clear on volume. So the majority is social engineering or phishing. Hard to say exactly, but say roughly 70 to 80%. The reason for this is kind of simple, just attackers choose the path of least resistance and there's no reason to burn, say a valuable zero day if you can just get away with sending a convincing email. And so that's what most people do to start with. And in terms of the technical risk, most actually comes from known issues. So vulnerabilities that people are aware of and just most systems have not been patched. A large fraction of what happens is just people haven't done the basics, they haven't patched what's known. It's like very solvable stuff. Zero days, on the other hand, are a lot more rare and tend to be exploited by the most sophisticated actors, like nation states. But you hear a lot more about them than you would expect based on what you're actually seeing in the wild in terms of what attacks are going on all the time.

22:58

So what works in terms of defense, like turn on your two factor auth, how much does that get you? Update your operating system, how much does that get you? If I do those things, am I good? Or how much risk remains if I do kind of the basic common sense things?

23:53

Yeah, it depends a lot on who good against who. For most people who are not the targets of nation state attacks or aren't big corporations, we're going to get ransomware. Doing the basics like mfa, regularly updating your software, checking your supply chain of vendors, checking their security, having some sort of automated monitoring systems, doing occasional compromise assessments. So proactively looking to a system to see if it's been compromised, that will get you pretty far against lots of these amateurs and some organized crime as well. Frankly, on the other end, it's just extremely difficult to defend against nation states. It requires if you really want to do this in a very determined way, if you vote some ip, that's super important. You need things like extremely strong limitations on software and hardware providers, like you probably just couldn't use most SaaS, for example. You need to be super careful about who your employees are and vet them in a way that's just super outside the norm and plausibly illegal in some contexts. You need to be way, way more stringent on proactive security, like searching for zero days in your system, doing compromise assessments all the time. It's just very difficult to do. And most organizations are just not there, which is why most nation states just have a lot of ongoing intelligence operations that are stealing IP all the time.

24:13

I've been advised I didn't make it this past summer, but I want to go to China and participate in some form of intercivilizational AI safety dialogue and hopefully mutual understanding building. And when I was thinking about going this past summer, people advised me, okay, you can't take your devices. You have to get burner phones. You know, the second you get back you got to throw that phone away. Or you could, you know, maybe sell it on or something. But like, absolutely don't take your devices, don't log into anything while you're there. You know, do all take all these like extraordinary measures? Do you think that's necessary? If I am just a regular American kind of AI yapper taking a trip to China?

25:31

Certainly if you didn't do it, you would probably get people reading over your stuff.

26:16

And by that do you mean like they demand my phone at the airport and take a look at it or that they have a way of getting into an Apple device and the latter gaming.

26:20

Yeah, you just have. You should assume if you do that that people are monitoring your communications in an ongoing way. And maybe that's fine for normal people, maybe that's often fine. I suppose most people would not love the idea of the CCP reading all your stuff. But yeah, I think people generally write that if you, if you go to China you should expect stuff like that to happen, especially someone like you with a public profile and so on.

26:32

And that would continue, like just to understand the architecture of the surveillance. It would be something they would be putting on the device such that when I take the device home that comes with me, not something that, like because I'm on the local network there that they would have access to only while I'm there. Is that the correct understanding?

26:50

Yeah, you should assume they will have some way of getting a persistence mechanism. This might be a physical thing or it just might be they've got access to your login credentials or something like this. But I assume they'll have persistence.

27:10

So that's a little bit like how should I sort of reconcile the idea that I'm not that important for one thing with the idea that there aren't that many zero days to be burned. Presumably Apple I think I should trust in terms of they've got a big team of people that are working on this all the time and identifying these things and closing them down. So how is it that they have so many of these mechanisms that they would use one? I can't be that high on their power rankings of targets. Right. So are there just a Lot more zero days than I have been led to believe. Or how do I resolve that seeming tension where I just don't feel like I should be worth using these things? Because presumably that gives Apple some ability to figure out what's going on and close it down each time they use it.

27:22

Right? Yeah. Apple security is generally pretty good. I think broadly what's going on here is just it's extremely difficult to get visibility on what nation states actually have access to from an offensive security perspective. And the amount of effort in terms of people going into this is estimated, I think in the hundreds of thousands for the ccp. So there's just a lot of man hours going into finding zero days and finding ways to break into systems. And often there are many cases where access that nation states have had weren't discovered publicly until decades later. And so we should just expect that there's a lot of forms of access that really sophisticated organizations like these have and are using all the time. And yeah, it is the case that for most normal people you won't be high on the priority list and so they're not likely to spend these zero days if they expect to be discovered. But certainly it's within the realm of capability and in many cases that probably can just get away with stuff without anyone knowing for a very long time.

28:15

So how do these things tend to come to light? This starts to get a little bit into what you're doing. So I don't know if it would make sense to talk about how people discover things first or maybe just talk about how the landscape is changing in light of AI, because I think pretty much everything we've talked about so far has been baseline. Like you could have said all the same things pre GPT4. Right? So yeah, maybe what's changed since GPT4 class models have come on the scene? How has this landscape started to shift, if at all in a meaningful way? Again, it doesn't feel like it's changed my life yet, but everybody's telling me it's going to.

29:18

Yeah, I think for the most part it hasn't actually had like a huge effect just yet. There's been very, you know, it helps with. Obviously you could automate phishing emails and things like this, but there hasn't been this really big uplift from an offensive perspective yet. I think that's about to change. Or it sounds like on most measurements that the labs and others are doing, the model's already on the precipice of these offensive security capabilities. And broadly, I think the way to think about the effect here is bringing down the threshold of sophistication necessary for any given kind of attack. So attacks that say moderately sophisticated ransomwares assume that a load more people will be able to do these. All of a sudden that's not coming online yet, but will be coming on soon. And from a defensive perspective, I think also hasn't been super transformative yet. And again, I think probably about to change. And that's part of what we're building at Asymmetric.

29:59

So how do we measure these capabilities? It's one of these weird things. I mean, I guess this is becoming increasingly common, right, where it's like across the board. When the tasks were easy and anybody could tell if the AI was doing a good or bad job, things were pretty, pretty easy. We're now in so many domains in a world where very few, you know, because the performance is so good, very few people can even really critique it. I'm staring down the barrel of an interview next week with the founders of Harmonic, who created this Aristotle model system that got the IMO Gold alongside, obviously OpenAI and DeepMind. And it's like, damn, you know, I would score zero on that test. So how do I even understand what's going on? I basically feel the same way in cybersecurity, and it feels like there's a very, it seems like very limited number of data points we can get to measure how good these things are when I know there are some things like, well, there's been some zero days since the training cutoff, so we can see if they can do those. But by senses, those are pretty small numbers. So how do we even get a handle on what the model capabilities are in a way that we can trust or base our plans on?

30:51

Yeah, I think this is super tricky in cyber for a reason that you were sort of gesturing at. One big thing going on is unlike, say, software engineering, where you have a ton of code on the Internet, most cybersecurity stuff is just not public in the context of incident response. For example, when a company gets hacked, they're not going to share the logs from the email getting count publicly. And so it's very tricky to actually get a sense of how exactly the models are performing in these contexts because you've got nothing to benchmark on. And I've talked to some of the folks at the Frontier Labs. This is a big bottleneck right now on trying to evaluate because you just have nothing to benchmark on. You're not seeing all the attacks that are happening out there. So this is a big problem. And secondly also, yeah, the models are getting sufficiently good that you do need a bunch of subject matter experts who themselves are very good on this, good at this. And there aren't just aren't that many such people like Crowdshake, one of the top incident response companies in the world. They have a team of roughly 60 people, it's not huge who can do this really sort of deep forensic, defensive cybersecurity investigations. So you're just really constrained on talent and really constrained on data. And it does make it very hard to turn out what's going on. By default I think, I think there are ways to solve this, but by default things are tricky.

32:13

Hey, we'll continue our interview in a.

33:27

Moment after a word from our sponsors.

33:28

Your IT team wastes half their day on repetitive tickets, password resets, access requests, onboarding, all pulling them away from meaningful work. With Serval, you can cut Help Desk tickets by more than 50% while legacy players are bolting AI onto decades old systems. Servl allows your IT team to describe what they need in plain English and then writes automations in seconds. As someone who does AI consulting for a number of different companies, I've seen firsthand how painful and costly manual provisioning can be. It often takes a week or more before I can start actual work. If only the companies I work with were using servl, I'd be productive from day one. Servl powers the fastest growing companies in the world like Perplexity, Verkada, Merkor and Clay. And Servl guarantees 50% help desk automation by week four of your free pilot. So get your team out of the help desk and back to the work they enjoy. Book your free pilot@servl.com cognitive that's S-E-R-V-A L.com cognitive the worst thing about automation is how often it breaks. You build a structured workflow, carefully map every field from step to step, and it works in testing. But when real data hits or something unexpected happens, the whole thing fails. What started as a time saver is now a fire you have to put out. Tasklet is different. It's an AI agent that runs 24 7. Just describe what you want in plain English. Send a daily briefing, triage support, emails, or update your CRM. And whatever it is, Tasklet figures out how to make it happen. Tasklet connects to more than 3,000 business tools out of the box, plus any API or MCP server. It can even use a computer to handle anything that can't be done. Programmatically unlike ChatGPT, Tasklit actually does the work for you. And unlike traditional automation software, it just works. No flowcharts, no tedious setup, no knowledge silos where only one person understands how it works. Listen to my full interview with tasklit founder and CEO Andrew Lee. Try Tasklet for free at Tasklet AI and use code COGREV to get 50% off your first month of any paid plan. That's code COGREVASKLET AI.

33:31

So I appreciate you taking a long time here to just give me the kind of baseline lay of the land. How are you going to make this better for us so we don't have to worry about this and maybe could start with a little bit of what does it look like today when a company starts to get the inkling that they have been pwned? How does that come to light? What do they do today? And obviously you have a different vision for how that can work in the future. And I think being more thorough, proactive, playing to AI strengths are a big part of that. But take us through the kind of before and after of how things are and how you hope you can change them to be.

35:52

Yeah, so the problem we're trying to solve basically is make it much easier and much more accurate to detect when you've been breached and have that happen as quickly as possible. Today, roughly speaking, you have these monitoring systems, detection systems that are based on static rules, basically software that hard codes, oh, this thing is maybe suspicious. And this kind of works a little bit, it does do something. But you end up in a situation where you get a lot of false positives, a lot of alerts going all the time. And the reason for this basically is that a lot of activity that could be suspicious is also just could be normal. So like if you're logging in from different places or devices. Yeah, that is maybe weird. But also maybe you're just traveling Tibet this week that happens. Maybe you're just up late in the middle of the night and that looks suspicious. But it also could be normal. Maybe you're downloading files in bulk. Again, sometimes you just download files in bulk and without ability to reason over the forensic evidence in a lot more detail, you just have no way of distinguishing between these things. Also, by the way, on the other hand, there's this genuinely suspicious behavior that looks normal. So if you're trying to exfiltrate data, you might just over the course of months exfiltrate very small amounts which are just not very noticeable. And then that typically detection systems at the moment won't pick up on at all. And again, if you had something like a human who suspected there was something going on here and had the time and energy to reason over all this evidence very deeply, you'd probably be able to figure it out. But that's not how this stuff works right now. And so generally status quo. If you have some work going on in say an enterprise, the security operations center that handles all the stuff will do triage. So an initial, basically rapid assessment to figure out when an alert fires fast and quickly, like, is this something worth prioritizing? Is this a real alert? Is it a false positive? Can it be ignored? And then in the minority of cases where they're like, yeah, this is actually a suspicious thing, let's pull out and do digital forensics, which is this methodology of doing very deep security investigations where you're trying to figure out in a lot of depth what exactly happened. Looking at every evidence source you need to, to figure out all the details intuitively. It's just like a detective, like a forensic detective and you're sort of reasoning over all the evidence you need in this and this sort of like judgment based way. So that's the status quo. I think what will happen? You can sort of think in principle what would be ideal here. We know that digital forensics is the sort of best way of doing this if you had sufficient time and sufficient energy and sufficient labor to do this. The reason you can't right now is because you're just bottling on people. There are just very few people with this expertise. They're very expensive and they work slowly and it would just be impossibly expensive to do this. And so at the moment, it's this thing that's most used reactively after a breach that I think will not stay the case going forward. One more general intuition here. I think a lot of people, when they think about what to do with AI, like when you're building AI agents, what should you focus on? Have this frame of, oh, this is like a human workflow that's really important in the world and what if I could just automate that work with AI? And that can work, obviously you're automating legal work. You can do stuff like this. But I think a better question is what would you do if you had 100x the intelligent labor at your disposal? Something like that. And I think the answer is that from many kinds of work, you can deform work that was previously too slow or too expensive. You can perform it at scale. Now, because of the change in economics and that's the bet we're making in the context of defensive security. This domain that is currently super accurate and super costly, only really applied proactively. You could just imagine eventually at the limit, getting this down to the cost of compute and you just have these sort of close to continuous proactive assessments of systems in a very deep way. And that's what we think will happen eventually. The question is how quickly can we get there and can we get there soon enough that we can deal with a lot of the security risks that are going to come on with AGI and offensive security changing and that kind of thing?

36:28

Yeah, I like the frame of, okay, you could take usually for guidance, usually say you can probably save 90% time and money on some task that's being done by humans if you automate it with AI. But yeah, I think you're totally right to emphasize that the other side of that coin is what can you scale that was just previously impossible to scale? And this is an interesting one.

40:44

Can you describe a little bit more.

41:14

What the humans do? So let's say I'm breached, that he set the scene in terms of who your ideal customer profile is and how it comes to their attention that they've got a problem and then who they go to and how long it takes these people to engage. And then if I just were to sit there and watch what they do, what are they doing? Are they just like grepping through logs kind of using the intuition they've built up over time or what is that sort of story? And then again we'll. We'll switch to the future state story next.

41:15

Yeah, so to make this concrete, so we're currently focused on what's called business email compromised more and more intuitively, just email based cyber attacks. And broadly what this is or like a common modality here is an attacker somehow got access to an email account. Maybe they've sent out a phishing email that someone's clicked on. Maybe they've got access to your credentials some other way and they're trying to use that access to trick people into doing stuff like sending them information or sending them funds. So this can often is financially motivated, but can also be quite different. So like when the DNC was hacked by Russia in 2016 was an email based attack. And so asymmetric. So we come in and we respond to these attacks as do other folks, cybersecurity folks. And what this generally looks like is okay, you come in and you pull down all the email logs from an environment. So the enterprise has been Hacked. They might be using Microsoft, they might be using Google. And you look through the logs, and first of all, you're trying to figure out how did they get access, when did they first get access, did they then escalate their access in various ways? Did they gain access to another account? Something like that. And very concretely, it's like looking through email logs and trying to be like, okay, this is a weird location to be logging in from. And maybe the user was in two places at the same time. That's impossible. So this suggests something is suspicious here. So you figure out how they got in. Then you figure, okay, what did they do? You're again looking through the email logs. Be like, okay, did they send emails? Did they read them, did they delete them? Often people will set up automatic inbox rules. So automatically forwarding emails, automatically deleting invalid emails. Are they looking at files in the Google Drive? So you're painting this very comprehensive picture of what happened and also whether the attacker is still in the network. And then finally you're looking at, okay, how did this happen? Was it a phishing email? And at this point, you also sometimes look at the email bodies themselves. And so you have a pretty clear sequence of things you're trying to do, but you're doing this in this sort of very flexible human way of like, okay, this looks kind of suspicious. This looks funny. Let me pivot off this bit of evidence. Let me think what comes next. And so it's been hard up until very recently to actually automate this in a meaningful way.

41:51

How much does access to, like, if you're running your own enterprise email server versus you're like a Google customer and your employees have Gmail, how much does that change how this goes down? If you're a Gmail customer, you obviously, or I would presume, you can't just get that level of access, right? So do they have a team that you work with to try to resolve that sort of stuff? Or how does it work when you've got like a SaaS provider, like a.

44:03

Gmail, So most tenants, like a user tenant, will have a bunch of logs that you just use. So actually they do have all the logs you need. You just get admin privilege to those accounts and you can go from there. The provider does actually matter a lot. So, for example, there are far fewer Google compromises of emails than Microsoft, but I think at least an order of magnitude. So big differences based on how they've configured the environments.

44:34

Do we know what causes that? If it's that big of A difference. Why has Microsoft not managed to close that gap?

45:00

Yeah, I can't recall the specific reason, but it is this very tractable thing they could be doing that have not been doing for reasons that are kind of baffling to me. I guess it just doesn't really affect them in this big way. But yeah, I know some folks who have considered working in the space and building various bits of tech to help deal with these attacks and were like, now probably we shouldn't do this because Microsoft's going to solve it one day. But yeah, up until now it's not been the case. Yeah, that's funny.

45:07

Okay, so how do we get AI to do this? And where are we right now in terms of how good is Claude code and where does it fall short and what are you building to make sure it actually works?

45:36

Yeah, so I can tell you a bit about what we're building in this context. We are a full stack AI digital forensics and incident response company where the mission is to accelerate AI cyber defense in particular by trying to automate this field of digital forensics as quickly as possible. And we're doing this broadly by having these human AI teams doing these kinds of investigations. And so we're currently focused on these email investigations. And so concretely what this looks like is we go in and do the kind of things I was just talking to you about. But on this AI platform we've built that ingests all the logs broadly, we have an agent that will do a first pass analysis over these logs and then the human investigators will click through and basically try to do something like a QC most of the time to just check the quality of the reasoning. And in a minority of cases, they do need to meaningfully change what's been done. And so based on what we're seeing in these initial cases and for context, these email investigations are on the much simpler end of investigations. The models are pretty good already. Even without doing anything fancy, without training on specific data, without anything like this, you can get maybe something like 90% accuracy, something of that order of magnitude out the box. That is super helpful for speeding up the process of this investigation. It's totally insufficient for actually automating this work. This is a context in which you need very high accuracy for various reasons, which means you do completely need the humans and you do like. It's also an area where the nines of reliability just matter a lot. So for the foreseeable future, and certainly as we get to more complicated kinds of investigations, it does feel to me like this would be an area where there's this long tail of needing humans to be around to really push up the nines.

45:51

Okay, that's really an interesting kind of possibly, you know, production possibility frontier that I'd like to understand better. But just as a little context, the I think that, you know, this is going to play out, I suspect in very different ways in different domains. And I'm always kind of like, well, geez, if AI can do 90% of the work, how much does that leave for humans? Is it 10% or is it still like can they finish it in 10% of the time that they would have had to spend? Or do they still have to spend half the time that they would have had to spend? Even though in some sense 90% is done, you can imagine that last 10% taking longer. And then of course there's also the question of how many nines do you really need or can you even measure in a given context? And then there's the question on top of that of like how much latent demand is there for different kinds of services depending on how much the overall price can come down based on how much can be automated and how much more productive the humans can be and whether or not they can hit key reliability thresholds. So my usual kind of somewhat tongue in cheek way of saying this is like on the one end we have dentistry, where I do not want any more dentistry, no matter how cheap it gets. I want the minimum, ideally zero. And it's never something I want and making it cheaper doesn't really entice me. On the far end you might put something like massages, where I potentially get a massage every day if it was close to free. I have limited. I mean, I guess one of the big bets you have is that there is a ton more demand for this than is currently served. How would think about the thresholds that matter in terms of the ratio of human productivity that would then enable subject to certain reliability thresholds that would then enable the vast explosion of this market that you're obviously trying to unlock.

47:34

Yeah, so I think a couple of ways to think about it from the perspective just applied reactively in the context of incident response that I think is a lot is not of the form that the demand will hugely grow here. Or rather it's not of the form. There is a bunch of latent demand. I do think demand will grow because there'll be a bunch of AI enabled cyber attacks, but it's not of the form. People, if they could have more of this, they would want more it's just like after you've been, you know, you're bleeding, you're hacking, you need to deal with it. That's basically the demand. On the other hand, if you get to this paradigm where you're using this sort of investigative reasoning but applying it proactively, then I think it sort of ends up substituting for the current approaches to doing detection in cybersecurity, which are not at all based on this digital forensics type approach. And it's a much, much bigger market on the order of a few hundred billion, something like this, maybe 500 billion. And so I think it's like a bottom line here. At minimum, you're substituting for this existing huge market of detection. Then there's additional question here that I think is sort of an open question of if it becomes really cheap to get high levels of security much better than what you currently have in detection, what is the demand for that? That I think is more uncertain and depends probably a lot on the type of actor. There'll be some organizations where it's just, you're getting what you need from security and it's like a meet the minimum bar thing. Then there'll be other organizations, say like AI labs who are trying to protect really sensitive IP or national security agencies and governments who have this much higher demand for very high levels of security. And I think those are the areas we should expect this additional demand above and beyond just substituting for the current approaches to detection.

49:41

So how do we get there?

51:23

It's a great question. So this actually relates to something you brought up before, around the difficulty of measurement in this space. And I mentioned these two bottlenecks around actually having access to data or context on what's actually happening, what these incidents, what these cyber attacks look like, and then having the people that can actually assess performance at the moment, the status quo is that it's very difficult. And we've talked to many of the folks at the Frontier companies. It's very difficult to make the models better in these situations because you don't have these two things. It's hard to build like verifiable rewards for reinforcement learning without having a lot more color on what's actually doing what's actually going on, like on the ground in cyber and without having people who can assess these things. And so I think the key to really accelerating progress in this space is solving both of those two things. And I think broadly the approach we are taking to solve this is actually having this whole cybersecurity team doing these investigations all the Time. So we've just hired folks from Crowdshake, from Palo Alto Networks, from the biggest cybersecurity companies whose day job is really just to do these investigations with the AI tooling. And as they're doing this, they're implicitly evaluating the model performance based on the use of the tooling. And then secondly, I think the tricky thing is you need to really just be seeing a lot of the different types of cyber attacks that are happening and off the back of that, building evaluations that are really high fidelity, really realistic building environments that are really realistic and that you can use to train on. And I think this just doesn't happen by default. And by default the model providers I think will lag on these dimensions. None of the big cyber companies actually are either pushing on this either. It's not like a huge, they don't have a huge immediate incentive to do this. So unfortunately, I think the default is just this stuff lags behind in a way that doesn't apply as much to offensive security, by the way, because from an offensive perspective you can just try hack something. You can just do that all the time. You don't need any sensitive data. So there is this asymmetry. Exactly. So there is this unfortunate situation where you can like a lot of bottlenecks to improving the defensive side a lot less apply to the offensive side. And I think what the space needs is basically a lot more companies trying to solve this data verifiability evaluation problem. We're doing one attempt, but I would love there to be many more.

51:28

So how do you position yourself in the market to. Because I can sort of see if you become the go to company that everybody knows is going to do the best job with this, then as we see in, I mean, this is kind of a general phenomenon in the AI space, right? Like the more of the share of the business you win, then you have more access to data and you get kind of a positive flywheel going. So I can see how that dynamic could work once the flywheel starts to turn. But it does strike me that it's a probably a pretty hard market to enter. So how are you thinking about entering it as a young company that obviously doesn't have the, you know, the track record, the sort of the old adage of nobody got fired for going with IBM, right. Like I assume there's probably something similar in cybersecurity where it's like, well, nobody got fired for going with crowdstrike. How do you overcome the just relative unknown factor and win business so that you can Start to accumulate this data and get that flywheel turning.

53:52

Yeah, the flywheel is I think actually especially interesting in cyber. And Crowdshake's an interesting example where about, I don't know, 15 years ago, whenever they got started, they actually also started as an incident response service provider initially before they productized a couple of years later. And so for the first two years they were just doing this kind of services. The reason for that was they could solve this problem of understanding what cyber attackers are doing all the time and collect this sort of data and secondly build these relationships with enterprises who they were serving in cybersecurity. Trust is really important for distribution because it's hard to assess how all the tools are working. So for most, I don't know, if you've in a bit of HR software, you can just tell there's a thing like does the thing pay my employees on time? In cybersecurity you're sort of taking it on trust, like, yeah, is this thing stopping attacks? Am I not attacked or do I just not know about it? So trust is a bigger thing. And so they figured out that there was this really strong flywheel both from data and distribution, from doing services. And this has remained the case since then. I think what's different now is that up until quite recently it's become possible to get AI enabled services to work really well and get these much better margins than you would otherwise. And I think this is what's creating this new opportunity in the space. So we are initially focusing on these email based attacks where we have built out this platform that basically takes the time required to respond to these attacks down from depends on the firm, but on the order of two days to a week to these investigations to a few hours. So a very meaningful difference here. And this means that we can just do these things much quicker and much more cheaply. And that is proving sufficiently compelling to get a bunch of folks to trust us. And from that initial trust it'll be much easier to get a lot of different types of cases here. So I think the opportunity is coming is again available because of this technological shift where in this context incumbents are just not adopting as quickly as startups.

54:57

And is that just the purely sociological phenomenon that we see in many places of incentives aren't quite there? Who wants to take a risk? Maybe they're billing by the hour or whatever in the first place.

57:02

The sum of all of that billing by the hour is part of it. Another piece that's more subtle is on the face of it, it doesn't look necessarily like this massive opportunity. Like it's a pretty big market. It's a, if I recall correctly, it's like a $40 billion market of this sort of services. But from the perspective of say, CrowdStrike, who is what? I, I don't know, close to a $200 billion company, I think, and the majority of their revenues are coming from their detection products, which are not these sort of digital forensics types of services. It's like relatively small fish. And so it only becomes this very compelling, you know, this comparatively compelling thing to like this other big piece. If you have this view that, yeah, okay, we're actually going to get to the point where we're meaningfully automating all this field and then this will change actually, not just how these forensic investigations are working, but how detection is working. And that requires you to really take AGI seriously and to really be like, yeah, okay, this work that at the moment is just entirely humans is in pretty short order going to be automated in a meaningful way and will change how all these other bits of cybersecurity works. And most of the players in the space, I just don't have that belief in general. And I've spoken to some of the folks who I know who work at the intersection of AI and security. The security people generally don't. They're by nature skeptical people and they don't really believe in AI in this big way. And so I think part of the opportunity is just like, do you really believe in AGI and have thought through the implications of that? And I think most of the big.

57:17

Players are, yeah, fascinating. Okay, can you tell a little bit more about the jaggedness of models? And I guess I'm also interested in. Of course we've got the age old debate of proprietary API models, which I would presume are the best at these tasks, off the shelf. But then you've also got, if you're using open weights, then you have the ability of course to fine tune or modify however you might like. What's the mix, what's the model mix look like and how much do you think that this is going to be? Like, how much value is in the harness versus how much is in the training data that you can create and how much of. I guess there's also another question of like training data can be monetized or can be commercialized in multiple ways. One is you could sell it back to the model providers and I think that's from what I hear increasingly a very good business. Or you could try to make your own models that outcompete them. So you have that asset. How would you describe the kind of what's working today and what is your strategy to not get steamrolled by just better and better frontier models kind of winning everything?

58:47

Yeah. On the jaggedness, I think there was actually this sort of underrated opportunity here to. To defensively and intentionally accelerate capabilities in various ways. So one intuition pump here in the context of alignment. For a long time, AI alignment people have talked about the automated AI researcher being one important part of the way we're going to solve alignment. There hasn't really been similar analogous things in other fields. And I think part of the reason was just with pre training and this very strong generalization, it was hard to see how you could accelerate in this intentional way rather than just across the board. That I think with the jaggedness we're seeing is no longer the case. And the implication here, I think, is that actually you can go out and pick various areas that you think it might be important to harden the world, like defensive cybersecurity or biodefense or bits of ash, for example, and you could very intentionally curate the data sets, the environments, the evals that you need to sort of pull out the jagged frontier in this specific direction. And people don't seem to talk about this very much. I think this is a mistake. It seems like this really huge opportunity. I mean, for many of the same reasons that AI is going to be so transformative in the first place. If you're a subject matter expert in one of these areas and instead of doing bits of work yourself, you can just put this expertise into AI systems and then scale it immensely more. That's like an incredible thing you could be doing. And yeah, it strikes me that very few people are doing this. And a few reasons, I think in some contexts there's like, in the context of cybersecurity, for example, people often have this idea that actually it's just inherently dual use. That's, I think, partly true, but also overstated. Like I saw Sam Ullman tweet this the other day. There are some areas like pen testing and vulnerability discovery that are in fact, yeah, they're just dual use. Like you can patch holes or you can exploit them, but this is not if cyber is a best field. And this is not true of all areas in cybersecurity. And so digital forensics, for example, I think is very much not like this, where fundamentally what you're doing is trying to detect whether stuff is broken into a system. You're asking is it evidence that an attacker is here rather than can I break into this? And so that's just like it's a defensive application. And I think there are other areas of cybersecurity too that of this feature. And so generally I think it'd be good if there was much more of this kind of stuff out there. On your other question around the models we're using, how much is the scaffold? All that kind of stuff Generally the way I sort of think about this, at least from how we as a company should be going about this first we should just see what we can get the learning fruit of the box. Pick the learning fruit. We've tested the different models, see how they go build minimal scaffold, see where the performance is, make sure we have sophisticated evals that are working well here and that we can really tell what's working. And then only then consider doing some more fancy stuff like training your own models or things like this. At the moment we are at the eval stage of that and on these simple incidents that we're currently working on, just models out of the box with some scaffold is actually getting pretty far. Like again, we're not so far down the lines of reliability, but it's actually unclear how much more effort is worth from us to get them good at these specific email based attacks going forward on these other kind of incidents that are way more open ended, long horizon, I'd be pretty shocked if the models were as good or anywhere near as good. And I think that's an area where all this differentiated access to data that we use as blueprints for evals and so on and the reasoning choices of analysts using the platform that I think will become this very differentiated and valuable asset.

1:00:04

And do you have an intuition at this point as to whether you license that data back to model developers or keep it for yourself to try to make your own specialist model?

1:03:50

Yeah, both of possible. And you could also imagine doing this sort of iteratively where you have some stuff that you've done in the past that you're then sharing. I think the benefits of sharing, at least from a mission perspective, are pretty significant in that if you're getting the foundation models better at these tasks, you then get this immediate insane distribution where you're deployed across huge parts of the world, you're naturally deployed across AI labs, across governments, some of the most important areas from a security perspective. And also, yeah, they will be big contracts and relative to other data providers, just the fact we're seeing all the cyber security Incidents all the time does make it quite differentiated. On the other hand, is it giving away some sort of proprietary thing? Possibly, but yeah, I think it's sort of an open question. But there are certainly big benefits to sharing. Not the data per se, I would say, because that's delicate, but sharing the sort of the evaluations drawing on that.

1:04:03

Going back to that question of generalization, I mean, I do think the. Obviously the jagged frontier is a major factor affecting all of us that are trying to use AI to do stuff on a daily basis. And the question of how well RL generalizes seems like a pretty big question for what the next couple years are going to look like. What would you say is the case for your specialists? Are they? I mean, I definitely take the point that it's very different to grep through logs versus try to break in in the first place, but I then would also kind of guess that the people that you have doing the incident response would probably be very good at breaking in. Right. They would know all the tricks of how to make a phishing email compelling or whatever. Right. It would seem that if you had tried to trained as a human, at least on the response, you would be maybe not the most elite on the attack side, but I would expect significant generalization. So how do you see that playing out at the human level? And is there not reason to expect that a similar kind of generalization could happen for models?

1:05:00

Yes, at the human level actually, surprisingly this is not the case where this digital forensics field is actually a pretty distinct thing within security and the people who do it are very different communities to the offensive white hat hackers, things like this. And so at least at the human level, there's not a ton of generalization. Interestingly yeah. And I suspect there is just actually relatively limited overlap in terms of the skill sets here. And I think this would also apply to models. Again, of course at some point we're going to self continue learning and the models will just get good across the board very quickly. But until we're in that regime, I think there is this opportunity.

1:06:18

Okay, cool. That's quite interesting. Do you have other things in mind that you think are like that we can get out of our domain pretty quick here, but is there a biosecurity analogy? Is there something that we could be differentially accelerating in biosecurity hardening that wouldn't lead to the same kind of advances in the offensive skill set? Or is this perhaps something that's like relatively idiosyncratic just to the way the cyber world happens? To be structured.

1:06:59

Yeah. I will caveat that I don't know bio very well, but you could imagine obviously you'd need robotics for this. But just producing a bunch of masks and having those ready to go, like stockpiling masks, stockpiling vaccines, certainly they seem like there are at least some like the mask example, I don't know to the extent to which the biological research is dual use. And then on, let's say what are other areas. I would love, by the way, for someone to just go through and catalog all a bunch of important domains and try to figure this out. I can imagine the context of AI safety. Maybe there are specific things here. I'm not sure, but it seems like an important thing for folks to go figure out.

1:07:35

Sorry. I recently did an episode with a couple pioneers in the formal methods space and obviously there's multiple. There's a general agreement that cybersecurity could become a big problem in the AI future. And then there's several flavors of ways that we might respond to this. The formal methods angle, I mean it doesn't really necessarily address email based hacks, but you could imagine a lot of things could get a lot better if the rate of vulnerabilities is just dramatically reduced. Based on the vision that I understand that they have, is using formal methods as a reward signal to train coding models such that it's not just that you're taking output from today's models and trying to verify them, but you're also really closing that loop and getting to the point where models should be writing superhumanly secure code by default in a super majority of cases. That seems very plausible that we will get there because it seems like the viability of the formal methods, I mean that's obviously its core strength. That flywheel should be pretty easy to get turning. How do you think about where you want to be in the cyber security? If it's a defense in depth type of thing where you can harden your outermost defenses and then you can have better forensics when things do get through. In the extreme limit, if the outer defenses are totally impenetrable, then the forensics becomes less valuable or again, in the extreme, unnecessary. How do you think about that? Is that just never going to happen in your mind or what?

1:08:16

Yeah, I think mohatting would be great. People should totally do that. But I'd be very surprised if it would be sufficient. A few reasons. One is just for very high levels of defense against say nation states, it's just extremely difficult to defend. And it's not just like Having no code vulnerabilities is sufficient. There are other ways you can get in and the strong assumption should be stuff is going to get in. Additionally, regarding these other forms of ways of getting in, generally to be very, very secure you need to do these things that trade off with your productivity as an organization. So say, I don't know, say you're in, you're an AI lab and you have this way of interacting with the model weights that restricts the output. That's kind of annoying, but it does predict against weight exfiltration and things like this. And so in general you do have this trade off with productivity that is hard to get around in many areas of security. And so that's the thing that's generally going to stay there. A nice feature of automated forensics or detection is that it doesn't really have this feature. You can just sort of run it in the background and it does increase your defensiveness. And so generally I think of these things as substitutes. It would be great if we could get this full in theory, full hardening. I don't think that's at all realistic. And so in general they're just like the more of each you can get, the better.

1:10:13

So wait, substitutes or complements?

1:11:37

I guess maybe both. Well they substitute in the sense that if you have more better detection you need less hardening for like a given level of security. But also having, yeah, having both together makes you more secure.

1:11:41

Is there. I've not actually done an episode with these guys yet, but there's a company called Jericho Security. I just talked to them offline a while back, but I have their website up next Generational security training trusted by the US government. Smarter cybersecurity training starts here. Jericho Security is designed to protect you against today's most advanced email, SMS and deepfake threats. And basically what they do is automate the phishing attacks, more or less spear phishing attacks to see like who in your organization is clicking on these links and make sure everybody's on their toes. And I don't know that there's to my. Not when I spoke to them there wasn't like a, you know, a deep forensics. They you, when you click on the link they take you to a sort of we got you page that's like hey, wise up dude. Like you just clicked on a bad link and here's how you should have known, you know and how you can know next time. But I could imagine that you know, in terms of like environment creation or something like that, there could be some interesting collaborations between A company that is systematically testing the, for finding the soft spots in the, in the human defenses or lack thereof. And what you guys are doing under the hood. Are there other, how do you go to market? I mean, do you have like partnerships or people that bring you in? Are there other like alliances that you have that, that allow you to get into the room? Because obviously people need to, this is like crisis stuff. Right? So they need to like either find you real quick when they're searching or they kind of need to have known you or they need somebody to give a trusted referral. What does that look like?

1:11:52

Yeah, the go to market is actually kind of interesting. It's via, for the most part, insurance companies, actually cybersecurity insurers. So they will ensure a bunch of folks who get hacked and when they get hacked, they have a pre approved panel of incident response vendors who they will call in to deal with the hacks. And so actually a lot of the motion is going and meeting the insurance carriers and talking to them and building trust there. And then off the back of that you end up building the specific relationship with the enterprise. But yeah, a lot of the distribution is through the insurance carriers. It's quite unusual.

1:13:36

The AI underwriting. You may probably know the guys from the AI underwriting company then, I suppose. Yeah. Okay, interesting. Insurance strikes again. Okay, well, let's zoom out a little bit here in closing and talk about like, what's the big picture look like You've got this kind of, you know, initial service that's kind of your wedge into enterprises. How do you see that line of service expanding? You know, kind of paint a picture of like, as this matures and you get really good at it, you know, what can we enjoy in terms of additional security? Does it impact my, you know, day to day life at all? Or is it just something that kind of runs in the background and is sort of a guardian cyber security angel watching over me and what's the time can I invest before you sell to CrowdStrike, when they finally wake up and realize AGI is a thing.

1:14:10

Yeah, the way I see this broadly imagine we have just AI systems capable of fully automated digital forensics and much sooner than we otherwise would because we've done all this work of bootstrapping with humans and building these evaluations and so on. And the implication, I think, is that this completely changes how defensive cybersecurity operations work. Instead of missing this huge number of breaches because there's no one investigating deeply, you actually just have this proactively on end systems and so you're catching a lot more than you would otherwise be. And the implication here is just there's a huge reduction in the number of breaches that go undetected. And then hopefully this is just deployed across all the most important parts of the world. So it's deployed across AI labs, it's deployed across the AGI supply chain, it's deployed in western governments safeguarding things like autonomous weapon systems. And when you look back at the overall effect of AI as a diffuse and you think it had in cybersecurity, you say, actually now look, it seemed like this was overwhelmingly beneficial for defenders. That is, I think, possible. It requires a lot of hard work, but it's pretty clear what you would do to get them.

1:15:04

Anything else we haven't touched on that you want to make sure people are thinking about as they try to absorb your extremely AGI filled worldview?

1:16:11

I think the big thing for me is just I think we are one important attempt of doing this, like pulling out the jagged frontier in a way that takes seriously what the world will look like when we get these really powerful AI capabilities. But I see very little of that and I don't see any reason why that's the case. And generally I don't think it requires any huge leaps of imagination or things like that. It seems just very tractable, very important to do and I would love for to be many more such projects across cyber defense, across biosecurity, across the whole space app.

1:16:23

Yeah, time is of the essence. It seems we're due for the Automated AI Researcher Intern edition in just coming sooner. Yeah, yeah, yeah.

1:16:56

Wild times. Cool.

1:17:09

We'll definitely continue to follow this with interest and maybe with your motivation, I'll even get around to changing some of those passwords.

1:17:11

Sounds great.

1:17:21

Alexei Karla Schroding Founder at Asymmetric Security thank you for being part of the Cognitive Revolution.

1:17:22

Thanks Nathan.

1:17:28

If you're finding value in the show, we'd appreciate it if you'd take a moment to share with friends, post online, write a review on Apple Podcasts or Spotify, or just leave us a comment on YouTube. Of course. We always welcome your feedback, guest and topic suggestions and sponsorship inquiries either via our website Cognitiverevolution AI or by DMing me on your favorite social network. The Cognitive Revolution is part of the Turpentine Network, a network of podcasts which is now part of a 16Z where experts talk technology, business, economics, geopolitics, co culture and more. We're produced by AI Podcasting. If you're looking for podcast production help for everything from the moment you stop recording to the moment your audience starts listening. Check them out and see my endorsement at aipodcast Ing. And thank you to everyone who listens for being part of the cognitive revolution.

1:17:30