Hey everyone, I'm James Wilson and welcome to Seriously Risky Biz. This is our podcast all about cybersecurity policy and intelligence. Earlier today, I had a really great chat with my colleague Tom Yuren, who is our policy and intelligence editor. And I sat down and talked with Tom about the Seriously Risky Business newsletter that he is publishing this week. In this week's newsletter, he's gone through two really interesting topics. The first one is around, I guess, the blowback from Mythos and the way that it was released and how between what Anthropic has done with the preview of Mythos and OpenAI just releasing GPT 5.5 without any sort of preview or staggered release, this seems to have prompted the US government to have a bit of a rethink as to whether they want to retain their hands-off, just-go-for-it sort of policy towards regulating the release of AI models to potentially considering a more hands-on role in regulating the release of these very powerful frontier models. The second topic that Tom covered into this week's newsletter is around Australia getting its own equivalent of the CSRB or Cybersecurity Review Board. And Tom talks about whether the The blameless framing of this review board and also the legislation that's behind it might render it a little bit less or a lot bit less effective than we would all otherwise hope it would be. But hey, if you're not already subscribed, please do head over to our website at risky.biz where you can subscribe to Tom's excellent newsletter, Seriously Risky Business. And I'd also like to thank this week's sponsor, which is Portswigger. You can find them at portswigger.net. I recorded a cracking interview with James Kettle from Portswigger that was released on Monday, which is all about the diabolical results that you get when a security expert like James codifies his knowledge and practices in a way that an LLM can access and begin to do research like James Kettle does. It's a scary look at what the future is going to look like. And this kind of dovetails into this first part of the conversation with Tom, where we talk about this seemingly abrupt change in stance around the US government considering regulating the release of these powerful frontier AI models that could have some serious cybersecurity ramifications. I'll drop you in here to the conversation where Tom talks about just how did we actually get to this point where the US has had this change of heart. Enjoy. Well, I think the Trump admin, when they first came into, took office, one of the first things they did was basically bin a whole lot of Biden era regulations around AI. And just out of spite or with good reason? Well, I think I would describe it as philosophical in the sense that you've, you know, one camp, any regulation slows down business and slowing down business is terrible. I think that's an exaggeration of their position. But, you know, that's basically what it comes down to. The other camp is, and I'll probably give it a more sophisticated view because it's more aligned to myself, is that companies sometimes do things that aren't in the public interest and we need regulation to manage that. And it is a balance, right? You don't want too much, you don't want too little. They went with very little. And I think it's easy to go with very little, like, what was that, a year and a half ago, something like that. when it is, the risks are somewhat theoretical. At that point, models were not very good at cyber tasks. There was no great harm that was coming because they were hacking the planet. And I think it really sharpens the mind, you know, it gives you some focus when you've got concrete examples of models doing things that are potentially very risky. So in the last month or so, we've had a whole heap of newsworthy cyber, not incidents, but discoveries driven by, in particular, Mythos preview, which is Anthropik's latest model. But, you know, there's a part of me, Tom, that just thinks, what's the old saying? You know, if you've got to go through hell, don't walk slowly. There seems to be an element of a hell of a time is coming for us because these bugs are going to be found, they're going to be exploited. Like, it just feels inevitable. And part of me wonders, like, do we really want the government slowing this down? Because when they slow it down and create these gaps between, I guess, who's got access to models and when the timing of the models, it's those gaps in time that are going to be exploited by attackers. So what is the real tangible benefit of even getting a government oversight of this? Yeah, so there's a number of different stories that appeared in a number of different outlets over the last couple of weeks. And one of them is the administration's thinking of setting up a body that would basically decide what the right thing to do is in terms of regulation. And that would consider everything from, you know, government review before models are released to kind of a light touch. And it seems like the administration is keener on what Anthropic has done, which is release Mythos preview to a limited number of organizations. There one report that Anthropic wanted to increase the number of organizations from I think 70 to 120 So like that like doubling the size but in the scope of the planet Still very small. And the administration actually pushed back. That's the report. Anyway, that seems like their approach is we're keen on the ring fencing and then like relatively slowly expanding that. That seems intuitively like the right thing to do, right? but I'm at the point where I think I've got no idea what the right thing to do is. I'm not sure that the advantages from doing that are as great as we think. So, for example, like your interview with the Portsmouth, what's his name? James Kettle. James Kettle, yeah. He's doing all sorts of crazy discovery with an older model and then there was another report this week from niels provost who was a google distinguished engineer at one point and he's basically yeah i can find all sorts of stuff with an older model when i give it a harness so you construct a scaffold around it and it does uh close to as well as the the latest mythos and right which which flies in the face of any potential benefit of regulating the model in and of itself. You know, like I think what those two instances, the James Kettle sort of work that he's doing, and he's open sourcing that at Black Hat US in July. So like, again, it feels inevitable that this apocalypse is coming. But those guys have shown us that, I think you cited it in the newsletter well, that the difference between a novice with a absolute bleeding edge, incredible model like Mythos and a expert with an older model not that different. And so what level of regulation is going to help if it's just at the model level, I guess is my question, and would they need to go broader? Yeah. I also think that the most advanced adversaries are the ones that will have scaffolds and will be working really hard. So I guess the argument for holding back models is that there's a whole lot of people who would do some hacking if they could. They can do the easy mode hacking. And I think that's worth considering, like what's the unmet demand for random hacking that would be enabled, right? That I think is actually what you're talking about by holding back models. I think if you're talking about national security, I don't think that actually makes much difference. The random hackers will hack random things and they won't enable China or Russia or whatever. But I think that you're not actually gaining that much when it comes to China, Russia, these sophisticated adversaries, because they'll be trying to use open source models. They'll be coming up with jailbreaks. They'll use harnesses. And I think for nation states, where it's at is getting the most advanced model you can and getting the most out of it by putting a harness around it. Yeah, yeah. And so from a national security perspective, I think that's where the game is at. But like I said, there's a whole lot of other issues as well. And it's not immediately clear to me what the right answer is at all. I just think that locking it down is not as good as we think it is. And so I'm much more keen on the idea of, yeah, let's get involved in model releases. Let's collect a whole lot of information about how they're being used, how many bugs they're finding, how quickly those bugs are being patched, what happens when they get patched. Does that actually result in an end-day apocalypse because attackers can now reverse engineer the patches? Yeah. And so I think there's a whole lot of second-order effects that are not clear at all. And so you think there's a more meaningful role for government there to collect that data, to sort of be, I guess, the arbiter of good process around observation, surfacing those things. And I think you said it in the newsletter well, Is it a little bit like maybe we should just wait and see before we start bringing down the band hammer on who can release a model and when? Is that sort of a good summary of your take? Yeah, yeah. I also think that it probably does make sense for each or every company to follow the same processes. And so I dive a little bit into the processes that Anthropik's taking, which is to release Mythos Preview to a small number of organizations. and OpenAI, their latest model, GPT 5.5, is basically just as good as Mythos. So the UK's AI Security Institute did testing and it found that it scored better than Mythos did, but within the margin of error. Yeah, and ironic that poor Anthropic is the one that keeps getting beaten up by the US government and yet they're the ones that are seemingly being safe and cautious. Yeah, exactly. And their OpenAI's model is anyone who's paying can get access to 5.5. Well, yeah, deliciously capitalism, right? Yes, yes. And I'm not sure that that's wrong either, right? So they have a kind of tiered model where it tries to funnel people into a trusted access program where you have to verify who you are and whether you're a legitimate defender. And if you can do that, you get access to more tailored versions that are a bit more cyber permissive. Right, right. A little bit less safeguards still same model I think is most of the sense Yeah and so that very cautious versus relatively open with safeguards I think those are quite I wouldn say they diametrically opposed They both notionally achieve the same goal of safety. But there's genuine questions about how effective is that program? Would we be better off with that program rather than with a more restricted, on the theory that more defenders having access would actually be better on net. Well, I think this comes back to your sort of point there around government needs to do a bit of a wait and see, because if we look at those two different approaches, the you know, restrict the model, slowly release it versus release the model to everyone, but rely on the safeguards. What we're really coming down to there is the faith in those safeguards and the ability to create a structured cyber, you verified your identity, you can have less safeguards, right? It all comes down to the faith in do those safeguards work and is giving sort of a smaller group of people access to the same model but with less safeguards a good way to still enable cyber research. But we're not going to know that unless we have someone that is actually objectively collecting that data, right? Yeah, exactly. Yes. Couldn't have said it better myself, James. Thank you. You say the nicest things, Tom. Well, let's move on to perhaps where you've not said the nicest things around Australia launching a hamstrung cyber review board. That's a clear bit of shade that you're throwing there. As I read in the newsletter and I quote here that you've called out their intention here is to deliver actionable recommendations to government and industry to help prevent, detect, respond to and minimize the impact of similar incidents. these incidents being serious cybersecurity incidents in the future. Now, Tom, that sounds like a great intent. Why are you calling this hamstrung? Yeah, so I've written several times about the US version of this, the Cyber Safety Review Board. I just want to say that a guy I went to school with is on the Australian Cyber Incident Review Board. So congratulations, Baron. Best of luck. Now the bad news. So the Cyber Safety Review Board did a number of reports. Some of them were, I would say, quite technical in nature. And the recommendations were very much this way of doing things isn't appropriate anymore. And I'm thinking about they did one on lapses and the kids were basically bypassing cybersecurity controls and they recommended specific things about SIM swapping, for example, or SIM porting or mobile number porting or whatever. And that's very technical. And, you know, those standards existed because they were appropriate some time ago. Now, there's another report they did where they basically said, Microsoft, you just don't care enough about security. And they talked about a cascade of failures. And I would describe that as a kind of political report in that it has a political impact on the leadership of Microsoft. It wasn't you did any one particular technical thing wrong. It was you don't care about security enough and you should because you're super important. and that had impact. A few months later, Satya Nadella stood up, sent an all-hands memo, said security is the top priority at Microsoft. That didn't last all that long, but while the board existed, it had impact. For a few minutes, it was good, yeah. Now, the problem with the Australian version is the legislation says the board's report cannot apportion blame. Right. And you've called this out. you said the board's impact will be limited by its approach to liability. But as I was reading that, it was quite insightful to me the way that you called out there's a difference between apportioning blame and also having protections against liability such as, you know, like you can blame, but you can then say this blame can't be used in court. So talk me through, I guess, you know, help me understand what should they have done here and just how limiting is this because of the way it's been structured. Yeah, so the analogy for the Cyber Safety Review Boards or the Cyber Incident Review Board is the US NTSB, the National Transportation Social Safety Bureau, or the Australian equivalent, the ATSB. And they can write reports and they can say, you know, this person stuffed up. They were drunk. They had failed their medical. They were psychologically unstable. Whatever. If the reason exists, they can say it. but that report can't be used in court for liability. Like that's the legislation. So what is the lasting impact then? Is it just a you're bad and you should feel bad? Like if it doesn't get to court, you know, what is this? The point of the NTSB reports is here's the cause of the accident, here's how we can fix it, and there's no liability because that encourages people to contribute to the report, to speak openly, to tell the truth. And you know that if you do that, it can't be used against you. Now, the Australian Incident Review Board, it's trying to get to the same place where people speak freely, but it chokes off liability at the wrong point. I think those reports should be able to say, you know, company, you were terrible. You just didn't care. The root cause is not some technical issue. The root cause is that you made stupid decisions about risk And I think like it the nature of I think businesses that deal with risk that there like technical issues and there like management issues yeah and you can separate the two and I think the problem with this is that it removes the ability for the board to say you guys were just so dumb yeah and that removes the political pressure to fix that kind of problem and that I think is a, it means that the impact of the board is limited. Yeah. Australia's had a couple of cybersecurity ministers who've been very good at removing the bull that victim companies have run, the PR that they've run. Right, the sophisticated attacker. Exactly. We couldn't possibly have defended against this. And she's just stood up and said, no, it wasn't. Was that around like, I think, was that the Medicare one or the Optus one maybe where I think she really cuts through. I think she might have done it at both. Yeah. And I think the way that the legislation is for this board, it removes the ability of the board to cut through in the same way. So I think that's a shame. Yeah, it is hard to imagine what the real teeth of a report is if it's got no blame. Like I'm trying to imagine in my mind how, like at the end of the day, an incident happened because something went wrong and something went wrong because someone did something somewhere, right? you always get to that point um i think this is particularly timely because of the rise of ai so i think there will be novel attacks that people can't expect i think there'll be like you know some proportion of major incidents will come from something like out of the blue well actually i did want to check this with you as well because i again quoting something here that you put in the newsletter you you quoted the again the uh i think the legislation around this and said that there's a criteria for what an incident has to meet in order for it to be eligible for review. And I found this a bit interesting. It says the criteria is that it involved novel or complex methods or technologies, an understanding of which will significantly improve its preparedness. But I would put it to you that making the criteria that the attack had to involve novel or complex methods cuts out a whole swath of stuff that should still be involved. Because frankly, sometimes what's novel is actually the level of stupidity and incompetence on the side of who's been attacked. AI at the moment that we're seeing, again, sort of to your point that there is sophisticated use of AI, which will be done by state level actors and experts. Then there is dumb uses of AI where it's like it's not dumb if it works and it's now working at scale. As I read it, those sorts of things wouldn't meet that criteria. And so is that going to limit kind of what we learn from this as well? Do you think they'll really stick to that criteria heavily? And does that, you know, again, further limit the success of this board? Yeah, yeah. I think there's a big difference between pre you've done the review, it looks like a novel or complex attack, versus after you've done the incident review, does it still look like a novel or complex attack? and so I think that there'll be you know I think these are political things if there's a major incident the board will be rolled out regardless of whether it looks like it was complex novel and so you think this is a catch this catches the marketing ball in the in the PR if someone dares to say oh we were attacked and it was incredibly novel and complex then they'll be straight on to Good, you meet our criteria. Please come in. Yeah, and so I think that stepping back, there'll be one class of attack or some percentage which will be like this, where it is actually something new and different and worth investigating. Like, you know, what do we do about this? And then there's another class which will be an executive thinks, AI is just so wonderful. Let's roll it out everywhere without thinking about consequences. And I think there is a line to be struck of, yes, this company rolled out AI. Yes, it actually thought about the way to do it correctly and it had mitigations in place. And, you know, it tried, I guess you'd call it due diligence or whatever. I think it did a reasonable job trying to mitigate those. And I think that would be the sort of straight down the line, here's the technical reasons that failed or why it wasn't good enough. What else can we do? And I think there'll also be some category of we just did really dumb stuff and we didn't think about it. And that's the kind of – and I think that might be quite large. Yes. And I think that is the sort of thing where you want to be able to say in a report this was just idiotic and people should not be doing this kind of idiotic thinking. And I think the way that the legislation is, it cuts out that as a possibility. Yeah. Well, either way, Tom, we've got to wait and see on this. Fun times. Yeah, strangely, I find myself looking forward to the next major cybersecurity incident in Australia, so we can see whether this actually works. But, Tom, let's wrap it up there. It was awesome to talk to you. And, folks, of course, if you haven't already subscribed, please do head over to our website at risky.biz, where you can subscribe to Tom's awesome Seriously Risky Business newsletter. Tom, thanks for the chat. This was a lot of fun. Thanks, James. Altyazı M.K.
