Hey everyone and welcome along to Seriously Risky Biz. This is our podcast all about cybersecurity policy and intelligence. My name is Amberley Jack and very shortly I'll bring in Tom Uren, our policy and intelligence editor, and we're going to have a bit of a chat about the Seriously Risky Biz newsletter that Tom has written this week. If you're not yet a subscriber of that, you can head along to our website risky.biz and find it, read it, subscribe, everything else. And first, though, I'd like to thank our sponsor for this week's show, which is RunZero. And if you're keen to hear a sponsored interview between Casey Ellis and RunZero founder, H.D. Moore, you can find that up on our Risky Bulletin feed as well. G'day, Tom. It's great to see you. G'day, Amberley. How are you? Yeah, really good, thanks. And I want to jump straight into the first piece that you've written about in the news edit this week. The US government, Tom, is kind of shaking its fists and it has sent a memo and it simply will not stand for Chinese distillation attacks against US frontier AI models. And that all seems kind of great, Tom, but you're left feeling a little underwhelmed, I guess, by the what's and the how's of this, you know, we're mad as hell and we're not going to take it anymore. So I guess, Tom, maybe start off, what are these distillation attacks? and what's the White House's answer here? Yeah, okay. So people spend a lot of time and money developing AI models. They go through these training processes and it turns out that one of the ways you can speed that up is just by using a more advanced model. So those are also called model extraction attacks and you just ask questions of the more advanced models and you use the answers to upskill the less advanced models. And so by doing that, you can get the performance close to the benchmarks. Like the model is not as good. It's still not as good, but it performs very, very well for a lot less money and a lot less training. And so back in February, I wrote about how within the space of a couple of weeks, Anthropic, OpenAI and Google all published reports or released information that said Chinese AI companies are pillaging our models with distillation attacks. I'm being a bit dramatic, but that was like in two weeks, I think it was, all three of them. And at the time they had you know here's the things we can do so in one sense the white house like it's the end of may they've produced that memo i think late last week um where they said uh this is what china's doing we're going to take these steps so that actually is really fast work for a government so i think that's commendable they've recognized that it's a problem and they've committed to doing something about it. Now, the problem is that the things that they've said they'll do are basically share information about distillation attacks, enable private sector coordination against attacks. So I don't know, we'll host a few meetings, facilitate development of best practices to identify, mitigate and remediate these attacks and explore a range of measures to hold actors accountable. So that's not even holding actors accountable. It's just exploring the range of measures. We'll have a look around and see what we can find and maybe. That's right. If we like the look of any of them. Now, in fairness, there's a number of moves in Congress where people are proposing legislation to do things like put the AI companies that do these attacks on things like the entity list and make it very hard for them to do business with US firms. So there's a range of options. But all of those felt very much like the let's do the things that we did for the last 20 years that didn't work against Chinese IP theft. So I think the really disappointing thing is probably the best tool that the US has is chip export restrictions. So those have been an ongoing effort I guess sort of kind of for the last several years maybe three something like that And there actually evidence that it does work So DeepSeek version 4 was released last week So DeepSeek made a whole lot of news at the beginning of 2025 when it released a model that was very, very good. It used a lot of innovative techniques. It was far better than people thought that a Chinese lab should be able to produce. And version four is also very good. And it's an open weights model. They've released it. Anyone can use it. But there is evidence that it's actually been delayed by the chip export restrictions. So I speak in the piece about a couple of how it was delayed. There's reports that they tried to train it on Chinese domestic chips produced by Huawei, the Huawei Ascend, and it just didn't work. And so part of the delay was that they tried it a couple of times, then they had to revert back to American NVIDIA chips. I just want to jump in there, Tom, because you sort of say they reverted back to using these NVIDIA chips and there are restrictions, right, on the exporter. So how did that all work? Yeah, so the chip export restrictions have been notoriously leaky for a start. There have been ways to circumvent those restrictions. The Chinese companies, a lot of them stockpiled chips when it came, so brought forward a lot of purchasing when it was known that there were going to be restrictions. So there's actually a stock of older chips in particular. The Chinese are still able to buy some chips. and they've subverted those restrictions at times. So if the export restrictions were perfect, there would still be older NVIDIA chips available. Yeah. They're not perfect. Good start. So now DeepSeek is trying to move to Huawei's architecture because, well, partly because the Chinese government said we should move to an indigenous architecture and partly because the access to nvidia compute is is harder to get so they're using the huawei stuff for actually running the model so there's sort of two different parts where you need a lot of computing power one is to train the model and the other one is to run it so they're using huawei to run it but it seems like the huawei's chips they're not as good as NVIDIA's and there's far fewer of them like domestic production is not very there's not a lot of it so that's all evidence I think that yeah the chip export controls they're imperfect but they've achieved they've had an impact so I don't think you can have a discussion about what are we going to do to maintain our lead in AI without really having chip export controls at the heart of that discussion and the memo just glides over it just going back to what is in the memo um information sharing tom to me kind of feels like it's almost like the thoughts and prayers of the intelligence and yeah infosec world i'm assuming there's more to it than that but it kind of like it does make a difference. So each of the AI labs, say OpenAI, Anthropic, Google, they're dealing with the problem on their own. They might not see the big picture. So facilitating information sharing is good. It will make some difference. Developing best practices altogether is good. It will make some difference. I just think that what's at stake is potentially so important that the Chinese government, the Chinese labs, they won't be deterred by countermeasures. They don't stop you doing it. They just make it more difficult. And like fundamentally, the way you do these attacks is you just ask questions and get answers. And that's what models are for. It's not like you're doing something nefarious, like hacking the model. It's, you know, tell me about this. It's a very interesting answer. And so that's the actual process of using the model that is the training. So I think it more complicated than that But it I think inevitable that those will be a speed hump and not a roadblock And so if you really want to make a difference you have to combine those measures with other things As you do mention in the newsletter, there may be a meeting coming up that is potentially part of why chips weren't mentioned in the memo. Yeah. So President Trump is meeting. There's a plan to meet in a couple of weeks. That meeting was delayed because of the war in Iran. I mean, Iran is still a problem. So at this stage, it looks like the meeting will go ahead. I think neither the Chinese nor the US wants to have a huge stink that would jeopardize that meeting. and imposing extra chip restrictions or announcing that you're going to impose them could be problematic. So that could be something that you would maybe hold off until after the meeting. I mean, chips don't define the entire relationship though. So there's a whole lot of different issues at play and this may end up being something that's not exactly a bargaining chip, but positions get moderated because of the broader relationship. Sticking with China, Tom, your second piece here, a joint advisory has been sent out between the UK's NCSE and international cybersecurity authorities. And it's basically a warning that Chinese hackers are shifting to using covert networks or botnets. So I guess, Tom, tell me about these botnets. Why are they beneficial for adversaries? so they basically take a whole lot of compromised you know things like small office home office routers cheap internet devices network attached cameras digital video recorders whatever is out there connected to the internet and has poor security and they you know it's just like any other botnet, they string them together, but then they use them for everything related to cyber espionage. So reconnaissance, just anonymous or deniable web browsing, data exfiltration, command and control. And the idea is that because these devices are spread all over the place, it's quite hard to figure out who's ultimately responsible. And the idea is that instead of having to go to the effort of buying or constructing your own infrastructure. You just basically steal computing resources from whoever's devices are not secure. So this is not a new thing. So the Russians, the FSB, they had this thing called snake malware that used a similar principle that would compromise lots of boxes and shuffle traffic around. They used that for 20 years from 2003. The Russian military intelligence, the GIU, they've had a couple of botnets that they've run, VPN Filter and Cyclops Blink, I think were their names. And in recent years, the Chinese, there was something called the KV botnet, which was used by Vault Typhoon. That's the group that's compromising US critical infrastructure. And there's another group called Salt, or was it Flax? One of the typhoons. Another Chinese group was running another botnet. And so the report talks about there's multiple botnets. They're constantly being created, renewed, maintained. Multiple hacking groups might use a single botnet. And I suppose the point is that it's both a threat and an opportunity. Yeah. So it seems like the driver, unlike the Russians, the driver has been that it's just a good idea that literally makes commercial sense. So in the same period of time, we've seen criminals use what are typically called residential proxies, which sometimes are a very similar idea where you take compromised devices at homes, hence the name residential, and then you sell them to other criminals. So what works for criminals also works for states. It seems like the Chinese cyber espionage community is just tremendously entrepreneurial in the business sense of the word Like we develop a cyber espionage capability and we sell it Very different from the US, the Australian, or even the Russian paradigm where you've got state intelligence organisations that do state intelligence things. And you do have commercial contractors, but it's not like they're just selling on an open market. But they've got these really tightly held contracts with particular people. This is just like, yeah, this seems like a good idea. I'll build it for my own purposes and I'll sell it to other people and they can go and hack on my botnet. That feels like the vibe. See, I've been working at risky business long enough now, Tom, to have heard about a lot of botnets being disrupted and taken down. So does there just need to be more of that? What's the answer here from a government perspective? Yeah, so I think this is both a problem. It's a problem for the organizations trying to defend themselves because the reason they use it is it makes it harder. There's no attacker-controlled infrastructure. It's very flexible. These IP addresses turn over very quickly, like the botnets turn over very quickly. So it's like a river. You never step into the same thing twice. But from a government point of view, there's actually a track record of quite a few disruptions nowadays of these botnets. So all the intelligence agency-led botnets I talked about, they've been disrupted by the US government and partners. And earlier this year, the FBI disrupted four distributive denial-of-service botnets. So there is actually a really long track record of just disrupting them, stopping them working. it's never permanent because the reason like it's an adversarial relationship so if you're the threat actor your botnet gets disrupted you don't go home and leave and stop you go okay well i have to do something else i have to recreate it sometimes they can recreate them sometimes they're dead in the water and they have to try a new approach but it's in the in the piece i talk about it the threat threat actors are on a treadmill where they're constantly building them recreating them maintaining them like a phoenix they arise from the ashes and so the government's job i think in this case is to recognize that they're they're on a treadmill and like try and push them off all the time and disrupt them degrade them there's opportunities because of the nature of the way these botnets work that there are places that you can actually disrupt them i think there just basically needs to be more of that. When I was looking at the timing of these disruptions, there was quite a few in one year, then another year where there's hardly any. I'd like to see them happen every couple of months that another one's disrupted. Since you and I have been working together, Tom, it feels like a lot of the things that we talk about feel like a kind of, I don't know, 30-year never-ending game of whack-a-mole, and you've just kind of got to keep on whacking, and that's pretty much what the story is here as well. Yeah, so there's a theory which has been quite successfully adopted. It's people kind of believe in it called persistent engagement. And the idea is that just because of the nature of the internet that the way you've always got to contest what the adversary is doing. And I think this is a really good example of where this perfectly applies. If you just sit back and let them create these botnets and run them, they'll take advantage of them. It becomes more difficult for you. But if you contest them, You don't win, you just make life more difficult and reduce the amount of harm that they can do. So I think that the dedicated, ongoing, persistent efforts to be continually knocking down these botnets, I think that's the way to go. All right, Tom, hey, we may leave it there, but thank you so much for your time yet again. And you can, of course, read and subscribe to Tom's newsletter over at our website, risky.biz. But Tom, have a great week and I will see you the same time next week. Thanks, Emily.
