Jordan Schneider Thinks the Real AI Race Is About Compute, Not Models

Suddenly, just in the past month, the progress of artificial intelligence, the sheer speed of the developments, the thrill, and frankly, the terror of what these developments portend for the rest of us, all of this has gone into overdrive. First up, OpenAI has confessed that two of its models escaped from their testing environment, advanced out onto the internet, and penetrated the defenses of another AI company, Hugging Face. Meanwhile, Hugging Face, the targeted company, announced that in order to figure out what the heck had happened, it had used a Chinese AI model. So China, often accused of undermining Western cybersecurity, actually helped to maintain cybersecurity in this instance. Also, meanwhile, each of the three frontier labs and a couple of non-frontier ones have released new models in the past month. China has announced a string of new AI models that compete with Western ones in capability. Various Western AI leaders, including Demis Osabes of Google DeepMind, have proposed safety ideas. The Trump administration denounced China for reverse engineering American AI models. And China's President Xi Jinping delivered a major speech on China's plan to be the top AI provider for the world, announcing what one might see as an AI version of the Belt and Road Initiative. So in short, the models are getting stronger. They are getting more dangerous. And politicians and lab leaders are floating a bewildering slew of policy ideas in an attempt to get on top of the chaos. I'm Sebastian Malaby. Welcome to The Spillover. Today, my Spillover co-host Rebecca Patterson is on the road. So I'll be discussing AI progress, AI safety, and US-China AI rivalry with Jordan Schneider, the founder of the China Talk podcast and newsletter. I am super excited to have Jordan on the show for two reasons. First, he's incredibly smart, especially on the Chinese side of the AI race. And second, he has vouchsafed an earnest promise that he will disagree with me strongly. Jordan Schneider, thanks for doing this. Oh, I'm so happy to be here. I'm an enormous fan of yours. If only I could write books like you do. I'm just a lowly podcaster out here. So honored you've decided to jump into the medium. Well, you know, I'm a beginner. You're not a beginner. We were joking before that you're an overnight success after 10 years of hard slog building your channels. But just tell us, you did history at Yale. You studied in China. You went to Bridgewater, the hedge fund. Tell us a little bit about your journey before we get started. Oh, man. Well, I mean, it's still Odyssey week, so we can start off with the Yale thing since you brought it up. I did a lot of ancient Greece and Rome. I was still Latinist back in the day. And it was so fun to just get to like come back to all that stuff. Are you more fluent in Latin or more fluent in Mandarin? Oh, I mean, yeah, I'm like down to 10 words of Latin at this point. It's a little embarrassing. But I think one of my colleagues who's in this building, Eddie Fishman, took all those courses with me. What can I say about my journey? You know, the big hinge was in 2017, I was sick of Bridgewater and applied to every fellowship I could find on the internet. And this was before ChatGPT. So you actually had to like look for them to send me to Asia, like broadly. I think it was like 10 different countries or something. And Peking, Yenching Academy at Peking University is the one golden ticket for free grad school I got. So I showed up in China in 2017, spent three years there, which is also when I started. China Talk, and I've been back in the U.S. since COVID. And now China Talk is not only a podcast and newsletter, but also a little think tank with a small but mighty band of five or so researchers. Well, congratulations. I know you've got a lot of people who follow, a lot of fans. So well done. So let's get into the AI topics. I mean, given your background and my obsession, you have expertise. I have just sort of, you know, compulsive infatuation for the China side of this story. We'll do a lot of China in a minute, but let's start with the US side of AI and maybe the sort of safety debate around AI. I'll give you my sort of capsule framing of how I see the recent history and perhaps you can fill it out, tell me where I'm wrong and so forth. I'd say that, you know, you get ChatGPT coming out in 2022. Pretty quickly, in other words, within a year, by the end of 2023, you've had the Bletchley Safety Conference in Britain with countries sending people from around the world. You've got the US AI Safety Institute, the UK AI Institute. That's end of 23. Then during 24, this builds and a lot of other countries set up national AI safety institutes. Of course, these have limits, but the infrastructure is being created. And at the end of 24, you get this meeting of all the different safety institutes in San Francisco, where there's some talk about combining perceptions on what makes a model safe, what safety, how you define it, et cetera. So there's this kind of fairly decent progress through the end of 24. Then Trump comes in early 25, France goes to Paris for the AI summit, which is in that line of meetings created by Bletchley. And he flips the script and says, we're not interested in safety. We wanted to roll out enough of this woke stuff. Let's just deploy, deploy, deploy. And so 25 is kind of a dead year for safety. And as late as March of 26, if you had said to the Trump administration that they would be trying to suppress an American AI model, decelerate the progress in the name of safety, they would have said, you're nuts. But actually, April comes, Mythos comes from Anthropic, and they do a 180. And they do suppress a model release, and they do care about safety. And so we've had this kind of helter-skelter, like gradual rise of safety, a collapse of the interest in it, and then a return. Is that how you see it, or would you add to that, change it? Yeah, well, I mean, it kind of comes back to like Demis reading sci-fi in the 90s, 2000s, and 2010s, because the creators of this technology in the in order to be interested in this stuff in the 2010s, had to be a believer and had to internalize that this was like a deeply weird thing, which would ripple through society in like very uncomfortable, I mean, exciting, but also very uncomfortable ways. And I think the Trump administration pre-mythos didn't believe it. They just, and I think the Chinese, we can get to them later, also still don't really believe it. They think this is a normal technology, which is exciting and has a lot of growth. But like, you know, we don't need the we're we're, you know, Republicans, we're anti-regulation. We took we have money from A16Z and Mark Andreessen's advising us on any and on everything. And he's telling us it's OK. So it's OK. And then there's been this fascinating dance that you've that you've written about of the model makers, like kind of understanding that they're going to need to get regulated, but like not in this way, Not in a way that is going to stop me and my first thousand employees from having nine figures to our name. Right. So we we have this interesting dance. And yeah, like was could you have priced in the Trump administration flipping at some point? I mean, I think if you're Demesis Abbas, the answer is yes, because you understood that we were on this curve where at some point, you know, you get to a point where governments are just going to have to step in and say, no, we're going to have a say in how this technology is going to be developed. But, yeah, it's been really dramatic. And on the China side, it hasn't happened yet for them. There's still if you look at or actually you're the interviewer. That's the next question. We can talk about something else. For the next question, I was actually going to stick on the US a little bit more. So, you know, we've kind of taken the story up to that mythos moment in April. The Trump administration changes position. You have Project Glasswing, the gradual rollout of mythos and the kind of, you know, back and forth about, you know, is Fable allowed and is it not allowed and so forth. And then most recently, just this week as we're recording, you have this episode with two open AI models that for the first time ever autonomously jailbreak, get out of the sandpit where they were supposed to be confined, find their way onto the internet, hack their way into another company's site hugging face. This is not a sort of amateur, you know, small company. This is a serious cutting edge AI company with lots of capability. They get hacked anyway. I mean, this is pretty unprecedented. They think it's the Chinese, right? And they're freaking out. They're calling the FBI. And then two days later, they're like, oh, wait, actually, it was OpenAI testing their models. So that's how serious it was. Like they were convinced this was a nation state level threat. Right. But it was just like a test gone awry. Right, right. It's the nation state version of the Turing test. This is wild. It's wild, right? I mean, like, and you've also had this discourse over the past year and a half where people keep saying, oh, we think alignment is more and more under control. It's less of a problem. And then something like this happens. Right, right, right. But so how do you see this impacting that arc of safety? Like we've got to the point where, you know, there's some desire to kind of clamp down on or control the release of Sol from open AI, Mythos, Fable from Anthropic. Does this just like intensify the same thing or do you see some fundamental gear shift? Well, look, like someone's going to have to do this work, right? The work being figuring out the policy response. No, like making sure that the models, as they get more powerful, don't do crazy things like this. And I mean, and someone needs to be able to be to like understand whether this stuff can get a thumbs up, right? And if OpenAI, who has every financial incentive in the world for this not to happen, right, isn't doing the work, doesn't know their models well enough, can't control it. Is Casey with its like $10 million a year budget going to be able to do it? Is UK AI Safety Institute with its $50 million a year budget going to be able to do it? I think it is just another underline to the fact that we've really crossed a threshold with these models, like having the potential to do really dramatic harm just on their own because we can't even physically watch them. Right. So the Casey that you just mentioned is the renamed version of the Biden era USAI Safety Institute, now called Casey, whatever that stands for. Which has $10 million a year and has gone through two leaders in three months. And they had a guy who showed up for a day who was fired because, like, apparently no one realized that he used to work at Anthropic or something. So I guess, like, we've now reached a point where this administration, and I think increasingly the world, is going to be turned on to the fact that there are real downsides and there are real scary things that could happen. But no one has the answer yet about how to do this in a smart way. So let's talk about one proffered answer. And that's from Demis Asabes, the one that he put out last week. And I think one of the things he says in that is let's have a self-regulatory institute. And I think I see this as an end run around the budget problem that the existing AI Safety Institute has. I mean, you said $10 million budget. That ain't going to get you anywhere. But if you had a self-regulatory body, you could not depend on appropriations from Congress, which slows you down and limits your resources. You get, you know, some sort of middle ground between something which is a government thing under-resourced. But, you know, you don't want to rely either on open AI to police itself because we just did that and it didn't work out so well. So you need a serious independent body, but maybe funded by industry, not by the government. How do you see that? Do you think that's going to work? It's just really tricky, right? Because there's so much money on the line. And it's like, it could be companies on the line. It's like, all right, if these firms are going to be public soon, like what is the, how much market cap are you losing because you're slower to market by two months because, you know, some obnoxious independent AI safety person says your model isn't safe, but you actually think it is safe. So, I mean, yeah, it's nice, but there are no clean solutions. Demis, of course, like he tried to have his own independent safety board, or I guess this was Mustafa with his. There's the two of them together. Yeah, and it was with their NHS medical stuff. And it just like got really obnoxious really fast, because I think that like the the problem is the the incentive structure is like if anything bad with AI safety happens with the board is like they're blamed for the lights going off in Denver, right? But we also have like an economy to keep afloat by not sort of taking off technological advancements from the future. So you mean the bias of any AI safety body is to be too restrictive because you'll take the blame if you are not? And then it slows it down too much and that's bad for the economy and China it gets ahead. It's just a really hard design problem. I mean, I have more faith in you doing it than like the companies themselves. Well, I think it's just like- Shoot, we're really in trouble then. I know, right? I need a few more, a little more tech than just three cameras to get us the answer to that. No, I don't have a great answer. Yeah. And is there any other idea floating around proposed by somebody else that you favor as a sort of like better thing? Or do you just view this as such a difficult problem? You know, I think a week ago, had you asked me this, I would have figured that the model makers understood. I think the, I would have thought that the model makers had enough incentive and had enough kind of technological capability. I mean, it's only two companies, but they're like the richest. I mean, I guess two and a half, three, sure. Well, it can be nice. they have enough technologists where like, okay, if the capabilities are exceeding our ability to control them, then you just like shift over compute and you shift over human beings until your ability to control and understand what the models are doing catches up to how powerful they are. But if that's not happening, someone going to need to make it happen Yeah exactly And I don actually know if like market force if like the competitive dynamics is you know if the cost of this like how, like, I don't know, you've written a lot of markets books. Like how much, if OpenAI was public and that news story came out, like what percent do you think their stock would have gone down? Wow, yeah. 15, I don't know. I mean, I don't think it would have been a zero, but I think, you know, yeah, it would be a hit to people's confidence in the technology, the confidence that they could serve technology to customers, and customers would believe in the safety of it. Yeah. Yeah, so it would be a problem. So maybe you actually just need the company, you need the companies to be public for them to really internalize the cost of this. Yeah, but I think the incentive distortion is a little bit different. You could frame it that, you know, maybe, to give them the benefit of the doubt, the lab leaders of Google DeepMind, Anthropic, and OpenAI all would quite like to spend more on safety, but they can't unless the other guys do as well. And we'll get to China in a minute, which is obviously a big part of this ecosystem. But there's a collective action problem which can only be solved by some sort of body that's regulating all of them. And so if they all were slowed down in the same place, they might be cool with that. Yeah, but if the body slows them down and one is in first and one is in second and one is in third, and the cost between being in first and second is like, I don't know, $250 billion of market cap, like it's a problem. Right. So I agree with this, but I think in the abstract, but like if you freeze everyone in place and someone's ahead of someone else, then you're the incentives. You could have a World Cup soccer game with a referee, but both sides will doubt that the referee is really fair. For sure. Yeah, yeah, yeah, yeah, yeah. All right. Let's switch to China. And I want to split this into two parts. One is to talk about internal actions within China. And then after that, we'll get to sort of U.S. policy to China, the kind of, we'll bring it all together. So taking the China internal story first, we've seen a slew of recent models. And looking at Kimi K3 and the rest of them, how do you see the gap? How big is the gap between U.S. and China at this point? So I think the, I remember having conversations even like five years ago about the idea that algorithmic innovations and data would kind of end up netting out between the U.S. and China. And the thing that was ultimately going to matter in the long run was compute. So in other words, just to clarify, so the data, China would have an advantage because it's... No, no, no. So data, a wash. I see. algorithmic innovations, how smart your scientists are. Also, you know, in the, in the, in over the medium term, awash. Got it. But when you're talking about like how much your, your nation, your state, your economy can gain from AI, it would come down to how much compute you have. Because you can steal weights, you can copy other, what other people are doing on the model side, you can steal dating train assets, you can make your own dating train assets. I mean, it's not like there aren't people in China to label data, right? They have lots of PhDs and whatever, just like Mercore can pay people. And at the end of the day, like, yeah, it's nice to have that model advantage. But what's gonna matter over the long term is like how much AI can you actually use? And, you know, we've run the numbers here at China Talk. The ratio of what the Chinese ecosystem has access to versus the rest of the world is still like 10, 12 to 1 in terms of both production and access to production of like raw chips as well as access to cloud compute. And so even though, you know, over the past, since ChatGPT came out, we've gone from 18 months to nine months, back up to 12 months, maybe now down to six months. that is the thing that I think is probably a more important lodestar than how is the best model in China today relative to the best model available to the US. In other words, I shouldn't be asking about how far is China behind in terms of the quality of the model. It's more a question of how much can China deliver the AI to users within China, given their lack of computational resources. Within China and around the world. There is, though, still a question about the power of the models in the sense that one significance of a strong Chinese model like Kimi-K3 is that it might be used by non-Chinese companies. I mean, it is being used. Yeah. And served on an American cloud. And to that extent, the Chinese lack of compute resources doesn't matter, right? What does matter is that, you know, Kimi K3, if it's really good, cannibalizes the revenues of the US frontier labs. Yeah. So point A is like a 10-year net assessment thing. The fact that Chinese models today are, you know, only five or six months behind as opposed to two months ago where they were more like nine to 12 is absolutely very relevant for OpenAI Anthropic and DeepMind if you're trying to make a business on selling API access. I think it's debatable. Like as long as you have a lead, it's debatable. And I think the point that you just made of like, it is like, it requires so much less compute to train a model than it does to deliver it at scale. And like having a business requires delivering it at scale, right? That, yeah, there's a lot of knots to unpack there, which maybe we should. Yeah, and just quickly, one last thing I think worth maybe putting on the table because I think there's some misperception about it. Just because the tokens generated by Chinese models are often priced at a very low rate, it doesn't follow that they're better to use because the capability, you should really price, What's the cost of the AI for each task you get done? Yeah. And if the, as I understand it, KimiK3 is cheap on a per-token basis, but not cheap on a per-task basis. Yeah, well, I mean, it's kind of like, we'll see, right? Because it hasn't been open-sourced yet. That's supposed to happen in three days. And like whatever they are selling their token, whatever the tokenomics of like... That is a new use of the word tokenomics, by the way. It used to be a crypto term. You've just- Oh, no, no, no, no. This has been used for a while, though. This is a shout out to Semi-Analysis. They started with this. The tokenomics of whatever their business model is of like selling it via Olicloud or whatever is going to be very different from once it's on open weights and then it's on Azure, right? Because presumably they're taking some slice. Or maybe they're not and they're just selling at a cost because people want to like have access to it. But also like the economics of like Azure is more well run than Alibaba and they have better access to cheaper chips. So I don't know. But I think there's I think there's like the this is like the more like CFRE question, right, is if you have sort of what does it do? And this is the thing that the U.S. labs are all freaking out about. What does it do to your long-term business model if you're trying to have this cash cow of selling these close frontier models that is then going to feed back into your R&D? And insofar as this continues and it actually turns out that we get Chinese models for years that can kind of prove that they can continue to sort of open way, like not quite best in class, but like pretty close for like a lot of your stuff. And there are a few other independent variables of like to what extent is like the marginal, you know, having like Claude Mythos 6 versus Claude Mythos 5, like actually how many business cases do you actually need for that? But yeah, I think it's a significant long-term threat to the business. There's significant security questions that I think come out of that. And I don't know, it's also a big, big mess. So I want to make an observation. We'll put a pin in it and come back to it. But the observation is simply that just like the US has chip export controls, which are a way of sort of damaging the Chinese AI ecosystem, trying to slow it down and so forth. I think of Chinese open weight as their kind of counter weapon in the sense that they can sell cheap models, which are sort of good enough to American consumers, American enterprises, and this will damage the economics of the frontier labs and it will slow them down a bit. To be clear, no, they're not selling them. They're giving them away right now. Okay, okay. They can give them away such that they are then sold to the Western users. Yeah. It's fascinating because this was an emergent strategy, right? Which has now been kind of adopted by, you know, Xi Jinping on a speech, like at a speech at an AI conference, like talking for 10 minutes about this thing. But this originally started because the Chinese models were not that good. Sure. So you had to open way. Yeah. The only way anyone would ever pay attention to them or even like try them out was by making them free. Yeah. And we're still there, right? Where like no one is paying for this stuff really, because as you said, on the sort of like cost capability curve or whatever, you can still get better stuff. And I think like, you know, OpenAI and Anthropic are very aware of this and are going to be pricing their stuff accordingly such that you don't end up having a big incentive to go over. And it's, you know, it's hard for those Chinese AI labs, right? Because they are working with less compute. They're smaller companies, they have fewer resources. And I guess to date, selling software in China has been really, really difficult. So when you look at the revenues of a Jirpu or a Minimax, and then you look at a revenue of an open eye anthropic, it's like a joke. It's like they made like $150 million last year or something like truly pathetic. So, you know, do they need to end up having a business model at some point? Maybe, maybe not. I mean, I sent you this, the Deep Seek CEO interview who is raising billions of dollars, but is basically telling all of his investors sort of the vision that Demis Asabas was pitching back in like 2015, saying, look, I'm running a science lab. It's gonna be like the Manhattan Project. If it works out, it'll be the biggest thing humanity has ever seen. Don't you talk to me about stupid business models. And, you know, he ended up having to make AI overviews for search at a certain point. Right. And but on the downside of that, like now he has to manage 6,000 people and deal with Satya and public markets and whatever else. So you have a guy like Liang Wenfeng in DeepSeek who's trying to build like a financial structure to make the bet of, no, just like give me 300, mostly dudes, dudes, and just let us make science and we're going to do a better job than these like giant hidebound organizations that are now stuck in an innovator's dilemma because they have business models. He basically says in that thing, products are for losers. Yeah. You know, the real winners, We just look at AGI and we're not going to be distracted by this nonsense of having a customer. Yeah. So it's the anti-business model business model. Totally. And look, you can go really far not having a business model if the public markets believe in you, right? Like Amazon lost money for 30 years. Tesla was like the biggest short for 20 years, right? And it's now worth, well, I mean, who knows what it is today. But look, like, like, it's not totally crazy to think you can sell people on an idea, especially if, you know, you're not Demis Esauvis in 2015. Like, you've proven that you can do worlds. You can deliver technological change, which, like, shakes the world. Right. So I wouldn't put it past. Which DeepSeek has proved as well. Which DeepSeek has proven. So I wouldn't put it past him in that perspective. There's also the layer of like, yeah, if the Chinese government ends up believing in this thesis that open weight is like, you know, kind of screws over America and is good for diffusion, is good for soft power, then, you know, having the state just like keep cutting you checks is another way to not have a business model. Yeah, so let's get to that. So, I mean, we're talking here about, you know, DeepSeek, on the one hand, raising money, on the other hand, telling its funders that it's not going to make any revenues, basically. Which gets to this issue, which applies to actually all of the Chinese labs. How are they economically sustainable? If they are producing open-weight technology, which means they give it away, the revenues may accrue to the cloud provider that serves the model, but they don't get revenue as the model designer. The algorithms are given away. So what the heck is going on here? How does the whole Chinese AI sector carry on existing when they don't seem to have a revenue model? Is this government subsidies? Is it billionaires cross-financing this? Like the deep-seat guy makes money from his hedge fund and he pours it into AGI for fun. How are they surviving? Well, no one has a good answer yet. But like basically what we've seen so far is investor interests, you know, people buying into a dream. And these are Chinese domestic investors or whoever? Yeah, well, yeah, the foreign money is basically gone, right? So it's Chinese domestic VCs. and then it's the current hyperscalers and like large companies in China. And by the way, it's like, these are like 15, 20 companies that are building models. Like Kling came out of a short video app that I used to work for called Kuaisho that make some of the best video generation models in the world. We have ByteDance. We have Meituan, which is a food delivery app. So imagine like DoorDash is having world-leading models. I mean, this is basically because the CEO is rich and he thinks it's cool. So like, yeah, we're gonna do it and we'll figure it out, right? I mean- I wanna click on Kling, you worked there. There's gotta be a story. Oh man it was just really dumb They had me like open I was like doing I was like the third foreigner there There were already 3 people They had me doing Brazil And I was like, guys, I studied Spanish, by the way, in high school. Like, I don't speak this language. Like, ah, you're a foreigner, you're figured out. I basically spent my whole time recording China Talk podcasts. And then they caught up to me at some point. But okay, so coming back to the, how does this work in China? I mean, it's not that expensive. It's pretty expensive. Look, it's not cheap. There is CapEx involved, but like the cost of training the models, especially if you can, you know, fudge around with distillation or whatever, these aren't like, it doesn't cost you like 10, 20, 50 billion dollars. We're not at those like God run things that we were talking about in 2023, right? There's another dynamic of the Chinese domestic market. Like, you don't, over the medium term, like, you are going, like, right now you still have a lot of Chinese consumers, like, using VPNs and these transfer stations to actually just, like, get Claude and ChachyPT tokens. But, I mean, that's a protected market. So, there's, like, revenue there, right? You may not have to open source everything or you can, you know, if you are the one who is supplying it because you're Alibaba. and you also own Olicloud, like, you know, that's good for business. So I think it's not obvious. Like people, but also look, two, three years ago, people asked, I remember there's a famous quote from Sam Altman from like 2021 being like, how are you going to make money? He's like, I don't know. We'll figure it out. And I feel like that's the same thing. It's a, if you can look over the pond and see trillion dollar companies being created, like, you know. You have a real story to tell. Right, right, right. Okay, so we've described sort of the Chinese AI private ecosystem, but let's talk about the government angle on this. And there was this speech just last Friday as we're recording by President Xi Jinping. And it's sort of, I forget whether your paper, when China gets a mythos level model. Yeah. Did you do that right before the speech, right afterwards? We've done like versions of it a few times. Yeah, we're updating it desperately. We're slowly updating. I mean, it's like coming back to that, like Trump pre-mythos, post-mythos thing, right? Who would have, you could have had all the exquisite intelligence of the world have read every single email that's coming in and out of the White House. Which I do, of course. Yes, there would have been nothing in there to indicate that, you know, Once a model as good as Mythos comes out, they're going to flip on a dime and Scott Besson is going to call up all of the heads of the banks and say, you know, this stuff is crazy. You got to lock it down and whatever. So given that we've seen that movie before, I do want to like put a bit of a, like I still want to keep an open mind as to the future of Chinese policy because what we've seen, What we've seen from 2018 to 2025 was kind of like benign neglect. We only really care if these models talk about Tiananmen. Starting in 2026, there was a little more focus on it. At the highest levels, you had a Polyp Bro study session about AGI, and maybe they're cluing in a little bit. They intervened to stop people downloading OpenClaw at one point. That was a small sign. Yeah, but it's still kind of like, okay, this is like a fun, cute thing. And then we have a big declaration from Xi at this speech saying, we think openness is good and we think it's like positive for humanity. It's positive for, you know, China's role in the world. We want to keep doing it. But there are outs, which you can read in that speech where he's like, we want this technology to be safe and controllable. And then he quotes a line from this like famous Chinese debate in 81 BC about like whether or not they should nationalize the salt industry. Debates of like salt and iron where the quote was basically like, look, you got to be smart about this. Like we can't really know. You got to be like flexible and nimble to base depending what's going on. So because they haven't hit, because these models, which are very good, have not crossed that April 2026 mythos, this stuff can hack anything if you just like press go and don't look at it over a weekend. I could see them getting freaked out. Right, right. So they could, just like with COVID, they went from lockdown to opening up super fast. They could change. Well, they went from, yeah, they went from this thing is fake to lockdown. super fast, right? And that's kind of the direction we're going on. I mean, you made a point in a prior edition of this show about like, won't the Chinese get freaked out when, you know, Xinjiang separatists and the Falun Gong get access to open weight models? I mean, doesn't seem so yet, right? Right, right, right, right, right, right, right. So I think I said at the top that the speech could be seen, Xi Jinping's speech could be seen as a sort of declaration of an AI version of a Belt and Road policy in the sense that the attempt is to provide, you know, AI for the world. By China, they want to be providing open models because that's very cheap and useful for both the global South, but for that matter for Europe. And if you pick this moment when much of the world mistrusts the United States because of the quality of its political leadership, I'm thinking of Europe particularly, you know, the Europeans are very keen to use Chinese models right now because they don't want to be fully dependent on American ones, given that the American ones were both were first, you know, allowed with fable and then disallowed when the Trump administration changed its mind. And it just made, you know, people felt, hey, I need an alternative to relying on the United States. And, you know, Chinese open models seem like, you know, a good hedge. So, I mean, do you think it's correct to see the speech? Like if you just read it on its face, kind of stipulating that they might change their mind later. For now, the policy seems to be, you know, try to just get the China stack out there, make as many people as possible use it, keep it open, put safety second. That's the policy. Yeah. Well, we should specify the China stack because it comes back to my first point, which I think is the most important one about how much compute does everyone have? Because, you know, you can debate the curves, right? But like, there are a few things I actually have a lot of confidence thinking about three to five years out, but you can really just count up the fabs and the tools and have an understanding of how much TSMC and Samsung and Intel are going to be able to print out relative to what Huawei and SMIC are going to be able to tape out over the next five years. And that ratio of somewhere between 10 to 13 to one is going to continue in the next five years. So even if the AI stack, like the AI stack that China will be able to export in the future is only going to be the model layer, not actually the sort of like cloud, the cloud layer that compute under it. Because like that, those chips are just going to go to not Chinese providers. So with that proviso, it's not really the China stack, but it's the China models. The idea is to disseminate them around the world as fast as possible, keep them open. Notwithstanding that language in Xi's speech around off-switches and so forth, there ain't going to be no off-switch. It's going to be open-weight. Yeah. Yeah. So I guess the next question is, China is pursuing a belt and road strategy on AI, But is that a smart choice? I mean, there are questions about how Belt and Road has worked out for them. You know, if the idea was to make friends and influence people, it turns out that, you know, you put Zambia in debt and afterwards they hate you. And you wonder slightly about how much leverage does China really gain by spreading its models around the world? People might use their models, but they've downloaded them. They have them on prem. They have them on a local cloud. What leverage does it really give China? Yeah. I mean, that's why the Belt and Road analogy is kind of weird to me, right? is like, okay, like if you're building a train and you're, or a road and you're gonna like charge a toll on it, you're gonna build a port and then that's gonna be where your boats do shipping from and it leads to more, like the whole point is, the whole point of Belt and Road was to lead to more economic integration with China over the long term, right? It was just like configuring trade routes and creating connections. But if all you're doing is like giving away a technology And yeah, you can like redistill it. You can do whatever you want with it. It doesn't seem like a particularly sticky thing. And then again, like making the models themselves, it's hard, but it's not that hard. And like, if you buy the thesis that we can't trust America, like it seems weird to go to that, from that to, okay, so we'll trust China. Like the answer is no, we need to do this ourselves, right? And maybe it's just as too much like collective action problems, like there's talent gap, whatever. All our good engineers are gonna go work for these American labs that can pay them more than like the sad, the sad domestic startup. But it just, I don't know. It doesn't seem to me to be like a stable equilibrium in the way where kind of giving up on Nokia leads everyone to just Huawei's arms. In a way where you're locked into Huawei as opposed to this world where you can just kind of keep trading things in and out. Right, right, right. So in other words, it's not a very good strategic move by China to give away these models in the hope that it buys them leverage. What it might do is work as an offensive weapon for just reducing the profits of the frontier US models. Well, again, you're playing with a tough hand, right? Because you have fewer chips. and by the way, like all those, whatever chips SMIC and Huawei are going to produce have more value to the PRC being deployed to run AI domestically than they do running AI in some like, you know, 4D chess, like let's build a data center in Zambia so then Zambia would like us, right? So I just don't think that there's going to be a lot of like excess compute that's going to be in the offing for the rest of the world. I mean, Europe may be the one exception because like if you can earn more money serving AI there than you can domestically, like that might have an economic rationale. But, you know, Belt and Road was exciting as a, I mean, and it's like, the dream version of Belt and Road was like to bring developing countries closer to China. Like I don't see how, So I just, I don't quite see the vision yet. Yeah. So let's try to put the two together now. We've talked about the US, we've talked about China, we're going to talk about US policy towards China. What strikes me is that, you know, the Trump administration having done its U-turn on AI safety with respect to domestic models, you know, the logical next step is if you care about safety within the US, you should also care about the safety of models coming out of China. Because if you hobble US frontier labs and say, you got to be very safe and very careful, and then there's a bunch of open-weight stuff coming out of China, which isn't safe or careful, you haven't really made anything safer. And, you know, you've just like slowed yourself down to no avail. And that is indeed the argument that sort of the deregulatory side of the tech debate in the US, that's the argument they make. So, you know, David Sachs, it's precisely, he says, you know, I don't believe we can slow China. Therefore, I don't believe we should try to slow America. But the administration has apparently chosen to slow America. That seems to be where they're going. So the logical corollary is you have to try to slow China. And yet that's not quite being said out loud. There's some, you know, press reporting about a September summit between Besant and his counterparts, Scott Besant, the Treasury Secretary, and his counterparts in China to get a discussion about USAI safety cooperation started. This is sort of, I think it's fairly characterized as rumor at this point. But there is no strong vision from the administration about how they're going to extend their domestic safety initiatives to China, notwithstanding the fact that unless you do extend it to China, the domestic stuff makes no sense. Do you agree with that? Well, okay. So I don't know if the domestic stuff makes no sense, right? Because like the theory of the case you have now is you want to have a lead and model quality in order to stamp out all of the bugs you can find such that when the Chinese open weights or whatever, the Chinese national security state can get its hands on, they are, you know, using those capabilities against much more hardened targets, right? So look, like if Mythos 6 is 10 times better, if like the next Mythos is better than this Mythos and it can find bugs and patch them even faster, then, you know, that lead is still really useful. Yeah, yeah, yeah. But you're articulating the David Sachs view here, which is a completely coherent view. Either I'm saying you race, you try to get mythos six as fast as possible, and then you don't care about Chinese open weight models because you think you can harden your own systems and protect yourself. That is a coherent view. What is not coherent is to be slowing down U.S. frontier development whilst not slowing down the Chinese. And that's where we seem to be. What makes you think they're not? Oh. Yeah, because right now they've told both OpenAI and Anthropic to chill it in terms of who gets their models They are apparently on the cusp of announcing pursuant to the executive order from a while ago some sort of upgrade of the national regulatory apparatus It seems like we are moving into a world of slowing release Oh, I understand. Okay, so like, so, okay, in the, like, your vision of like rational one is you're pulling every lever to make Western cyber the most, you know, Western AI do the most defense it can around this ecosystem, as well as pulling every lever to slow down the Chinese model development. Yeah, I think you either want to go slow and safely everywhere or slow and safely nowhere. Because if you just try to be slow and safe yourself whilst not making the other guys be slow and safe, then the threat from the other guys is increasing, but your own defenses are not increasing. Well, I mean, in this administration's defense, which is a phrase I say very rarely. Look, I think you've seen just over the past few weeks an acknowledgement, I mean, maybe a few months, an acknowledgement that or recognition that they do have levers to pull to slow down Chinese AI development. And obviously, there's like we've had a lot of hand wringing about should we export the chips, not export the chips. This distillation thing seems to really have pissed them off. The idea that there is like USIP getting stolen, that they're sort of like tailwending or piggybacking off what we have developed. You saw some tweets out of Kratios and Secretary Besant saying, we know, Kimmy, we're onto you. Like we're in your systems. We saw you distilling. But where does that go in policy terms? I mean, I think we'll see by the time, we'll see how fast this production team gets this out. But I got a feeling over the next few weeks, I mean, there's lots of, there's levers you can pull on. Right now, everybody distills everybody, right? Elon is admitted to selling one of the Frontier Labs. Sure. You know, Thinking Machines distilled a Chinese model. I mean, it's- Yeah, my brother can take $20 out of my wallet, but like someone on the subway can't. I mean, I think like, look, you can put them on sanctions lists. You can put them on enemy lists. You can file lawsuits. You can make sure that they get no revenue from the U.S. by making it illegal to do business with the likes of Kimmy or whatnot. Like, I gave Kimmy 20 bucks because I wanted to try out the model this week. Like, if that's illegal, that's a big thing. That's a, you know. Maybe, but we were saying earlier that their game is not really commercial anyway. Yeah, look, again, we're feeling towards the world in which you want to do the most cyberhardening America and make the Chinese models as bad as you can or like slow their curve, right? I think we haven't quite, or this administration hasn't quite pulled all the levers to live in that world. But let me put it to you like this. Okay, so you're saying there is a coherent policy which consists of the U.S. wins and China is like behind. And I guess I'm doubtful about that because at least in terms of frontier capabilities, as you said yourself earlier, the gap has shrunk. Yeah, well, the idea is two things, right? You have this structural lead in how many chips you have, right? So look, if we have the same, if the models flatten out, right? And Kimi's newest model, Kimi K4 is exactly as good as what Claude has. The MSS will have an order of magnitude or probably two orders of magnitude, fewer chips to try to do hacking. So that's what the MSS is. Ministry of State Security. I see. Chinese NSA. Yeah. Well, there are lots of other things, But like Chinese hackers, you know, gangs in Russia, the North Koreans will just have less firepower to shoot their models into hacking things than the world. And yes, there's like a balance and we'll see how that plays out. But I still think like it's not hard. You talk to cyber people and they're just like, look, it's going to be real be a really awkward two or three years. But we're going to end up at a point where you'll have you'll have the ability to sort of like monitor this stuff really well. And your code will get so good that it'll actually be like, you know, provably cleared. Right. So I think that is a potential world you can live in where you don't end up having an accord and you don't have. cyber apocalypse. Okay, so one question about that vision is maybe with cyber hacking, you can imagine a world where American models are so fantastic that we harden everything. By the way, you'd have to harden not just in the US, you'd have to harden like globally, because you have things called multinational companies, US ones, which do business in Korea and France and wherever. Intelligence too cheap to meter. We'll get there. Sure, why not? So we're going to harden all the French banks too, is the point. Yeah, fine. And a kind of America first vision here doesn't work. We agree on that. Yeah, or the American banks are the only ones that you can trust to keep your money safe. So then we have even, the dollar is just, is backed by the shield of American models. Well, my sense from talking to American chief executives is that when they, you know, hear the administration's policy, what they point out is, hey, we do business all over the world. I was being sarcastic. Okay, good, good. Okay. So one issue here is that, you know- I mean, well, staying on that for a second. Like the countries I would be worried about is the like, you know, the Nigerian central bank and the Kenyan central bank, where like maybe the financial, you don't quite have as much sophistication. There's still like, you can still extort a lot of money. You can, there is a piece about how Boko Haram was using these models to figure out how to set up their guys to raid a village or something. Okay, now nobody's going to sleep. But yeah, once the... I think larger, more functional governments will end up getting to the other side of this. Right. But look, so one set of issues around this vision of let's have America win and harden systems is that you've got to harden all the systems. Okay. Yeah, not easy. Second thing is, there might be other kinds of threat coming down the pike where the defensive option of hardening systems doesn't exist. And I'm thinking about bioweapons, right? The individual wetware human beings who wander through the shopping mall, you can't really harden them. So if the bad guys have used AI to build a bioweapon, there's no defense. But the pushback I get on this point is, well, actually, AI is not going to help build bioweapons. Is that your position or maybe I don't I don't have a good answer on this. I think like I think the the the the cyber risk is the one that is like so obvious that it's staring everyone in the face that it is like like I feel like it is possible to have mental models about the the virus one is I'm you should you should have another guest to to to dive to dive deeper in it. I mean, it's interesting, like, to what extent, again, because China, the Chinese system has, like, not really bought in. Or they, I mean, as of today, they see the calculus of the cyber stuff as something that they can deal with, right? Will they get to a point where they see the calculus of the bio stuff as something they can't deal with? Yeah, yeah. I don't know. And then also on a sort of, like, when you're talking about a negotiation, right? Like, either they're going to believe it or they're not going to believe it. And if they don't believe it, then like, okay, we're like giving them chips to do something to save humanity. I don't know. It's just, especially given how, sorry, I'm going to take us down. You're the interviewer. Please. So I guess, just to back out, what we're discussing here is whether there is a plausible strategy that makes America safe, which consists of saying, America is going to win this AI race because we have more compute and the Chinese will not be able to damage us and we will live a happy and secure life without needing to talk to them about closing down their open weight production cycle. And I'm just skeptical of that. I'm skeptical partly because there are certain kinds of capabilities that a future open source model might have, like bioweapons, where there is no, you know, however strong our AI is, I'm not sure it enables us to protect the civilians who wander through the shopping mall. I'm also skeptical because just on cyber, you know, you got to remember the famous, you know, terrorist line to Margaret Thatcher, which was, you know, when they blew up her bedroom and she happened to be in the bathroom so she survived, They put out a statement saying, you know, you have to be lucky every time. We only have to be lucky once. The attacker in most of these threat scenarios can try multiple times. It only has to work once. The defender has to succeed every single time. And so there's a fundamental asymmetry in these cyber warfare situations, you know, where the threat attacker may well have the upper hand. So I worry about that as well. So I'm just skeptical that we can move forward to a future where, unlike in the nuclear scenario where we had a non-proliferation regime, we're happy as Larry without anything equivalent. Yeah, two things. I would, I would like bound my comments on this of like over a two to three year window. Like I would prefer the worlds in which the well, I mean, excuse me, over over a 10 to 20 year window. I prefer the worlds in which the U.S. has more compute to just do whatever it wants as opposed to, in many futures, that is just like a better hand to have. On the cyber piece, one like interesting nuance of that is like CapEx of cyber used to be basically nothing. And now there's real CapEx involved in doing this. For the attacker and the defender. Yeah, well, I mean, in our example, for the attacker in particular. So you are going to need a data center. You're going to need, you know, multiple millions of dollars of these chips and to be able to keep them running. And I think that kind of like sort of at a certain point and as this evolves and the models keep getting bigger and bigger such that like your table stakes for being a cyber hacker keep going up. You know, hopefully we get to a point where it actually comes back to the nation states and then, all right, we're crossing our fingers that North Korea and Putin and Xi don't want to like do a, you know, kill 100 million people with a bioweapon or something. So that's a hopeful thing. But I don't know. I think we'll, like, the dynamic that needs to happen, and this is sort of what happened with climate change over the course of the 2010s, really, is the Chinese government just convinced themselves that it was a problem. And we're not there yet. And there may be futures where Xi is right. And all the doomsday scenarios that people have been freaking out about since they've read, you know, the culture series, like are not actually real. You're referring to the science fiction series. Yes. The Ian Banks series. Yes. Okay, just for people who don't know. But if he's wrong, like, look, this guy's changed his opinions on things. And it's much more, it's much easier to come to some settlement when both sides agree on the problem, right? as opposed to like, you know, looking at Iran right now. It's like someone wants a straight open, someone wants a straight close. Like how you, it's not that easy to solve that. Yeah, yeah. So you're basically advising me to be patient. You're saying, you know, I'm feeling anxious. I feel like we need to get a deal on controlling open-weight models out of China sooner rather than later because we're getting to a point where these capabilities are getting scary. and you're telling me, just cool it, you know, we'll see, the Chinese may come around to the idea that this is scary stuff. If it is objectively scary, they will see that. They're not dumb. And at that point, we can talk to them, but don't force it yet. Look, once Kimmy, deployed in China, does the thing that OpenAI's models did this week, that's going to freak people out. There's no way it won't freak people out. Once it, you know, like, oops, I just like released some, you know, secret party documents or something. There's going to be there's going to be some response function to that. If if it doesn't because alignment capabilities have advanced to the point that when they release a model that this is good, like you don't get those problems, then maybe we're in a world where it's OK. But and so that's whatever. Give that a 30 percent chance in the 70 percent chance where that's not the case. Like the Chinese state is going to respond in some sort of way. And once they kind of internalize the fact that these things are dangerous and need to be controlled, then like you may not even need your global summit. You may not even need to give any chits because they recognize that there's that there's risks to this technology. Or it's just a much easier conversation. And it's the sort of conversation that, you know, you have about global financial crisis where it's just like, yes, we're all on the same side. We want you know, we don't want another Great Depression. Let's like see how we can use use our tools together. And I think you're more likely than not to just get there naturally if it's something that's really needed. Okay, well, we should wrap up. I could talk to you forever, but this has been great. I've ended up feeling somewhat reassured that we'll get to a deal with China when we need to, or even without a deal, that we'll have a mutual agreement on being safe. Thank you for joining us on The Spill Ever. Okay, thanks. Want to stay up to date on the latest episode of The Spillover? Sign up to receive an email alert when new episodes drop at cfr.org slash newsletters or click in our show notes. If you have an idea or just want to chat with us, email podcasts at cfr.org. Be sure to include The Spillover in the subject line. This episode was produced by Molly McEnany and Gabrielle Sierra. Our video editor is Claire Seaton. Our audio producer is Marcus Zacharia. Special thanks to Justin Schuster and Todd Jäger for their recording assistance. You can subscribe to the show on Apple Podcasts, Spotify, YouTube, or wherever you listen to podcasts.