They want to see your Anthropic bill keep going up because they need to not go broke when the VC subsidy runs out or when they finally can do an IPO and cash out. The drive for Anthropic and OpenAI to try to survive to IPO. And that's a question because OpenAI has got a lot of debts coming due in 2026 that they put off for about three years and they're all coming due this year. But they're not supplying a compelling product is fundamentally the problem. Because if agentic coding worked as well as the boosters say, this would not be a question. the risk and then quietly handing a chunk of your profits back to the platform. This is where IG comes in. For investors who are curious about crypto, you can now invest in Bitcoin, Ethereum and Solana, commission free. You can buy, sell and swap across 150 cryptocurrencies alongside 12,000 global stocks and ETFs, all from one app. IG. Trade. Invest. Progress. Search IG. IG charges zero commission. Don't invest unless you're prepared to lose all the money you invest. This is a high risk investment and you should not expect to be protected if something goes wrong. Take two minutes to learn more at ig.com slash UK slash crypto. Hi, I'm Isaac. And today on the tech report is author and host of Pivot to AI, David Gerrard. Thanks for coming back. Good afternoon. So the cost efficiency of AI is becoming a bit of an impassable sticking point for the industry as enterprise customers start to question if they're getting enough from it to warrant the price at the same time that the average amount of tokens being used is increasing. If I can ask you to sort of embody an executive keen to use AI in their company, how close do you think we are to a tipping point where the cost of using AI is sort of outweighing the value it provides? I think we're getting there. They're just not admitting it because there's a key point, which is this stuff doesn't work very well. and it's sort of selling itself on the future promises. We don't want to be left behind, you know, and it'll be great tomorrow and next model, bro, and all those other things that people say when their thing is a bit of a failure. So it's getting expensive and the bills are coming due and with obvious things like Anthropic keeps ratcheting its prices up in a fairly obscure manner. Microsoft comes straight out and says, we're going to charge you 10 times as much now. And it's like, what? And that's the sort of thing that makes a CFO sit up and go, it's costing how much? You sure we can't do it cheaper? Hire a developer, perhaps, you know. So I think that businesses are thinking very much along the lines of, wait a minute, what are we getting for this? What are we getting for it today, not in six months? Can you just briefly outline or kind of spell out why modern AI use is so expensive and getting more expensive? Well, it was always expensive. That is, it's been sold as a loss since it launched. open ai and anthropic the largest vendors basically always were subsidizing the price from their venture capital funding the open ai price of 20 a month the first for the basic paid open chat gpt that was always a number that sam altman plucked out the air because it sounded like a good price to charge people, which is fine in the loss leader stage. But then it never quite got too profitable, or working well enough that people would dive in and use it anyway. So Anthropic will talk about how token prices have gone down, down, down tokens are so much cheaper now. But your cord bill doesn't go down. It never goes down, your cord bill never goes down, because that sort of bill doesn't go down. It only goes up. They're spending so much on tokens. They're trying to get... The trouble with your large language model is they're very much at the top of their S-curve of performance where all technology goes on an S-curve. It starts off slow, it accelerates, and then it flattens out. And then if you're at the top of the curve, getting incremental improvements takes exponentially more effort. there's a great open ai graph from 2024 talking about um the o1 model which i don't think they ever released it was a test model they literally showed this on the graph they would show that it would take exponentially more time to get small linear improvements and i went you're giving away the game guys and we're there right now so this there they can only eke out slightly less bad output by spending thousands more tokens per query as they get the AI to run or the large language model to run. They get the chatbot to talk to another chatbot. There's another 10 chatbots that review that. Then they try to construct a reasoning chain and so on and so on and so on. It's a lot of systems checking each other for slightly better results and a lot of slightly worse ones. And it's really hard to measure this stuff too, because what are you measuring? it's like all benchmarks are fake um like sam altman says that chat gpt 5.6 sol is 54 more percent efficient in agentic coding what's that mean what are you measuring you're putting the two digits of precision number on this claim and it's like our unicorns have 54 percent sharper wing feathers. It's what are you measuring? But it doesn't help that literally all chatbot vendor benchmarks are made up. They're marketing. They're not science, they're marketing. They will often pay for the benchmarks to be made. They will often have early access to the benchmarks. And it's largely nonsensical. It's like, it is not an independent science thing because it's being paid for by the vendors. It's marketing. And I don't know what they're measuring. It's not clear. And it's all the promotion is vibes based. I feel so much faster as a coder. And then there was one study that measured this and it showed them going 19% slower. And after that, no one did another such study. And so it's hard to say what's more efficient. there's guys who will swear this does wonders than that ai is a good coder now we all agree that ai is good at coding now then you get someone who's actually a developer who looks at the code and goes this is trash this is insane trash you know like when the code for clawed code itself the bot that writes all the code for everyone when that leaked uh there was a great thread on mastodon by johnny from neuromatch um and he went through the actual code itself going this does things that are 11 different ways and none of them work and you've got basically the code was a series of prompts to the bot with increasing elaborate ways trying to tell the bot not to screw up this time it was like absolutely hilarious bit of trying to make a really long prompt of all the stuff it's done wrong before. And it's nuts. So it's quite right that the people who pay the bills and sign the checks are going, what are we buying? What are we buying? Perhaps it's the future, but is it the present? That's a question that the AI vendors aren't so keen on people asking, because the present isn't all that great you'll have a lot of people going trust me bro it works amazingly i'm so efficient and they ask what you ask them what they've got and they call you a you know and it's not a convincing argument anymore i do want to get into more of the chat gpt 5.6 soul stuff but i just briefly want to go back to the the s curve you're talking about because i mean I think it's fair to say open source kind of models are catching up with the frontier ones. And it kind of makes... I'm not convinced of that at all. No? I'm not. Because this is the local, the open model, open weight models running locally. That's the big excuse that AI boosters use these days. Yes, yes, it sucks now. And we can't afford it. Everyone will just run local models on their PC. Well, firstly, no, they won't. You cannot get the same performance from running a model on your NVIDIA card that you can renting the API from OpenAI or Anthropic, where it's backed by a data center containing huge NVIDIA boxes, the sort that costs tens of thousands or millions and just as much to run and you can compete with that The performance is not there They are not usable for real work And anyone who says they are is being a bit forward Local models will not replace the APIs in performance for the near or medium future. Perhaps far off in the fabulous future, but show me, I'll say. it's um you can do some jobs maybe but then you you could do use a local transformer model for those and we're getting a bit in the weeds there technically but jobs that are not like a coding assistant if you run it locally you can but frankly it's a nerd toy you know it's something you do because you're interested and you want to see what it does i know people who do this you know i'm not too worried about that but it's not systemic and it's not an answer to systemic concerns like this stuff costs so much and the prices are only going to go up much further um it just isn't an answer to the question i local models try and have fun with them but no they're not catching up there's no evidence they're catching up and that's just the thing that boosters say in my experience they're they're not up research them by all means but they're not going to literally replace your chat gpt subscription what about though for for enterprise customers and things like that who will have access to server racks and and that sort of stuff that would be able to then look at their ai spend and their subscriptions and whatever else and say well actually if we get an open source model and we run that on our infrastructure we pay for that infrastructure we pay for the maintenance of that does that not create a more viable sort of uh challenge to this well what is a big part of of current ai market is is the enterprise space maybe for some purposes but not for purposes of cost i've um i mentioned before i've spoken to people who've done this and there are there's a business in doing this the business is people who don't want to uh send their data to california for good reasons but somehow they think they still need a chatbot. The claim is that they can get chat GPT Pro like performance, renting serious Nvidia servers at AWS, if you've got those in your region, and you can book them and so forth, at a cost of something like $87.50 per user per month, instead of 200. It's a 40 odd times as much. Now, that's not a price advantage, because that's the unsubsidized price. And those, the API prices are subsidized, they're heavily subsidized. I don't know how much buy because divining the cost is like, look like reading tea leaves. You know, like, we have Ed Zitron writing 15,000 words on exactly what I've managed to diagnose from these guys. And he can get to, it's about this much. Yeah. And I love Ed's work, by the way. It's fantastic. Everyone should read it. But it's really expensive. And you are not going to beat subsidized APIs with unsubsidized local models. If you spend a lot of money, you can get something the user says is equivalent. Maybe it is. Maybe it isn't. You know, I don't know what you're measuring there, but it's expensive. Maybe you can do it cheaper than $8,000. Probably you can if you do something else, but it's going to be expensive. My finger in the air guess is about 10 times as much as the subsidy level that we're at with the API providers. So local models are not going to be cheaper. They are not. if the commercial models go up to 10 times the price people will look at local models or they'll think about what if we didn't use a chatbot because that's always an option you can always decide what if we don't use the chatbot so if we do go back to the like you say olman saying that 54 percent more efficient in agentic coding i think broadly that's supposed to apply to sort of the well agentic coding is the idea of having llm sort of carry out ongoing tasks rather than singular prompt tasks that sort of spit out whatever code or whatever it might be but putting that sort of into context what does the 54 improvement really mean in yeah the context of an always-on looping agentic ai use case which never it just is constantly using all the tokens ever i've got no idea he didn't specify what he was talking about and um what is being more efficient is it churning out more lines of code you know like typing was never the problem with coding it was always thinking but the bots don't think they just churn out code and the whole model of where you get a bot to supervise another bot supervising another bot, it's really obviously marketing for selling as many tokens as they possibly can. You know, like a token is a word fragment, and it processes more word fragments. And somehow that's become the billing units. And it's just the imaginary number, they could not use that billing, they just use that billing, because it seems like a number and they want to pretend it's real. so but it's obviously just a way to sell you more computation and then bill you for it per token and does it work well what do you mean work you know it's do you have better software well no ai code is notoriously fragile and buggy can you fix it with the bot that wrote it no you end up with something that's a little, that's sort of fried? Can a human dive in and fix it? Review the code? No, humans cannot review more than about 200 lines of code and then their attention runs out. If a bot turns out 10,000, they just go, you know what, my boss told me to use this. Looks good to me. You know, I mean, it's, I don't know what he's measuring there unless he like gives what he's testing and then I haven't seen it, but I expect it will be another benchmark where I'm going, wait, what are you even measuring? All of this is market. If you assume that every word out Sam Altman's mouth is marketing, I mean, obviously it is. That's his job. He's the CEO of an AI company. His public relations is the other half of his job from running the company. What do you make of the AI agents creating a subagent thing? because I must admit when I first heard that idea of letting your bots create other bots, which can then create other bots, the cost of that is going to be insane. It's potentially infinite and then presumably the hallucinations are compounding on top of that. It's amazing stuff. It's like one of those incredible marketing things where people who are formerly well-respected engineers post like they're on cocaine or something. And it's just wild. And, you know, as the finance industry learned, cocaine does not make you a financial genius. It does make you think you're a financial genius. And the same applies to bots and coding. It doesn't make you the best developer in the world, but it sure makes you feel like you are. And that's why these guys act like you're trying to take their drugs away when you say maybe the bot is not all that so it's obviously a marketing scheme for selling more compute the thing they sell they want to sell more of it of course they do that they're an enterprise sass business they want to sell more of what they sell so they say oh you if you don't go into churning through all the freaking tokens in the world and spend hundreds of dollars a day just on Anthropic, then you'll be left behind. And at that point, in most people, a little red light would come on going, maybe I should be left behind. What's the cost to benefit here? What am I getting out of this? And Claude Code is quite successful as a piece of software, and it's all vibe coded. And the code itself is sort of rubbish and weird, but also it's, I say about AI, the use case for AI is jobs that should not be done. And I think that that might well be one of them where it's basically a bunch of, as I said, increasing elaborate prompts to try to tell the bot not to mess up this time. And what are you even doing? You know, and people forget, I keep seeing people forget that they can code without the chat bot, where basic things like, wow, what if we take this code written in C and we use the chatbot to turn it into machine code that can run directly? That's called a compiler. And it's like, what are you even doing here? I'm using the chatbot to sort out whitespace in an XML file. That's a one-line thing in Unix. What are you doing? And they've forgotten the most basic things. These are people who did a computer science degree and they forgotten what they were taught I expect a pretty horrifying crash when the prices go up and the venture capital subsidy runs out I say when on that it will run out Will there be a sustainable business after that? I won't say there won't be, but I think it'll be much, much smaller. I had a good example of the use case of AI as well, where they had to cut back on the people in the office basically just using AI to convert PDFs, which really, yeah, that does kind of make sense. It's a job they can do. The thing that the chatbot sells, and this is important, and it's an actual thing it sells, is convenience. Exactly. A small specific transformer model for translation will do a better job than a chatbot will because the chatbot will know a lot more words, but it also smooths out language. So the translation sounds better, but also it's wrong, which you might think was the point. Don't be wrong. But, yeah. And some people refer to the smoother translation, but it's wrong, you know. But it's also very convenient. You can translate right at the command line. You can scan a PDF and do an OCR on it right at the command line. You can transcribe audio right at the command line. You can draw me a picture of Taylor Swift in Studio Ghibli style right there at the command line. It's so convenient. It's hard to translate into real work sometimes. Converting a PDF, like what? We have programs for that. We have what's called computer programs. They work quite well. Anyway, we should move on from that. There was Paolo Alto CEO, Nikesh Arora, said that the AI pricing needs to come down by 90%, 20% this year, 90% by next year. And what kind of breakthrough is needed to achieve that kind of price reduction? Or is it kind of the simple fact that the compute is just too expensive for business models like OpenAI to make sense? Well, I'm sure he says that because he's the customer. but also I think that he's being very hopeful there. I'm sure it'd be lovely. And, you know, I know that Anthropik would love the cost of generating this stuff to come down because then they could spend 10 times as many tokens anyway to try to make it a little better. But it's wishful thinking because this stuff's expensive. No one's going to accept chat GPT-3 level output anymore. they're not going to accept gpt3 level output um it's like humans rapidly learn the tells of particular chatbot models um because that is what we are we are discernment machines that's the purpose of intelligence is telling good from bad and we're very good at it we're extremely experienced so you rapidly learn writing styles and so on there are some ai bros who think that writing styles don't exist and you can't possibly tell a given writing style from another one but you sort of can if you like write with if you brain write then um you can certainly learn it the um people won't accept the lower performance models anymore because they'll accept a model they haven't worked out how it sucks yet i think is what the uh why the new models keep coming out and then people get disappointed in those as they realize, oh, this is limited too. Also, remember that training is ongoing. Most of it's training after the fact, trying to fine-tune the model so it doesn't annoy people in particular ways this time. And I guess that's real work, but some of it's as crass as we've put in a special module to count the R's in strawberry, but it can't cope with blueberry. so he wants the cost to come down but of course he does but it's wishful thinking because the cost is not going to come down um they're not going to cut the api price for chat gpt pro from 200 to 20 that's not going to happen they can't afford for it to happen um so of course they won't do that but also they can't afford to do that unless they're effectively giving it away free they're giving away close to free you know it's maybe 10 times the cost to provide it and the price the 200 dollars is a sort of token charge on top but even if it feels a lot to you so i think he's very hopeful so if we if we do assume that a 90 reduction whether or not it is possible or would would be put through if that's what needs to happen where would that leave the the ai builder all of these data centers being built if suddenly everything is 90 more efficient and therefore 90 cheaper and now you have 90 more data centers than you needed is is where what what problem was problems would that cause well we're sort of there already um remember that grok xai built out two huge data centers and powered them with gas turbines pumping out nitrous oxide and causing a massive rise in asthma in the local people who were all poor, which is why they built them there, because they thought they could push them around. So what's happened is there wasn't that much demand for Grok. So he's renting them out to Anthropic. And so I hear to Google. And it's like, there aren't so many people wanting this stuff. They're like building the data centers first and then finding customers for them. So a lot of this, I've seen a great thing recently, I can't remember who wrote it, which describes the AI bubble as a real estate bubble. And that's very plausible. A lot of it is real estate deals, where you do deals through deals through deals and there's a data center involved somewhere. and none of this stuff will get built in three months or six months, remember? A one megawatt data center takes two years to build. Remember the days when one megawatt was a large data center? 10 megawatt, at least two years. Hundreds of megawatts, more than two years. Gigawatt, there are no gigawatt data centers in the world. They don't exist. There's a lot that say they're going to be gigawatt data centers. They've switched on maybe a few tens of megawatts, if that. but they've got great ambitions you know a lot of it's building stuff in the anticipation that they will come and or trying to get planning rights in the anticipation you'll get to build this thing in 2028 and maybe it'll come and maybe it'll all happen it's very forward-looking isn't that a great word forward-looking forward-looking statements forward-looking statements are when a CEO can just make stuff up. But it's not a lie as if you say, this may not happen, it's the future. So there's a lot of this stuff. The whole point of the bubble is gigantism. Gigantism is the moat. OpenAI's moat was that they spent so much on electricity and so much on training, no one could compete with them. That's a feature. that was their pitch to their investors no one can beat us on this we are so big no one can beat us anthropic did a good job but also anthropic was a schism from open ai um but and they were doing much the same thing as the competitors spawn from the big guy um open i is still much bigger but anthropic is number two and everyone else is trivial so including gemini by the way for all Google's attempts. This was, by the way, why DeepSeek was such a shock to OpenAI and to the tech market and the NASDAQ. In January 2025, DeepSeek came out telling what I can comfortably call straight-up falsehoods about spending nearly no money on training. That's not true. They spent less than OpenAI, but then again, DeepSeek was going for optimization. OpenAI is a machine for spending money as fast as possible and not for optimizing it. So DeepSeek spent less, but not game-changingly less. But what DeepSeek did was they sold their API cheaper than OpenAI's, and you could just change the endpoint because everyone talks OpenAI protocol, and you would suddenly have much cheaper chatbot. um so that's what that that was deep six advantage they're still cheaper um you're dealing with oh no it's the evil chinese ai i suppose the evil american ai so pick which one you want or you could use mistral but nobody likes mistral i mean in less than six months or so we've gone from token maxing maxing to basically token austerity if i can use that phrase and sort of the the attempts to move customers onto usage-based pricing pretty much failed i think it's fair to say it's still being done in areas but no one's happy about it and people are budgeting extensively and i just want to get your take on on that kind of terminology because people talk about token or usage and that kind of stuff quite casually but they all they all sort of euphemistic terms to refer to the price needed to make AI development a functional business model And the fact that it's talked about so casually kind of alarms me that people are saying, yeah, well, usage-based pricing, it's not really working out. So you mean this being a functioning business is not working out, is what you're saying, rather than, oh, it's just the token price. What's your take? I was actually surprised when Sam Altman walked back some of the price increases on OpenAI because the GPT customers were so outraged they were quitting. They were thinking, because they were asking the question that the AI vendors need the customers never to ask, which is, do we need this? Do we actually need this? And this is the question, what are we getting for this? and times are tough and even people who are doing well worry a bit so they're going to look at costs and go do we need this is it working and if the answer is we're not sure they're going to say maybe we need to limit our token usage or where you get the hilarious spectacle of companies going we're all in on tokens we're all going full ai by the way you're on token limits and what are you supposed to do? We still want you producing 10 times as much with no tokens. That isn't going to go well. It's like there's only so much that even the most cynical engineer can do in those circumstances. So they could offer cheaper models, but then again, their whole point is to sell you more stuff. they want to see your Anthropic bill keep going up because they need to not go broke when the VC subsidy runs out or when they finally can do an IPO and cash out. The drive for Anthropic and OpenAI to try to survive to IPO. And that's a question because OpenAI has got a lot of debts coming due in 2026 that they put off for about three years and they're all coming due this year. So yeah, I roughly predicted it would go down around sometime next year. That's a very squishy prediction that isn't much of a prediction, I admit. But it's like there's a whole lot of things converging on that point. So I'm going to feel a little bit right when it happens. And I'm pretty sure it'll happen. At some point, the subsidies will run out. And I'm going to call that the end of it. So they don't want that day to come. They want you to keep paying more and more, but they're not supplying a compelling product is fundamentally the problem. Because if agentic coding worked as well as the boosters say, this would not be a question. The fact that it's a question means it isn't as good as it's painted and isn't doing the job it's painted as. That ridiculous thing a while ago, the SaaSpocalypse, where all the stock prices of software companies went down. That was ridiculous. Like everyone hates their enterprise SaaS vendor. Everyone. Because enterprise software as a service has no incentives to be any good whatsoever. It all sucks. It all is about abusing the customer and making your bill go up. There are good companies, but not many of them. Mostly they work on abusing the customer just enough to keep them paying. So they turn to another enterprise SaaS vendor. That's Anthropic. and they say, save us using your AI tokens. We'll write a new version of Salesforce in-house on my PC. Well, no, you won't. That's ridiculous. It's not a wrong thing to want, but it's not a reality-based thing to expect. And so a lot of people went into the stock market, went, this is stupid and bought up those stocks cheap and made a lot of money from other people being dumb, which is a fine and noble art you know i i can't i can't question that one people were being extremely stupid much stupid than they should have been and other people who weren't quite as stupid cleaned up from rich people being foolish that seems like a good time to talk about meta's ai business plan um what business plan well well supposedly they're selling it for just cheaper than everybody else is is that's the rumor that's going around is that they just gonna put out their model and say it's cheaper than everybody else's therefore you should use it because we're gonna subsidize it subsidize it with all of our ad revenue from facebook and whatever else and i heard some analysis kind of kind of analysis that sort of the future of ai will look a lot sort of a lot more like how air transport airfares are now there's zero loyalty sort of incredibly high competition and rock bottom pricing as a result just so you people come in through the door and you charge them baggage later maybe well would you agree with that sort of prediction that it's just going to be a bunch of people like a bunch of crabs in a bucket just trying to pull the other people down or do you think that like there i've heard people talk about model lock-in and that sort of thing might shape the industry in a different way i think those are both very loose analogies with that don't really work out when you look at how ai actually works like the air travel one assumes that there is a version of air travel which doesn't just hemorrhage money and that's probably true you could charge more you can't charge more because people won't accept it but you could and like something like private jets certainly do charge chartered jets, whatever they charge accordingly. But yeah, with AI, though, it's all subsidized, it's all money hemorrhaging, and there's no version that does not hemorrhage money. So you can go cheaper, sure. You can undercut them. DeepSeek successfully undercut open AI, and they still undercut them. So they have to scaremonger about China to keep people from trying to use it but um a lot of companies are still looking to deep seek instead because it's cheaper they go well it's chinese so what we have perfect trading relations with china that's just fine you know and you know it's business so with meta's ai it's important to remember meta has never shown signs of any coherent plan with their ai ever it's as coherent as their plans were for the metaverse the 80 billion odd dollar product that nobody anywhere wanted and that now sells some apparently quite nice headsets to gamers the oculus headsets are reasonably popular beasts because they do the job and the people who use them like them and say this is a good headset I like it, but that's not worth $80 billion. How do they even spend that much? I don't understand what they could have spent all that money on. With the AI, you can spend all the money you like doing training and burning tokens and stuff. And Meta is currently selling its sort of disreputable camera glasses to I think their actual pitch there is to try to get more training data. They don't have any actual solution or way to make this useful, so they think more data, that'll fix it. And it's a testament to how far you can take a bad idea if you are the second biggest advertising company on the internet, which all of that money, Meta's ad business still makes money hand over fist. And unfortunately, that lets them get away with a lot of really bad ideas that anyone else would have killed. I don't see Meta's AI being that useful. It's another chatbot. I don't know if Llama is that amazing. People mostly use Llama because people know about Llama because Open Llama leaked and Facebook went, you know what, fine. And now it's the basis for everyone playing with local models. And again, I must stress, that's not systemic. That's a hobbyist thing for people who are into playing with weird computer things, which is good and noble, but it's not systemic. So it's not going to change the world in the way that, say, selling chatbots to everyone does. So I don't know what Meta is doing. They don't know what they're doing. I think they are floundering, and I don't know what they will sell or who will buy this thing. I'm sure someone will, but I don't know that anyone loves it or will do much with it. I guess we'll see. But even Meta is limiting its engineer's access to tokens after telling them to use as many tokens as possible. But Meta's staff relations are, there's a lot of stories that Meta's staff relations are not great right now. It turns out if you lay everyone off and treat them not well, they are sort of disgruntled. and I'm surprised. They're very well paid still, but that only goes so far. Well, David Gerard, thanks for taking the time. And thank you very much. If you enjoyed today's episode and you want to hear more of The Tech Report, please consider liking and subscribing. Also, you can get episodes of The Tech Report wherever you get your podcasts.
