Power ranges: AI faces supply crunch

Why get aaa? Because bad roads, bad tires and bad luck. Out of gas, locked out or in a ditch. Stuff happens with 24. 7 roadside assistance AAA's got your back no matter whose car you're in. Join today@aaa.com AAA expect something more. Roadside assistance is provided by independent facilities contracted by aaa. Coverage in taxis, limousines and other ride sharing conveyances is excluded. 2024 AAA US markets rack National Surveys offers terms, Conditions and policies are subject to change without notice. Visit AAA.com for details.

0:00

What if I told you there was yet another tool where you could get surface level data insights in static uninformative dashboards? There are 170 of these products and luckily for you, we're not one of them. Hex is a new platform for working with data. We combine deep analysis, self serve and trusted context in one platform with purpose built AI tools for data work. Over 1500 teams like RAMP, Lovable and Anthropic use Hex. Learn why at Hex AI.

0:30

The economist.

1:06

Hello and welcome to the Intelligence from the Economist. I'm Rosie Blore.

1:11

And I'm Jason Palmer.

1:16

Today on the show, a new leap in sophisticated scams and why all birds is stepping away from shoes. But first, It may not have escaped your notice that AI is everywhere in Silicon Valley. Techies apparently like to do something called token maxing, a race to burn through tokens, which are the fundamental units of data, usually a word or symbol that power AI models. They are of course trying to show off just how much artificial intelligence they use, and it is a lot. Open Router, which is a marketplace for AI models, reports that from January to March, the number of tokens processed each week quadrupled. But there's a problem. The industry that supports AI isn't expanding at the same rate. Could we be facing a looming supply crunch?

1:20

The AI companies are actually struggling to keep up with demand, and in recent times they have actually been throttling access to some of their tools.

2:35

Shahlesh Chitnis is a global business writer.

2:44

So for example, Anthropic, which is a large AI model maker, recently changed the terms of its service so that it dissuades a lot of its users from using capacity at peak times. OpenAI, which is another big AI company, shut down Sora, which is its video generation tool, because it wanted to allocate its scarce computing resources towards more lucrative ventures. So across the entire technology stack, AI companies are looking at different ways in which they can meet the demand with the amount of processing power that they have.

2:48

So you talk about processing Power. But what is there actually a shortage of?

3:21

Yeah, that's a good question. So if you can think about it, AI models need really powerful processors, which are called GPUs, graphic processing units to be able to run. And these are placed in really large data centers where you have thousands of these GPUs lashed together using optical network and all kinds of devices. Essentially the tech world is running out of processing power to be able to enable the kind of demand that we are seeing. And the reason this is important is because inference, which is when an AI model responds to a user query, every time you are using an AI tool, it's using that processing power. So the more people that are using AI, the more computing resources you need. And since AI demand is growing exponentially, processing power is struggling to keep up.

3:26

So what are the AI firms doing about that?

4:13

I think most importantly, they're just pouring vast sums of money to build out these AI data centers. To give you some numbers, this year, five of the largest cloud providers in the us, companies like Amazon, Meta, Microsoft, they are spending close to 700 billion in building out AI data centers. And it's not just these big companies, even model makers like Anthropic and OpenAI, which are really well funded, they are splurging on different deals to be able to get access to these resources. So the answer for the tech companies actually build, build, build a lot more.

4:17

So throwing money at the problem seems to be a sensible solution. Will that work?

4:53

I think it'll work eventually. The problem is building out this capacity takes time. Especially now in the us a lot of these data center construction are running into local opposition for various reasons. Citizens are concerned about its use of electricity, land usage, water usage, so on and so forth. So a lot of these data center constructions are running into opposition which is delaying their bills. I think the more pressing problem is the tech industry is just running out of kit to equip these data centers, which seems really odd, but that is true. So if you think about these really old tech instruments like transformers and switches, which I think we've never even thought about, literally the tech industry is running out of them and lead times for some of them stretching between three to five years. So that's one problem. The second problem is the processors themselves, the GPUs that I talked about, companies just can't make them fast enough or make enough of them to be able to supply these data centers. So if you go across the entire so called tech stack, which is the different layers of technology that enable AI, there are Shortages that are showing up every day everywhere. And that is what is making this particular supply crunch more concerning.

4:58

So if there's a shortage in the supply chain, essentially what are those suppliers doing about it?

6:10

So the suppliers are definitely starting to build a lot more. But I think we need to keep in mind that software and hardware move on very different timelines. We are all used to great improvements in AI every few months. In hardware, it actually takes anywhere from two to four years to build out excess capacity because you actually need to go in and build a physical plant that builds these things. So there is a fundamental disconnect between the pace of software and hardware. So that is one problem. The other problem is, particularly in the semiconductor industry, there are some really critical choke points where key components are controlled by a few companies. So the first one is of course Nvidia, which I'm pretty sure at this point everybody has heard of, which is the world's most valuable company. They account for over two thirds of the world's AI processing power. So a hugely influential company. Now, chips from Nvidia are essentially sold out. And it's so severe that a lot of companies are actually resorting to using chips that are really, really old. And by really, really old, I mean two or three years. In tech terms, they're actually using those chips, which is unheard of. So that's the first problem. The other choke point in the semiconductor supply chain is actually making these chips in chip factories that are known as fabs. And today there is only one company, which is called tsmc, a Taiwanese chip manufacturer that basically makes most of the AI chips used by the tech industry today. TSMC has been expanding capacity. Its own capex capital expenditures are increasing by 60 billion this year. But again, it's still not as much as companies would like, so.

6:17

So presumably all of that is frustrating the AI firms who are seeing surging demand and now can't meet them.

7:54

Absolutely. I mean, they've been pretty vocal about it. So Sam Altman, who's the founder of OpenAI, there's a quote from him where he says that TSMC should just build more capacity, which kind of explains his frustration. Elon Musk, who's the boss of Tesla and SpaceX, his solution is to simply go out and build his own fab. And not surprisingly, he's called it Terrafab. And his ambition is, by 2030, to build a fab that'll have more capacity than all the current fabrication plants put together. And to give you some numbers, some analysts ran a back of the envelope calculation. That would require anywhere between 5 to 13 trillion, I mean trillion in capital expenditures. So clearly we are talking about huge sums of money and it's probably not going to happen. But that just tells you the scale of the problem and the frustration that a lot of the software makers are feeling right now.

8:01

Shahlesh, I'm interested. As you say, we've got used to AI improving so much every few weeks, every few months. And of course there are concerns about AI nicking our jobs. How will this supply crunch actually affect these developments of AI?

8:48

I think there is a danger that the longer the supply crunch continues, the more pressure it will put on the firms to raise prices because we have been used to a world where prices of inference keeps dropping basically every six months. But behind that is a fact that a lot of companies are actually burning cash to be able to enable that. Now if the supply crunch continues and prices continue to rise, that's going to increase pressure on these firms to raise prices potentially. That could then slow down adoption as well. And so there have been those that have called the supply crunch a quote unquote natural break on this reckless AI spending. That's one view. The other view is this could potentially slow down AI adoption in a very real way if it continues.

9:07

Shailesh, thank you very much for talking to me.

9:52

Thank you for having me.

9:54

Why get aaa? Because bad roads, bad tires and bad luck. Out of gas, locked out or in a ditch. Stuff happens with 24. 7 roadside assistance AAA's got your back matter whose car you're in. Join today@aaa.com AAA expect something more. Roadside assistance is provided by independent facilities contracted by aaa. Coverage in taxis, limousines and other ride sharing conveyances is excluded. 2024 AAA US Market Truck National Surveys offers terms, Conditions and policies are subject to change without notice. Visit AAA.com for details.

10:13

What if I told you there was yet another tool where you could get surface level data insights in static uninformative dashboards? There are 170 of these products and luckily for you, we're not one of them. Hex is a new platform for working with data. We combine deep analysis, self serve and trusted context in one platform with purpose built AI tools for data work. Over 1500 teams like Ramp, Lovable and Anthropic use Hex. Learn why at Hex AI

10:43

Foreign.

11:15

Covering the scam industry for around three years now. But I've always thought that calling it the scam industry wasn't quite right. It's bigger than just scams.

11:22

Su Lin Wong is the host of Our podcast series Scam Inc. Which last week won a prestigious Peabody Award, which, if you haven't heard of it, is like the audio version of a Pulitzer Prize.

11:32

I came across a story recently that shows just that I heard about a spyware attack. One that illustrates the very, very concerning leap in the capabilities of the criminals I've been following for the past few years. I spoke to a victim. I'll call her Amber. It's not her real name. She's 52 years old and works for an accountant for several charities in Indonesia. And at the start of last year, she received a message from someone claiming to work for Indonesia's tax office, asking her to update some basic information. It was a very mundane kind of message. She's been an accountant for a quarter of a decade, so she's on alert sometimes for possible phishing messages. She texted this person and said, let me check and I'll get back to you. The person on the other end of the phone purporting to be the tax office got back in touch about an hour later and was like, have you figured this out? We really need your information. And gave her a link to download. So she downloaded this app that looked exactly like the government's own tax app, created an account. But what she didn't realize was that the software was collecting all her biometric data, and it had also gained access to her photos, her camera, her microphone, her contacts, her notes app, and her bank accounts. While this was all happening in the background, she couldn't see it happening on the phone. It just looked like the tax app was running. She remembered she had a friend who worked in the local tax office and so texted her. The friend immediately wrote back and said, stop what you're doing. Its malware. But it was too late. Two bank accounts had already been emptied and withdrawals were starting from a third. And as is the case with many NGO workers in Indonesia, those accounts were actually in her name, even though it was money from the NGOs that was stolen. Roughly more than 450 million rupiah, which is around 26,000 thousand US dollars, which is just a huge amount of money anywhere. But especially in a place like Indonesia, where that sum could have paid for the salaries of 10 employees plus electricity, Internet or utilities for a whole year.

11:45

That really is the stuff of nightmares. Just a bank account emptying while you're watching. What does this represent, though, as you say, about the sort of expanding criminal capabilities of the scam industry?

14:20

For years, the online fraud industry mostly relied on slow, labor intensive methods like the ones I focused on in Scam Inc. For example, investment cons, romance scams, and people. And in those kinds of frauds, victims are groomed, often over several months before being robbed. But what happened to Amber happened in minutes, if not seconds. And Amber's far from the only victim of these new Trojan schemes that are enabled by rapid advancements in tech and AI and really mark a change in how criminals are approaching cybercrime. What's really scary about this isn't just that it happens so much faster than a slower burning scam. It's that once the criminals got access to Amber's phone, they were able to target everyone, in theory, in her network.

14:31

But the idea of a cybercriminal sending links that are clicked on by someone who's not paying close enough attention is nearly as old as the Internet itself. How is this different?

15:26

This is all part of a global malware as a service operation where the criminals don't actually build the technology themselves. Instead, they buy it from messaging sites like Telegram. And so they're not just buying the malware itself, they're also buying stolen data. They have quite personalized information. Because all of this is being sold on the dark web or on messaging apps, this is likely what happened to Amber. It's likely that her tax information was sold to the criminals, who were then able to send her a very convincing text message. This malware is being constantly updated, so it's more efficient and better to evade detection by antivirus software. It's operating in lots of different languages. AI is making that even easier. And it's sold by Chinese speaking vendors, mostly to transnational criminal organizations based in Southeast Asia, the kinds I've been tracking for the past few years. And these gangs are running vast, vast businesses. Some of my reporting suggests that every year, this industry as a whole perhaps brings in US$500 billion, which is the equivalent of the global illicit drug trade. So about a year ago, Infoblox, a security firm, noticed this huge surge in suspicious Internet activity. And it led them to uncover this specific software cluster that targeted Amber as well as many other people. And the firm has since identified hundreds of domains that have targeted people in more than 20 countries.

15:35

So given the pace of change that you describe, it sounds like this problem is only going to get worse.

17:12

Yes, I think you're completely right, Jason. We're already seeing new forms of spyware being developed. So the criminals are trying to figure out what to do with all the stolen photos and messages that they now have access to. And Infoblox found signs of AI chatbot integration, deep fake voice tools, and tests of ways to evade facial recognition systems. So we should expect that the criminals are going to ramp up their AI usage. At the moment, it's mostly been countries in the developing world who have been targeted, along with Japan, South Korea, Spain. But experts I spoke to said that they expect to see this kind of malware target people across the US and Europe within the next year.

17:17

And when you look at cybercrime like you look at any other kind of crime, I suppose it's a bit of an arms race between the bad guys and the good guys. What are the good guys doing? What's going on on the enforcement end of this?

18:04

So the good guys are doing a lot, a lot more than they were doing when I first started covering this beat. We've seen the US government come out and say that they were going to try and crack down on this industry. Governments across Southeast Asia are very, very aware of the huge harms caused. The problem is that the criminals are making so much money that they're incentivized to find workarounds to these government crackdowns. And the other issue is corruption. There are many, many countries where corruption is endemic and the criminals take advantage of this by off the political and business elites. So it's not really as if one country can clamp down on this and the whole problem will go away because the criminals will just move to another country where it's easier to buy off the political and business elites. And so these kinds of banking trojans and malware are just the latest business line of these really sophisticated criminal networks.

18:14

As ever with your newish beat. Su Lin, terrifying stuff. Thanks very much for your time and congratulations once again on the Peabody.

19:11

Thanks very much, Jason. If anyone's curious to really understand the scale and sophistication of these criminals, you can listen to Scam Inc. Wherever you get your podcasts.

19:19

I had this idea for a wool shoe.

19:50

Tim Brown, the chief executive of Allbirds, used to tell the origin story of

19:53

his company and a series of really bad looking prototypes. How would we make it? Could we scale it? And I was, oh my gosh.

19:57

He also shared the advice he got from his business school professor.

20:05

But I'm concerned that maybe you're going to be one of those guys that tries and tries and tries and then never stops trying and it's going to be a failure. So fail quickly, like put it on Kickstarter or something so it can fail and then you can get on with your life and go find a real job.

20:09

But a decade after the brand was founded, those wool trainers once adored by tech workers seem to have lost their. Their soul, so to speak.

20:24

So Albert's has decided to make a pretty significant change to its business.

20:34

Shira Aviona is a business writer with the Economist.

20:40

So it's selling off all of its footwear assets. No more sneakers. And it's decided to rename itself New Bird AI and to pivot to compute infrastructure for artificial intelligence.

20:44

That is some pivot. Shira, what's going on here?

20:58

So I think what Allbirds is doing is maybe an extreme version of a broader malaise within the generation of brands that grew up alongside it. Many of them were founded between, say, 2010 and 2015, and they marketed themselves as what I like to call Millennial brands. So their consumer base were upwardly mobile millennials living in cities who older brands, boring and wanted beautiful and minimal products that they could buy online that would be shipped to their door. So many of them had a direct to consumer business model, meaning that they did almost all or all of their sales just through their own website. And a lot of them were backed by venture capital companies and kind of had this, say, just a very millennial aesthetic.

21:01

I mean, it sounds like a decent business strategy to appeal to up and coming millennials. So what went wrong?

21:50

Part of the story is that they got very lucky when they were founded. So two things were going on. One is that interest rates were low, and venture capital companies that we kind of associate with tech became really interested in the consumer sector. So they were willing to hand out lots of money to Casper a mattress company or Warby Parker to sell glasses, or indeed to Allbirds to sell sneakers to Tech Bros. And that has changed. Interest rates have gone up, especially after the pandemic. So that meant that they needed to find other sources of cash, which has proven more difficult than maybe they might have guessed. Another thing that's changed is that if you only sell through your own website, the best way to get customers is by serving them ads on social media that used to be really cheap, but now it's become significantly more expensive as larger brands with more capital have flooded into the online advertising space. And everyone basically realized that this was a good idea. And so the competition has really heated up in a way that made it very difficult for these startups to compete.

21:57

So what's happening to those other brands?

23:00

Some of them initially got really lucky. For example, Dollar Shave Club, which was acquired by Unilever, but for most of them, including, in fact, Dollar Shave Club, because Unilever later sold it. It's proven to be quite a difficult road. So some of them, like Allbirds, have decided to abandon consumer goods entirely. Others are limping along and trying to make the very difficult transition from fresh millennial upstart to established brand.

23:02

And to return to Allbirds. Doesn't that just seem like they're starting a new company? I mean, surely there's no such thing as a pivot from shoes to data centers.

23:30

There is when you would like to retain your stock ticker.

23:40

So is this even legal? And isn't it essentially just rebranding a business failure?

23:43

I think it is legal, but it's not necessarily indic that the business is very strong. I think in essence they have sold off all of the intellectual property of the shoe company and what remains is the stock ticker bird. And they're trying to figure out, is there something we can do with this alone?

23:50

Shira, thank you very much.

24:14

Thanks very much, Rosie.

24:15

That's all for this episode of the Intelligence. See you back here tomorrow.

24:31

Why get aaa? Because bad roads, bad tires and bad luck. Out of gas, locked out or in a ditch, stuff happens with 247 roadside assistance AAA's got your back no matter whose car you're in. Join today@aaa.com AAA expect something more. Roadside assistance is provided by independent facilities contracted by aaa. Coverage in taxis, limousines and other ride sharing conveyances is excluded. 2024 AAA US Market Truck National Surveys offers terms, conditions and policies are subject to change without notice. Visit AAA.com for details. This episode is brought to you by Progressive Insurance. Do you ever think about switching insurance companies to see if you could save some cash?

24:54

Progressive makes it easy to see if you could save when you bundle your

25:31

home and auto policies. Try it@progressive.com Progressive Casualty Insurance Company and affiliates. Potential savings will vary. Not available in all states.

25:34