Anjney Midha's Plan to Radically Lower the Price of Compute

OddLots is brought to you by VanEck. For years, investors basically forgot about real assets, energy, gold, and infrastructure. But look at what's driving markets now. Central banks loading up on gold, massive CapEx cycles, currencies doing weird things, these assets are at the center of it. Racks, the VanEck real assets ETF, is an actively managed one-stop shop for real assets spanning gold, commodities, natural resource equities, Go to VanEck.com slash RAAxPod to learn more fun disclosures later in this episode. The Big Take podcast from Bloomberg News keeps you on top of the biggest stories of the day. My fellow Americans, this is Liberation Day. Stories that move markets. Chair Powell opened the door to this first interest-rate cut. Impact politics. Change businesses. This is a really stunning development for the AI world. And how you think about your bottom line. Listen to the Big Take from Bloomberg News every weekday afternoon on the iHeart Radio app, Apple podcasts, or wherever you get your podcasts. Bloomberg Audio Studios, Podcasts Radio News. Hello and welcome to another episode of the Odd Thoughts podcast. I'm Tracy Allaway. And I'm Joe Wazenthal. Joe, we like to talk a lot about physical constraints on this show. And this is one reason why AI is a really fascinating area for us right now, because there are a lot of physical constraints on what is ultimately the sort of ephemeral technology. And I think that the tension between those two things is really interesting, right? Like you type a prompt into chat GPT or Claude or whatever, and it's the sort of like disembodied digital platform. You don't necessarily think about the power usage, the real resources, the transformers that have to go into data centers to get compute. The thing that I've been on my mind lately and have written about it, and I plan to write more, is this idea that the canonical AI thought experiment is what happens if you tell an AI to make a lot of paper clips, and then it destroys the world. Because in the pursuit of marshaling all of the world's resources, it just turns everything into paper clips, because it doesn't know. I have to ask, is this canonical example, is this based on your traumatic fear of clippy? No, but that is, you know, it all comes back full circle. But what we're seeing in real life, is that everything from access to electrical grids, GPUs being the big example, energy turbines, talent, and now even including residential real estate, are being reproposed to make more and more advanced AI. And in the original paper clip thought experiment, they envision, or at least in one version, the philosopher Nicholas Bostrom, envisions the AI having exhausted all of the world's resources, then sending a probe into outer space to consume star energy to build more paper clips. Just eat the universe. And to this point, we're even talking about going into outer space for data centers to build more AI. So every version of the thought experiment is being replicated, except it's just more and more resources to build the AI by humans rather than paper clips by AI. There's this other connected theme here. So we've talked before about how one of the reasons valuations seem to be getting insane in the market is because all of this activity is being driven by like this existential need to become number one in frontier models and this new technology. And so if you say you absolutely have to be the first to invent AGI, then you can justify any amount of spending on earth, right? And so what we tend to see is like the biggest companies just keep getting bigger, and they're the ones that can get resources for all this stuff. And I think one of the most fascinating things right now is that at least as of right now, June 4th, 2026, the frontier models are really close to each other, right? So and the 4.8, GPT 5.5, like they're not that different. And one of the things I'm curious about is, is there's something inherent in market dynamics in this space that will always keep, you know, whether it's being able to distill results from another model and cause I steal them, whether it's information sharing among employees. Is there some inherent reason why we've seen the stability or could it be that at some point one lab just like breaks out and establishes permanent possibility? But like I am personally on the side of commodification and everything just becomes kind of basic or basically available. I know. Okay. Well, all right. Thank you. All right. That's a polite prompt to get to the guest. We do in fact have the perfect guest. We're going to be speaking with Anjanay Mida. He is of course a former general partner at Anderson Horowitz, a Stanford University visiting scientist who teaches the viral AI lecture called Frontier Systems. Also, one of the first guys to write a check for Anthropic is now the founder of a new company called AMPPBC. So thank you so much for coming on OpLots, Anjan. Thanks for having me. One correction it's pronounced AMPPBC, but that's everything else you got perfect on the intro. AMP would make sense, wouldn't it? Yeah, as an energy. Yeah. Yeah. And just remind us the PBC is Public Benefit Corporation. That's right. So you're doing this for the public benefit. We're governed by a public benefit charter, which means everything we do has to follow our mission. We have a public charter mission. We are for profit in the same way Ben and Jerry's or REI and Anthropic are public benefits. So we aim to make a healthy, modest amount of profits that can sustain our mission, but we have the flexibility to choose what that margin is. Can I just start? I want to establish your credentials, although I feel like that very long list did a pretty good job. But writing the first check for Anthropic, tell us that kind of origin story because the anecdote that you hear is like 25 VCs turned them away initially and you said yes. It was a little bit of the other way around. I said yes, then we tried to get another 25 VCs to say yes and I failed. It was a harrowing experience. It was a bit of a wake up call. It was late 2020. I had just sold my last business. It was called Ubiquiti Six. It was a 3D mapping business, an AI business that we had founded in 2017. I felt like a failure at the time because I was in San Francisco. I just had a big picture, my life stories. I was born in India. I went to high school in Singapore and I came out of college to the United States at Stanford for my undergraduate degree. When I arrived at campus in 2011, deep learning had just started taking over the world in Silicon Valley. Andre Carpathi was a computer science TA to Andrew Ng, who was one of the modern founding fathers of deep learning, this idea that you can teach machines to think without having to give them prescriptive rules. I went into machine. I got swept up in that moment and started studying. A lot of my coursework was in machine learning. My primary department at Stanford was in bioinformatics, which was machine learning applied to healthcare. I got sidetracked to a venture firm called Klein-Aperkins for about four and a half years, where I got the chance to work for some of the great investors like John Doar and Mary Meeker. Then I left and started my own company. As is the case in Silicon Valley, when I was 25, I went and raised about 47 or 2 million dollars from some of the usual suspects like Benchmark and Index and so on. I thought I was the coolest kid in town. I got the ****** beat out of me because we built this incredible technology, which is this AI system that could map any location in 3D and then the pandemic hit. Location-based mapping, 3D mapping, the only thing you can control is how you react to what happens. I did feel for a moment like it was bad luck and then you just have to pick up the pieces and make the best of it. I did with my co-founder. We figured it out. It was a tough few years where we had to pivot the business, but we landed the plane. Essentially, a lot of the distributed systems we built on the back-end side ended up being quite valuable. We sold that to a company called Discord, which is a chat-up for gamers. We had an all-in-all-lost Discord time to plug them. Oh, amazing. We chat with our fans in there. Awesome. Our listeners. About a month after I sold the business, I got a call from some friends who were running research at OpenAI. We'd all been friends in the machine learning community in the Bay Area. We've trained a little model called GPT-3. We think it's the best since. Just a little model. Yeah. Nobody really paid attention. They were like, nobody cares, but we think it's the best thing since sliced bread. We want to leave and turn this into a standalone business, but it'd be helpful to get some of your advice on how to do that. I couldn't really come on board full time at the time with them because I had to integrate my company into the Acquire, but I came on as their angel and nights and weekends, I worked with them on the business plan and who we should raise from. That company is Anthropic. Daria and Tom and I started doing these weekly working sessions in early 2021. I assumed that if we went and talked to a bunch of venture capitalists on Sandal Road, especially some of the ones who were involved in the biggest hits of the last decade before that, they would get it. These are the creators of GPT-3, and they were like, we just don't get this. We've heard the whole AI story before. This whole general intelligence thing is a pipe dream, and it was painful. We tried to raise $500 million. We couldn't. We instead scraped together about $100 million, which I know sounds like a lot better the time was a rounding error compared to how much Google has done on the same kind of systems. It was all angels in that first round, a bunch of cats and dogs, all of us who believed in the mission. Then over the next 18 months, Daria and Tom and team put together a plan that we workshopped on getting Amazon involved as a strategic. That resulted in a $4 billion compute and capital partnership that made me realize infrastructure, especially compute infrastructure, was just a key requirement to create any kind of modern AI lab. Since then, I've spent the past five, six years figuring out how to unblock that compute bottleneck for research teams. Amazing. Obviously, an incredibly well-timed- It just emphasizes how much things have changed, right? People are literally throwing money at almost any model now versus a few years ago going like, ah, AGI, I don't really know. Let me ask you this question, because this is a very top-of-mind question for me. We can skip around on the timeline here, but there are three labs that are seen as genuinely at the frontier right now. That is obviously deep-mind within Google, open AI, and Anthropic. Then, of course, a lot of people say that the Chinese labs are very close, if not quite there. Maybe they're a few months behind. When we think about part of your mission is you say, okay, a new lab should be able to get access to compute. If you're really bright, that shouldn't be the bottleneck. Does that imply, therefore, that you expect more labs to be able to, were they to have access to the compute, also reach the frontier, and that there is something inherent about this seeming stability or parity that we see among frontier models? So the answer to your first question is yes, there are many frontiers to be conquered and pioneered. It's not just one frontier. I think that's a fundamental misunderstanding people have about the frontier. They talk about the... The jagged frontier. Exactly, jagged intelligence. In a poetic sense, in a historical sense, if you think about the wild west or the western frontier, it wasn't just one frontier. There was a frontier of gold and there was a frontier of jeans. It turns out Levi's turned out to be a new modern behemoth of a company. I mean, there were so many new businesses founded in the Industrial Revolution. I think that's the reality is the software engineering frontier, which is where Anthropic is clearly a leader, is one frontier. I think the chat frontier, the consumer chat frontier is another frontier where OpenAI has been a leader. Arguably, bite dance is at the video frontier with seed dance, right? Absolutely, yeah. And so I think there's just many, many frontiers to be conquered that are pioneered, rather. I think Anthropic is clearly a role model for the rest of the community on how to do it in an efficient way. They're, I think, fewer than 5,000 people and they've been able to put out state-of-the-art models that teams like Google, which have 60,000 people, are close to, but not yet quite there. So actually, I don't really agree with your assessment that they're all at parity. If you use the models day in and day out, they're quite remarkably different in meaningful ways to the person with hands on the keyboard doing the engineering work. And I think those differences reflect the focus of the teams, right? What is the actual mission that the team working on that domain cares about day after day after day? So in the Stanford class I teach, the first lecture was a breakdown of how frontier models are even created. And it's actually quite simple. The recipe is super simple. There's basically four steps. There's pre-training, mid-training, post-training, and then what we call the continuous feedback loop. So pre-training just is, it just says, hey, you collect a bunch of data from the internet and train a model to be a generally good pattern recognition machine. You then do mid-training, which is to say, in a particular domain that you really care about, you inject more capabilities. So if you want this model to reason about science or math or physics, then you give it science or math or physics data. And then you get a pretty good model that's specialized in that domain. And then you deploy it to the real world where you have people using it. And the context feedback, which is when the model is able to do a task well or not, and you can verify whether that task was done correctly, gives the model the data it needs to keep improving on that task, on that distribution. Data centers need electricity. AI needs copper. Reshoring needs steel. And Gold's Run may tell you something about how the world is repricing money and debt. All of those point back to real assets. The RACC ETF is an actively managed one-stop real asset shop from gold to commodities to natural resource equities, adjusting as conditions change. Visit vanech.com slash raaxpod to learn more. And investors should consider the investment objective, risks, charges, and expenses of the fund carefully before investing to obtain a prospectus and summary prospectus, which contain this and other information. Visit vanech.com. Please read the prospectus and summary prospectus carefully before investing. RACC's is distributed by Vanech Securities Corporation distributor. The Big Take podcast from Bloomberg News keeps you on top of the biggest stories of the day. My fellow Americans, this is Liberation Day. Stories that move markets. Chair Powell opened the door to this first interest rate cut. Impact politics. Change businesses. This is a really stunning development for the AI world and how you think about your bottom line. Listen to the Big Take from Bloomberg News every weekday afternoon on the iHeart Radio app, Apple podcasts, or wherever you get your podcasts. This is slightly tangential, but I give a lot of feedback to the models because Joe made me paranoid about the basilisk theory. So I want the models to appreciate me once they take over the world. But when you give them feedback, if they spit out a wrong answer and you say, that's wrong, they immediately apologize and fall over themselves to say that they're sorry, but then you ask them, give me another output or would you do it again the same way? They often say yes or they give a very similar answer. They don't seem to be responding in real time. Correct. So when I say feedback, I mean a very specific kind of feedback, which I call verifiable feedback. So when you say that wasn't right or that was wrong, that's an opinion. Verifiable feedback is when you can have as close to factual verification as possible. The reason... So what does that actually look like? That's a great question. So let's take reason by example in two or three cases. In the case of software engineering, in the way software engineer is actually code is you write a piece of code and then you submit it to the main code base and then you usually have a peer on your team review the code and approve it or reject it. And if it gets approved, that's the first step. That's called a PR, a pull request. And if another human on your team that you trust approved it, that's one kind of verification of quality. And then two, before that piece of code usually gets deployed to a production system, you have unit tests. And those are quite objective tests of is this code performing the function we needed to? And if it passes both those tests, it's a verifiable piece of code that accomplished the goal. So in software engineering, the reason we've seen such a dramatic improvement in capabilities is that a lot of these labs are using feedback from that verification loop. In the case of another lab I incubated called periodic labs, which we started a year ago and you should come by sometime. We've got 40,000 square feet in Menlo Park where we've got AI models that are predicting you. The goal is to try to find a room temperature superconductor. And so these models predict... Oh, I forgot about that. I forgot about that. That was a fun summer. Yes. This time we will verify there. If we ever put something on you, you will know it's not in the middle. It will be real. That's not going to be us. But the AI system predicts new materials candidates. Then we have robots that synthesize the new material in the lab and then use X-ray diffraction machines to test whether the material has the properties the AI said it would. And that's verifiable feedback from reality, from physics. And then we pipe that data back into the training loop over and over again. That context feedback is very factually verifiable. And that's where progress is the fastest today. Because that feedback doesn't result in the hallucinations that you often experience with these models on more subjective tasks. It's also, by the way, why the models are terrible at subjective tasks, like creative writing. And sometimes they can get quite toxic, to be honest. If you get them down the wrong loop, I don't know if you've been using it as a therapy bot and so on. I have not, just for the record. That's great. It did ask me to defy the laws of gravity at one point because I was trying to create something in my backyard and I was asking how to do it and it was like, then just set this up like the following way. And I was like, that's not within the laws of physics. Whatever. Well, what's interesting, and this is actually a trillion dollar question from just a very broad standpoint is, as you point out, even prior to AI, the field of coding had a very systematized approach to the feedback loops already. And so then it's like AI could sort of replicate that. Anyone who's done any vibe coding can see in the chain of thought sometimes, oh, that didn't work. Let me try this. That didn't work. Let me try this. Most fields don't really have that by and large. Journalism doesn't have that. I mean, there are things, there are outputs that are better and worse. We don't really have that sort of like formalized approach to the yes, no. Does that just zooming out to my mind that would apply that maybe, at least to some extent, coding is a little bit special from a sort of white collar knowledge work that in terms of like, is it going to be as good as say, sales or something like that, because it has a coding as a long history of that structured pipeline. Yeah, that's a great point. So where progress will be made most predictably is in parts of knowledge work, where the task is essentially a workflow that's fairly structured. And so somebody who spends most of their day inputting cells into an Excel spreadsheet, well, that part of the job will get automated pretty fast, because that's actually verifiable. And you know what? That's frankly, often the most tedious part of the job anyway. And so I'm quite excited to see that progress because I'm terrible at spreadsheets. And I think if we could free up more of my time and hopefully other people's time to focus on the art of the spreadsheet, not the tedious part of it. The entry and retrieval. Yeah, exactly. And in journalism, I think it's the same thing. There's so much craft that goes into the verification of a story before it goes out. That's not legible to the world. I've had a chance to spend some time with some of the journalistic institutions of the barrier like Kate Metz or Brad Olsen at the journal. And as you spend time with them, you realize, I mean, they're verifying every sentence that goes into each fact checking. So fact check, that's an example where I think we should be leaning on these tools and you should expect more progress. And the parts then that will be more to borrow your jagged frontier framing there, that that's we will be in a regime of jagged frontier progress where wherever parts of workflows that are verifiable factually will essentially, you will see progress there very predictably over the next few years. And consequently, wherever that progress that the workflows are not verifiable is actually where humans are going to shine. And I think that's where parts of the economy are you're going to see extraordinary gains in the wages of humans who have creativity and craft that are not typically verifiable, you know, through traditional objective means. Does that make sense? Yeah, it does. And it dovetails with a lot of what we've been talking about on the show recently. Just going back to verifiable feedback. So, okay, the model spits out something and you can check whether it's right or wrong. Right. Is it important to understand how the model actually got to that answer? Because we have discussions with like big bank CEOs who are using more AI and their response to this question is always like, well, if we can put restrictions around the AI, if we make sure that it's like released into a sandbox before it's released into the wider world, we're all set from a regulatory perspective and regulators don't actually need to know what's in the black box model and how it's working. But like this seems a bit concerning to me. Yeah. No, I'm quite strongly opinionated about this one, which is that technical literacy should be non-negotiable. This is the reason I spend so much time teaching this class at Stanford, putting it up online. And the idea of the frontier systems class is that end to end, it's a full simple, but first principle is a breakdown of how these AI systems are built from scratch, from land power shell, like the energy, where do we get them, the data centers, then how do we train the models. And the final project, the class with the kids was actually the one person frontier lab, which is at the end, they're creating their own models and so on. Because the idea is that a person with the right tools today can scale themselves infinitely, but they need to know how to use the tools, what the limitations are, when to lean on them versus not. And I think this is a generalizable piece of technical literacy that all leaders should have. It's like saying, in the 90s, I imagine, if you knew you could use the internet without really knowing how it worked. But on the margins when the page doesn't refresh, or you're like, this cookie thing is annoying me, over time, people who are more technically literate just realized sometimes you got to debug the browser. And those of us who've learned over time to do knowledge work are more adept at leaning on them versus not. Like just now, when I was trying to get onto the internet, I realized, okay, there's this, you know, Wi-Fi password, whatever. And then you don't end up relying on them in ways that they can't fulfill your need anyway. And what's a little bit more dangerous with these systems is because we tend to anthropomorphize them without the technical literacy that I wish all leaders had about reasoning about how these systems were built. What you end up doing is projecting out in your mind what the capabilities are in ways that are inaccurate. You project out their impact in society that are not accurate. You project out their business models in a way that are not accurate. I mean, in the very fact that when you started this conversation, I don't blame you for it. You're like, there's three models at the frontier. Yeah. I'm like, well, which frontier? And which models? Because from where I'm sitting, there's like 17 different frontiers right now. And there's four different players in each one. And the businesses of all of them are kind of breathtaking. So I think that technical literacy should always for leaders be a basic requirement. And then if you're deploying these systems at Goldman Sachs, you won't oversimplify and get tripped up later when, you know, two years later, you realize half your employee base has been leading on this, like sandbox framing, when in reality, inside the sandbox, they were doing all kinds of, they were using the tools in ways that were prone to hallucination, prone to risks, prompt injection, they were leading on it in ways that were not informed in the appropriate ways. Is this making sense? Yes, like at a minimum, they would not be using it in the optimal way. Correct. Okay. Or relying too much on it. It's the, you can't outsource your understanding to a model. You can outsource your thinking. You can outsource part of the tedious workflows, but you can't outsource your understanding. And if you keep thinking, if you say, if you create these simplistic frameworks of, oh, here's a sandbox and this is safe, you have to use that sandbox in the right way. Because if you say, well, now everything that happens in the sandbox is totally fine. If the model says, use the spreadsheet, the spreadsheet is good, it's deployed in our servers, but you didn't actually check the spreadsheet and what went into the spreadsheet. And did the model actually understand the particular structure of the business, the physics of the business that you're trying to model out? Then you've outsourced your understanding to it. Does that make sense? Absolutely. Let's talk about AMP. And because you're never going to get the frontier in anything unless you have access to compute. It seems pretty obvious. And there are various arrangements for acquiring compute. You have companies building their own data centers. You have smaller labs, and maybe they use someone else's data centers or a neocloud, et cetera. What are you building at AMP, such that at least as part of this story is trying to solve the compute bottleneck specifically? Yeah. It's very simple. What we're doing at AMP, we're doing two things. We are trying to standardize the format for compute, which today is super fragmented. So in the history of infrastructure, if you look at whether it was the Industrial Revolution, the internet, streaming, there were usually formats of inputs that were quite heterogeneous. They were fragmented. And then to unlock productivity, you had to standardize a format. So in the case of electricity, until ACDC was standardized, megawatts would just sit in stranded pockets around the United States being unused. And then once we standardized the format to ACDC, then the question was, okay, great. Now we turned all these stranded pockets of electricity into one sort of interoperable universal format. Now how do we distribute it to everybody who needs it? And we came up with this distribution layer in the United States called the grid. That's all we're doing. Yeah. So for compute. You're building a grid for compute. Correct. We're trying to standardize the compute layer today. Different chip types, different manufacturers, different clouds. I mean, it's a complete mess. And if you're... Yeah, go ahead. Say more about how you plan to do this, because we've talked before about, you know, there are various people out there that want to create indices of compute features potentially on compute. And the issue that always comes up is fungibility. Right. Exactly. So we've got a couple of ways we solve the fungibility problem. This is a pretty thorny challenge. We solve it in two or three ways. The first is we have a system called the grid, which actually makes the compute fungible at a consumption layer. So under the hood, we have a bunch of different chip types. We support various different manufacturers. And there's a system that was built to do this already inside a little company called Google. And one of the technical leads on that project was called Borg, internally at Google. He's my co-founder, Sebastian Lobo. He was my roommate at Stanford 14 years ago. He's my engineering co-founder. And we're building Borg for everybody else, which is essentially a translation layer that says no matter what the underlying chip type is, the machine learning researcher who's using the chip just has to worry about the workload. And we handle everything else underneath the hood. When you say system, is this hardware or software that's doing this? It's all software. Okay. Yeah. So we handle that translation layer in software. And it's a pretty gnarly challenge, but today we're able to do that in ways that improve utilization sometimes from 50, 60% at labs that we have incubated are on the grid to close to 95, 96%. At Google, the utilization is roughly 99%. When Sebastian arrived at Google, it was about 62%. By the time he left, it was roughly at 99%. At Google, if utilization is at 96%, that's considered a major outage. Today, the average data center in the industry, in the ecosystem, in the independent ecosystem is running at less than 70% utilization. The Colossus II, which is running in Memphis, Elon's 500,000 GB, 500,000 GB 300s, was running at less than 60% node utilization and less than 11% MFU, MLFlop utilization is how much of the chip is actually being used. So there's two kinds of utilization people care about in the data center. First is how many chips are being used. That's the highest, that's just the most naive measure. If that number is not at 90% plus percent, no excuses. Say you have the chips, they should at least be doing something. And then within the chip, how much of the chip is being used, within a workload, that number is usually much lower. I'm very intrigued by this latter point about that even if the chip itself may not be even used at full capacity, because I see these numbers and you say like a lab has, like we have 200 chips, we've acquired 800 GPUs, etc. And when I see these headlines, I assumed that optimal utilization techniques must be so good that you can infer someone's capabilities simply by how many Nvidia GPUs they've acquired. But you're saying is that there is actually quite a bit of heterogeneity about the techniques and approaches to getting the most juice out of ad chip. Yes, you have to measure what matters and what matters is output. Okay. When any time I start a new lab, in the case of periodic labs, we started with Liam Fettis who's the co-creator of ChatGPT and Dois Trubock who led the physics teams at DeepMind. And when we sat down and we planned out the company's roadmap, the most important thing to us to measure was not the number of chips we had. Yeah. It's the eval what we call... So all this chip bragging, they're like a we acquired is just a sort of... It's a lot of bravado. Yeah, all right. This is helpful. You don't measure the inputs, you should be having the outputs. I'm actually fascinated that there is a software solution to what I perceived in my head as a very physical constraint. How does this actually work? Feel free to get technical here. I want to understand the system. Yes. Let me give you the technological answer and the economic answer. The economic answer actually is a simpler one reason about... The way the compute business works today is primarily on the construct of the atomic unit of long-term leases. So I'm a researcher. I need some compute. I show up to a compute provider and say, hello, I would like some compute, please. And the compute provider says, no problem. Here's, you know, 500 AMD chips or Nvidia chips that you can lease from me on various time scales. And you got to pay for it 24-7. It's like leasing an apartment. And whether you use it or not, that's your problem, but it's $2.50 per hour, $3 an hour. So instead, you take a long-term lease. And now the cloud provider, the compute provider said, great, I just booked revenue for the next two years that this guy rented. Now, what happens with that compute, whether it's used or not, is the researcher's problem. They've outsourced that problem. As a result of this wastage that we're talking about, and I'm happy to go into why it's hard for individual teams to utilize most of the capacity, the primary reason is because research is spiky. It's hard to forecast. So you over-provision for your peak, not your base load. Because what happens, you're researching on these algorithms, and the minute like one is working, you go, guys, let's scale. We want to ship this thing. So let's throw as many chips at it. And then once we ship it, the needs go down. So between these spikes, there's just huge pockets of unused compute. As a result, the effective price per hour that you're paying is closer to $25 to $28. Whereas the marketed rate that you think you're paying is $2.50. So that spread due to wastage is just insane. So from an economic perspective, that's the wastage. That's the deadweight loss. So now how do we, from a technological perspective, how do we utilize that opportunity? Literally, all we do is from a software perspective, we take all of that unused compute, and no matter what format it is, it might be Nvidia, it might be AMD, we love AMD, might be some other chip, and we turn it into one fungible resource. And that, we standardize the format on something we call grid credits. So researchers don't even need to think about what chip type is under the hood. They're just paying what they need or what they use. And so from a fiduciary perspective, I'm on seven boards. As an investor, I get very excited when teams switch from this sort of long-term lease model where they're paying $25, $26 per GPU hour to now they're actually only paying the $2.50 that was marketed, because everything they're not using gets reallocated to the grid, and other research labs can use that that resource. The Big Take podcast from Bloomberg News keeps you on top of the biggest stories of the day. My fellow Americans, this is liberation day. Stories that move markets. Chair Powell opened the door to this first interest-read cut. Impact politics. Change businesses. This is a really stunning development for the AI world and how you think about your bottom line. Listen to the Big Take from Bloomberg News every weekday afternoon on the iHeart radio app, Apple podcasts, or wherever you get your podcasts. Get match ready with the German Donica Bab Boss Box, a main, fries, side, drink, and three signature sauces all in one epic box, packed with premium lean Dona meat and fresh salad made fresh to order. For a limited time, get 20% off Boss Box on delivery. Plus, thanks to Coca-Cola Zero Sugar, you could win one of five TV entertainment bundles, teasing C's supply, competition ends 18th of July. Order now on the GDK app or at GDK.com. GDK, kebab's done right. 18-month contract. Prices may vary. Verify at GigaClear.com. This is the problem you're mostly solving for is the training part because they're training, right? Training in English or is it both? Yeah, that's the beauty about having diverse types of compute on our grid is that once you make the resource fungible, you can do any workload. You just fill all the unutilized pockets with inference and then all the reservations would train. Can you explain why is it that every lab also seems interested right now in customized silicon, including Microsoft announcing a chip that says, oh, our new MA, I don't know how it's pronounced. Oh, MAI, I believe. We had Sajja in the class yesterday at Samford and he pronounces it as MAI. Okay, their new MAI model. And he's like, oh, we also have a new MAI 200 chip or something that's optimized with it. Why is it that so many labs or companies that are in a lab, I guess, feel impelled to also design a chip that goes along with the model and long term is what you're doing saying like, this really is not necessary to have that sort of model chip alignment. Yeah, there's two technological reasons and two economic reasons. Okay. The first is from an economic perspective, about 80 cents of every dollar a lab spends today on their R&D close to a chip provider like NVIDIA. Okay. Right. And so as a result, your margins are just super, super rough. So from a unit economic perspective, you want more control over your margins. And therefore, when you look at your like unit economics, you're going, wait a minute, for every dollar we make, there's this massive chunk that's going to somebody else. So instead of spending 80 cents to NVIDIA, you spend 78 cents to TSMC and keep that two cents for yourself? Well, I think that the better our software gets, the more that margin should flow actually to the researcher. Okay. Because that's where the value will be captured. But like, wait, sorry, you're going to say, what's the technical reason why they're trying to do optimal model chip alignment? On the technical side, the primary reason is you want control over your supply chain. Because today in a compute, well, we've been a compute capacity constrained world now for at least four or five years. But if you can't get the chips you need, you're not in control of your own supply chain. So you're dependent on compute allocations that the compute manufacturer thinks is optimal, right? By the way, that's how it works at the foundry level. Today, TSMC gets to decide which compute provider, compute providers business grows or not. Because they only have so much production capacity. And so the technological reason is you want supply chain independence. And so when you want economic independence, unit economic independence, anyone supply chain independence, you want as much control over your own chip. But Microsoft doesn't have a fab. That's not what I'm saying. What I'm saying is in inference, for example, Sacha would like more control over his unit economics. So he's making an inference chip, right? Because if you are dependent on a third party to give you the inference chip, okay, and you need and if you don't have an inference chip, you can't sell more, more product, you want more control. So he's about having a predictable supply of chips for you, rather than a predictable supply of chips. Okay, yes. So there's a lot of discussion right now about more efficient model allocation. So this idea that like you do not have to be using the latest model to ask like what the weather is going to be tomorrow or something like that. And you also don't want to blow through your entire one year token budget in the space of four months, as Uber apparently did. So the spikes in usage that you're seeing that allow you to do the system and you know, have grid credits, does some of that go away if people become smarter about which models they're actually using? Okay, so there's an embedded assumption I think I should tease apart in your question. Usage is different from the production of the model. So what's happening, right in terms of the pipeline is you use the grid to produce the system, the model, and then the model produces tokens. If the end user is only using tokens, then as long as everybody, we have enough diversity in the end user base using models hosted on the grid, things actually even out. Okay. That cyclicality in the same way electricity in America evens out if you have enough scale at scale, basically, except when like, there's a heat wave, exactly. So some of that infrastructure we are having to reboot. But you can think about AMP in the broadest sense as a utility company, where what's called an independent system operator of the grid. So we don't own our own data centers. We don't own our own labs, but we coordinate the capacity needs across different parties. And at sufficient scale, that you those usage patterns actually just gets evened out. Does that make sense? Yeah, well, you're an investor. Well, let's you are an investor in open router, I believe, which I think is an interesting company. Do you see, setting aside AMP for a second, do you think that there is a at this point, still within say corporate America, a certain lack of savviness about knowing which model to route to for the query and that there will be an improvement and learning within companies within users, so that you don't have these incidents for like massive token consumption, because perhaps everyone was using the wrong, the Cadillac model and the Ford model would have been just as fine for that purpose. Oh, yeah, we're absolutely in the medieval ages of this technology. I think what will happen is increasingly based on my conversations with corporate American leaders and corporate leaders across the world, they don't really care about the models. They don't care about the underlying model, the technologies, they just don't care. It's like too much complexity. We just want the work done. Yeah. Can you guys please figure out how to get the work done in the cheapest way in the most efficient way in the most secure and trusted way. And increasingly what you'll find is that which particular model is helping you out in a particular task will just be abstracted. You won't even think about that. It'll just be a companion. You're just going to talk to it. It'll be a companion provided by a brand you trust. And under the hood, they might be using 200 different models to orchestrate your task. And over time, that efficiency will get better and better and better and better. And that's why I just don't think there's only three frontier models that are going to win. It's going to be an ecosystem. This is, I know you don't want my take, Joe, but this is my coffee pod theory of AI. I want your take, Tracy. I love your take. I'll save it for the outro. Actually, on this note, we have seen some headlines recently. Obviously, there's the Uber one about token spending. And I think it was the COO said he wasn't sure if the ROI was there on Uber's AI usage. And we've seen there was a Goodvox article recently about a corporate reckoning with AI spend. Since you're going out and talking to CEOs, has anything shifted in the past couple months or so in the way people are thinking about the return on this initial investment or the return on spending on tokens? Yes. I think it's a barbell distribution. So there's two types of CEOs, broadly speaking. The first is the CEOs who are using the tools themselves. And those folks are going, aha, I understand the jagged frontier. When they understand the jagged frontier we talked about, their strategies, their questions they ask me are completely different from the CEOs who are outsourcing their understanding. They're not trying the tools. They're mostly asking their kids, like, hey, kiddo, this chat GPT thing, it's good, right? And your kid is like, yeah, it's pretty good, dad. And then you go. Kids think it's really dumb, by the way. Yeah, so that's the other thing. So the kids are super smart and they're using the tools and they're like, it's good at this thing, but not at that. So they understand the jagged frontier part. Actually, you know what, they think I'm dumb for using it. They're like, dad, you're not doing anything smart. They don't think the models are dumb. They think it's dumb. Exactly. They might be going the way you're using it. Exactly. It's not optimal. So the CEO. No, what I'm saying is my kids are six and they have no intent and they have no idea about anything and they just think I'm dumb. That's the whole point. That just might be a generalizable. That's really the only point I'm trying to make. Okay. Well, you can send them over to me anytime. I'm happy to be the fun uncle. Yeah, that would be great. You can show them that actually this is fun to play. My wife and I are happy to host your kids anytime. Okay, that would be great. That's really what I'm trying to get at. It's the summer, right? We have two nieces in London and we call it Camp Middashen. My last name is Middha. My wife's name is Shen. And so you're welcome to send them to Camp Middashen anytime. That's amazing. But that's the bifurcation is leaders who are actually trying the tools out. They realize they're extraordinary at some things and not at others. And so depending on whether you get it or not, or you're actually getting your hands dirty or not, I find the questions are completely different. So this has been an incredibly helpful conversation in terms of like understanding basically the problem of essentially tons of money is being spent and your thesis is that it's massively suboptimally used up and down the stack. You mentioned this, okay, you get a credit, et cetera. Do you actually see that being financialized in a way? Okay, I bought this capacity. I have a lot of unused time. I don't always have a research idea that is going to require a big model run test. I can resell that. Is that something that you see like something that genuinely resembles a financial market? I hope not. Because when you had speculation to production goods, it creates scarcity of a different kind, right? Because then you have financial traders and markets trying to trade the speculative value of the asset. And that's going to hurt a lot of our research teams in technology. On the other hand, I think that creates a need for innovation inside of the research teams. And so one of the core operating functions we have inside of our business is a forecasting capability, where we have a team that's very similar to actually the kind of forecasting team you'd have inside of a hedge fund. We're constantly predicting demand and supply. And then we're actually procuring capacity in advance through call options on compute clusters. But our needs are similar to the kind of internal trading desk you'd have inside of a large steel company, where they need to lock up iron ore and so on for their production needs. So I'm a big fan of efficient markets. And I'm trying to actively invest in and help entrepreneurs out and teams out who are trying to drive more efficiency in the service of more productivity in science and engineering. I'm not that thrilled about the financialization of these products if it ultimately results in more speculation. Does that make sense? Yeah. I'm just curious, since you're tracking demand in that way, if you were going to describe the slope of demand right now versus say like a year ago, is it steeper? Is it starting to plateau? Perpendicular. Oh, wow. Okay. If you look at the compute prices of long-term rentals, over the last six months, between January and now, they're trading up to X. So we started, for example, for 2026, we started securing our capacity in January at these long-term rates. We could resell that at a 2X markup if we wanted to. Part of the reason that 2026 has become just totally AI has consumed everyone's mind, I think, is because people got very excited about Claude Coat specifically. But that was a breakthrough at the hardest level, not the model level. Suddenly, the real excited is like, wow, this is just so fun. It's just so easy of a computer inside your computer. That was a hardest breakthrough. Do you see when you think about investment among AI labs, do you see any shift in allocation away from pure scaling and improving the model towards tooling and harnesses as a way to get more juice out of the models? I'm sorry, I have to correct you there. It was not just a harness innovation. Those two things go hand in hand. It's a symphony of improvement between, it's a dialectic between the model capability and the harness. That harness was designed specifically for the capabilities that the new model was going to have. And so when you design these things, in the industry, we call this co-design. So you have the harness designed side by side with the researcher who's designed the next generation capabilities in the model. And you get a little bit of visibility in where the model is going to be good because as I described earlier, the pipeline is actually quite predictable. Pre-training, mid-training, continuous feedback loop. Once you have that visibility, you go, aha, we specifically want to improve the capabilities on this type of task. It's going to take us about three months to get there. Start designing the harness for that improvement. By the time they show up, then you can have the harness assume that the model will be able to do X, Y, Z on its own, whereas ABC, it's going to need third-party tools. So then the harness says, remember that three months ago, you were terrible at understanding a spreadsheet. So then we had to go use a third-party tool to use a spreadsheet. In the last three months, what we've done is added the ability to actually reason about a spreadsheet in the model. So now you don't need to use a third-party spreadsheet. And so then the harness gets updated to say, don't go out and use a third-party spreadsheet, which by the way, collapses the time required to do that task by like sometimes a minute to two minutes. Now suddenly I've improved the user experience. And that's when things really sing. It's when both of those parts, the model and the harness are co-designed to create a symphony. Does that make sense? Yeah, absolutely. All right, Anjanay Midda of AMPPBC. Thank you so much for coming on OddLots. Really appreciate it. Thanks for having me. And everyone go out and check out the Stamford lecture series. It's on YouTube, right? It is CS153.Stamford.edu. Perfect. Oh, I have a big flight coming up, so I'll watch it then. You should download all the lectures. There's quite a few. Thank you so much, Anjanay. That was fantastic. That was great. All right, Joe, that was a great discussion. Yeah. I should emphasize just how big a deal that lecture series actually is at Stamford. Like students are beating down the door, basically, to get into that. And if it's free on YouTube, you should definitely check it out. I just want to establish that if I had given you the avian simulation that I didn't want to hear your take or B, the idea that I would have wanted to hear Anjanay's take instead of yours, I want to hear your take. No, it's fine, Joe. I realize that most listeners are here for the guest takes. I get it. But I thought his point about the jagged frontier was an important one. This idea that maybe the future, it's not going to be a winner takes all thing in terms of models. You're going to have a bunch of different models doing different things that might suit different companies. And also, the idea that a lot of companies aren't going to care about which specific model they're using. They just want the cheapest one that basically gets the job done. In my mind, that sounds like more of a commodified market, right? Rather than like, oh, people are going to pay up for, as you said, the Cadillac model. So what I would say is by listening to Anjanay and AMP is that people will want a commodified service, but that under the hood, I mean, this just sounds like what he's really trying to solve. And it's very interesting. I, as a user or company, buy a commodified service, but under the hood, the commodity has an incredible amount of variety of models through which it can route. Some of which will be the Cadillac. Some of it will be the Currig Coffee Cup. Yeah, absolutely. But my point is maybe in terms of valuations, right? If everyone is assuming that the Cadillac is going to be the one that everyone is going to get and the total available market, the TAM, infamously is not just the world, but potentially the universe. Like that seems a stretch to me. Totally. And just generally, I thought it was super interesting. And the idea, we've done a couple episodes recently specifically learning more about both chip level and box level optimizations, both how many chips you're using and how well you're using ad chip. Definitely way more to do on that. It still blows my mind that this is a problem that can be solved with software rather than like something physical. Like you just come up with a way to efficiently allocate the compute. Because in my mind, it's such a physical problem. And we've talked to previous AI market participants like Brandon McBee at CoreWeave. And they talk about like, oh, it's difficult to standardize because of the configurations of chips and things like that. But if you could solve it just through a software system, that's pretty crazy. I guess Google's already done it. Yeah. All right, shall we leave it there? Let's leave it there. This has been another episode of the OddLots podcast. I'm Tracy Allaway. You can follow me at Tracy Allaway. And I'm Jill Weisenthal. You can follow me at the stalwart. Follow our guest, Anjanay Midda at Anjanay Midda. Follow our producers, Carmen Rodriguez at Carmen Armit, Dash O'Bennett at Dashbot, Kale Brooks at Kale Brooks, and Kevin Lozano at Kevin Lloyd Lozano. And for more OddLots content, you should check out our daily newsletter. You can find that at bloomberg.com forward slash oddlots. And you can chat about all of these topics 24 seven in our discord, discord.gg slash oddlots. And if you enjoyed this conversation, then please leave a comment or like the video or better yet, subscribe. Thanks for listening. The Big Take podcast from Bloomberg News keeps you on top of the biggest stories of the day. My fellow Americans, this is liberation day. Stories that move markets. Chair Powell opened the door to this first interest rate cut. Impact politics, change businesses. This is a really stunning development for the AI world and how you think about your bottom line. Listen to the Big Take from Bloomberg News every weekday afternoon on the iHeart radio app, Apple podcasts, or wherever you get your podcasts. Virgin Media gives you show stopping TV and broadband. You get the channels you love, including Netflix and now Sky Atlantic at no extra cost. That's epic entertainment. You can't stop watching a bit like an elephant cruising on a truck through the bright lights of Bangkok. Plus, you get lightning fast broadband too. Yeah, that's entertainment. The Virgin Media way. Visit virginmedia.com. New customers only, Virgin Fiber areas, restrictions and credit checks apply, time to apply.