Durable Execution and the Infrastructure Powering AI Agents

Building stateful applications require orchestrating calls across dozens of microservices with variable availability characteristics. You can imagine state management became a big mess there. Imagine a really busy kitchen at a restaurant, all sorts of chaos going on. The real world is complex. There is all sorts of chaos going on in a restaurant. But at the end of the day there is a very clear outcome the restaurant is looking for. Every order gets processed in a very specific sequence of steps. Then eventually every order transfers templates to being delivered to a customer exactly once. This is exactly what durable execution provides. We completely abstract out state management for you as a developer building an order management system. You just code up your business logic and we are the execution authority of making sure every order gets processed exactly once in the presence of all sorts of chaos and failures in the system.

0:00

What happens when an AI agent fails halfway through a task? If it's a short prompt, you start over. If it's a three hour deep research job burning thousands of tokens, you've lost real money and real time. Durable execution solves this. The idea started at Uber in 2015 where two engineers built a system that could remember the state of any running process and recover it seamlessly. After a failure, that project became Temporal. Today, Temporal Powers OpenAI's codex, processes every snap story and runs transactions for Coinbase and Yum brands. As AI agents get longer running, more autonomous and more expensive to restart, the need for guaranteed execution has gone from nice to have to mission critical. But scale is only part of the story. The real question is what infrastructure the agent era actually requires and what it's still missing. A16Z's Sarah Wang and Raghu Raghuram speak with Samar Abbas, CEO of Temporal.

0:55

Maybe to kick off, just for the audience, can you share a little bit more about what exactly is durable execution and why does it matter?

1:51

So Temporal is an open source platform which ensures durable execution of your code. What that means is if during an execution of your function, if a failure happens, we remember all of the state and then we seamlessly and transparently resurrect that execution on a different host along with that state and continue executing exactly where it left off without you as a developer writing a single line of code for it. So in a nutshell, that's the core value proposition of what we are trying to enable with Temporal as a platform

2:00

in a modern application. The reason this is hard is because a modern application consists of so many distributed parts, correct?

2:39

Exactly. And so that's why, when you think about the core primitive we are building is super Straightforward. But to Raghu's point, now it is so broadly applicable to such a broad spectrum of problems out there to kind of bring this to life is imagine a really busy kitchen at a restaurant. Okay.

2:47

Yeah.

3:08

And at any given night, a busy restaurant is probably taking hundreds of orders. Every order might look different. Some get prepared quicker, some takes a little bit longer. They can come in not in a uniform way. Suddenly a lot of customers come in. You might see spikes of orders coming.

3:09

Chaotic. Yeah.

3:29

But at the end of the day, there is a very clear outcome the restaurant is looking for. Every order gets processed in a very specific sequence of steps and then eventually every order translates to being delivered to a customer exactly once. That's what the outcome they are looking for. Imagine all sorts of chaos going on. But real world is complex. There is all sorts of chaos going on in a restaurant. Stations might go down essentially and oh, a particular chef might take a break or some new person come in without any context on how much of preparation has already been done. What the thing that restaurant cares about is how many items are on the menu and how to prepare each one of those items in a specific fashion. But handling all of those failures, handling these spikes, or handling all these failure cases of stations going down and coming up, or when failure happens, how do you make sure you have enough state to continue processing that order where you left off? Those are not the things that restaurant cares about. This is exactly what durable execution provides. We completely abstract out state management. We completely for you as a developer building an order management system of a restaurant, you just code up your business logic and we guarantee all of that state or each and every order gets processed. We are the execution authority of making sure every order gets processed exactly once in the presence of all sorts of chaos and failures in the system.

3:31

So I love this kitchen analogy. Can you maybe put into a real world example, Obviously you and Max started Cadence, the open source project at Uber. Can you share a little bit more in these concrete terms? Maybe what some of the use cases were at Uber, both me and Max,

5:01

I think a funny thing is we started at Uber within a month of each other. Back in 2015, Uber was going through a pretty interesting transformation. Like as you can imagine, like any startup, they started with a monolith and at some point the company started running because of the like hyper growth, started running into both technical limitations of the tech stack to grow the company and even organizationally they want to build more and more things also. So this is where the company started moving towards a microservices Architecture. And I think they went so extreme where by the time we joined like they had more microservices than engineers in the company. So as, as you can imagine, building stateful applications require orchestrating calls across dozens of those microservices with variable availability characteristics. And you can imagine state management became a big mess there essentially. Right.

5:18

And at Uber scale, no less.

6:12

At Uber scale, essentially. So this is what kind of led both me and Max to kind of convince the leadership there that oh, this is one problem we know how to fix. And that's what gave birth to Cadence, which was the predecessor for temporal. As a platform where the whole idea is you write a function and we take care of durability for you and we handle all those infrastructure failure that your function, we guarantee execution of that function from start to finish. You don't have to think about that as a developer. And that actually started opening up Cadence for a broad spectrum of use cases at Uber. Some of the good examples there is as simple as like tipping flow. And by the way, this is a technology you can adopt in a very piecemeal fashion. And Uber tipping flow use case was actually a very good example of that was oh, they were integrating it, I think one of the banks in South America where the SLA was three days, the entire system was powered out of Kafka, which is an awesome technology for processing large streams of data. Retrying a single message for three days. It is not designed for that. Right. So the very first use case there in that team is oh like you try the tip, like the message to process and the banking API call where the message comes in. You try if it returns a failure rather than keep on retrying there, you start a Cadence workflow with a retry policy of three days and then that done essentially. But then over time they actually ripped out the entire platform which is dozens of those event driven system, dozens of Kafka messages and replaced the entire thing with temporal layer, essentially. Cadence. Yeah.

6:13

You're bringing up an actual second important attribute of temporal. Right. Which is very well suited for these long running transactions. And we'll talk about that in the modern world later. But the recoverability and the long running transactions seem to be two things that are very attractive to your customers.

7:48

So which is where I think the extreme end of that example was the Uber loyalty program. Like you take a trip, Uber won.

8:07

Yes.

8:16

Yeah. The more trips you take, the more points you earn and then you can those points for oh, $5 rewards or other things. Essentially that entire system was running on top of cadence, essentially where like you are running a workflow for every Uber rider essentially, which is there forever and it is just taking events. Every time a trip completion event comes in, it is a signal to that workflow. And that whole business logic of awarding points based on your trip data is all captured within that workflow. There was literally no database backing that entire state for each one of the rider essentially is kept within that workflow itself. And at least one interesting characteristics that you get of that is I think there was an incident where there was a business logic where you get an event for trip completion and the business logic has a bug where it will reset your total credits your points to zero.

8:16

Not much of a decision that would

9:22

make a lot of people unhappy.

9:24

Yeah. So somehow that bug got rolled out into production and then suddenly within an hour Uber support started getting call. Where are my points Essentially?

9:25

Yes.

9:35

And imagine trying to recover from something even even though they immediately rolled back and then fixed workflows which were not touched, but the workflows which were touched are already corrupted. I think one of the very interesting dynamics you get from a system like temporal, which is built on the principle of like event sourcing, is we actually build a feature while during that outage is oh, because we know when the workflow made forward progress, what was the build id? Yeah.

9:36

Wow.

10:08

So we actually could reset the workflow, go back in time and reset the workflow to that point and replay all of the events after that point with the new code and suddenly you recovered all of the corrupted workflow. So like when you look in the platform. So that's the awesome thing about temporal. As a platform, we solve so much of the problems around worgening long lived applications, recovering these applications from user errors. And those are the things which typically I don't think there is any other technology which even comes closer when it comes to those class of problems.

10:09

I mean there's a huge economic impact. If you think about even that example that you mentioned. Right. Turned customers. Just any sort of complex workflow touching a customer that fails midway through starting over. The economic impact of that is huge.

10:42

Yeah.

10:55

Has there ever been a study that you guys have seen like with or without temporal, that you can share?

10:56

One of the things we were able to deliver through temporal cloud is five nines of operational sla.

11:02

Incredible.

11:07

And I think, and first of all, like what value you put to that level of reliability. It's like if you go talk to people who experienced the cloud provider outages during fall of last year so painful, they can clearly tell you economically and I think so. That was actually one of the. Another big thing that we were able to deliver last year is we've been at the end of the day, core value proposition of our cloud product is specifically built around this reliability promise. And during those outages we have features like which allows you full business continuity in the case of full region or wide outages. So we have a feature called multi region namespaces where people who were leveraging that feature, they were actually able to fail over their namespaces to be powered out of different regions and they didn't have more than few seconds or few minutes of disruption for the service while those outages were going on, essentially.

11:08

Wow, wow. That's massive value. Especially when you're powering these mission critical applications.

12:10

And that's the key thing is right like today, if you go talk to any organization which is now putting applications in production, just building these kind of business continuity things is a huge part of their roadmap essentially. And literally the moment you start building these things on top of Temporal, you are getting all of that value for free, essentially, which is very big for people who are looking for higher level of reliability and availability guarantees. Yeah.

12:16

And you know, I think with AI the stakes are actually only getting higher and maybe this is a good time to transition to Temporal's role in the Agentix space because, you know, Temporal is actually powering some of the world's most utilized agents currently and the ones, you know, in the process of being built. And so I think one of the things that got us excited about Temporal is not just the product that you built five, six years ago, but its relevance to today is probably greater than ever. Can you talk a little bit about that journey and maybe even share a little bit about the agents that you're powering? And I think just starting with why Temporal is so perfect for the Agentic framework today and sort of the new stack that's emerging would be really interesting.

12:45

Yeah. So at least here is the thesis for us here. Temporal is if you think about, I think there's a massive, as you talked about, there's a massive platform shift happening right now. And if you what are the core ingredients of that platform shift is which is driving it is. Yes. Previously the way we used to build applications is oh, there is a business domain and you write a product spec about how to automate certain parts of business processes. You talk to an engineer, pass of all of that one thing and that engineer turns it into a working app or working application essentially with the whole agentic Way, I think as these models, by the way, there are two things happening. First of all, these models, they are coding agents where you can generate these strongly typed app now.

13:33

And barrier temporal also goes down.

14:24

Yeah, exactly. So basically now I think like there will be a lot more apps being generated which is specialized for very specific needs essentially. And the other thing is, as these models are getting smarter and smarter, what we have discovered is new what we call the agentic apps, which is the agentic loop, which is you have a model, you give it a prompt with some context and a set of tools and then you let that model kind of plan and then make those tool invocations to give you the right business outcome. Essentially though it is the agentic loop, there's a second class of those apps there, essentially. So I think this platform shift is happening. The thing we are excited about at Temporal is two aspects. Temporal is a company which is obsesses about developers. We at the end of the day, if you look at what's a core bottleneck for the company is oh, a developer getting excited about the platform and building their next app.

14:27

Yeah.

15:27

And I think what this platform shift is going to do and which we already start seeing is a lot more developers are going to come in the fold of being developers now essentially, which were traditionally or previously not developers can now actually build these apps themselves. We are super excited about that possibility. The second thing is the cost of building these apps is going to be significantly lower, which means there is going to be an explosion of those apps. Both of those things is actually the core bottlenecks for Temporal. More developers building more apps. At the end of the day, the core value of software is now going to be shifting towards how do you put those apps, make it operational with all of the guardrails where an enterprise is ready to handle this explosion of apps which is about to happen. And I think that is the thing that we feel durable execution is pretty uniquely positioned to capture that wave.

15:28

Yeah, yeah, no, I think you made a great point there about where the agentic application universe is headed. Interestingly, especially the last, I would say six months, right. There has been a shift from these interactive agent applications that are essentially short lived agents to these long duration agents that are just sitting in the background and doing a whole amount of processing. Right. And so that is a wave that's playing in your favor because those agents now need the same thing that you built for Uber. Right. They need durability, they need long running state management, they need to be able to recover for all sorts of Scenarios and these API tool calls go off and come back in an indeterminate amount of time. So you're very well suited for that shift.

16:37

So 100% Raghu here is if you think about it, probably last couple of years was where a human is prompting LLM getting responses. Now we have seen already seen the shift from that to now. Give it context and tools and now agentic loop where now these agents are doing work for you also essentially without human kind of continuously talking to those. Which means we have already started to see these agents are getting longer and longer and doing more and more meaningful work and more asynchronous. And to Raghu's point this is exactly where state management of those agents is going to become a very big problem.

17:30

Essentially the agent loop gets mapped very easily to the temporal workflow. Got it.

18:11

And so we see, at least the way I describe it, if you take a step back is I feel we are in the Ms. DOS era of agents right now where you have an operating system where you build these apps, you give full control to your application because it's in a sandbox, right at the end of the day what yes you can going to mess up only one machine or VM essentially or sandbox. So you give it that access. But still there people are seeing huge value. A very good example is CLAUDE code for instance. I'm pretty sure everyone after the holidays is super excited about CLAUDE code. And then yeah, it's an awesome example of an agent which you run on your either developer machine or the VM that you are working on. And it's actually doing very useful work for you. It is actually starting to take instructions where it's actually running more and more longer live but still within the scope of a single sandbox. I think it's very natural where these models get more and more smarter. We will run out of what things you can do in a single sandbox where there will be a whole swarm of those agents. We have already have organizations who are already running hundreds of agents on top of temporal right now. So we already see that industry is kind of moving into a multi agent tech world. Yep, absolutely.

18:19

So are these, is it a scale out pattern or is it each agent doing a different task and then there are sub agents or what is the pattern that you see when these customers using hundreds of agents?

19:47

So I think what we are doing lots of specialized agents. A very good example of that is let's say if someone is building an app which is I want to book my next vacation and at this point Someone like people, someone that person don't want to reinvent the wheel of course. Oh there is already lots of APIs available which is doing reservation for airlines or rather who have all of the knowledge built for like hotels. So why would. Even though yes if someone wants they can generate all of that. But why? Essentially there is no business value in that. So I think what we are going to see is I think a lot of value will move to specialized agents which is doing meaningful stuff essentially which means that you will be handing off work more and more. These APIs are going to get more and more longer lived essentially.

20:00

Exactly.

20:55

And then you need a durable RPC to connect all of that together for building end business outcomes for the use case you are going after. We clearly see the world moving in that direction essentially and I think that's a big gap right now. People are very excited in this Ms. DOS era of oh I'm just talking to my agent locally or within the sandbox and doing meaningful work. But we are clearly headed into a very distributed model of a swarm of agents collaborating together on solving complex problems.

20:56

Yeah, specialization is always the way of the world. So there is no reason it would be any different this year, this time around.

21:25

Yeah, at least another thing that I see is at least this AI is very model similar to human. Right. It's about the context. The more context you get give that exactly you the real less correct things come out of it essentially. And it's the same thing right. Like as a developer, like if I am thinking about everything at the same time I am going to produce bad software. Like I'm a very first principle thinker and which is you take a complex problem, you break it into smaller chunks and then you work solve these smaller chunks and compose complex solutions. And I think this is exactly. This agentic wave is moving towards and this is where I think I clearly see the world heading towards.

21:32

We totally agree with you by the way. Before we move off coding agents, just since you brought it up, I wanted to ask a quick question. So you actually are powering some of the world's top coding agents. I don't know how many of them you can talk about publicly, but obviously OpenAI has one replit. There's many different layers of coding agents out there. Given your seat, what do you think differentiates these coding agents longer term? And why is durability and reliability so important in this space?

22:19

In particular let's talk about like some of the characteristics of these coding agents and how they like what are the good characteristics you want to see as they evolve. Essentially initial version of these coding agents was people thought of that as just tap completion.

22:49

Yes, yes, right.

23:06

And then yeah, we will give like our developers in the company, oh, go try out a coding agent. They said like I already have tab completion like this. But like especially, especially after the holidays everyone came back. Max had this moment with Claude 4.5 essentially is oh, like these coding agents is no longer like. I think we are clearly entering a world where a product magnet engineer is now empowered to produce software without raticizing a line of code for it. And this is an insane world. So what we are clearly starting to see is you provided the right context and right prompts, you can get really complex problems done. Like these coding agents then burns a lot of tokens for we are clearly seeing they have started to work longer and longer to produce the right outcome for you. So the time. So one clear dimension to is these agents are going to get more and more long lived. Exactly where you will be handing off work in the background because then what an engineer would be doing is suddenly I think the agent tech engineer would be working on 15 things simultaneously because they will be basically working on something and then giving AI the right context and prompts and stuff to kind of work on a task and move on to the next one and the next one and the AI agent on the background like works on 15 things for you. So that's where I think the productivity. That's why I believe the productivity we are going to see in the industry is just going to be insane.

23:09

Yep, yep. So going to be an exponential leap over time of course compared to today.

25:07

So this is where for example, if you one use case I can talk about is like about OpenAI how they are leveraging temporal essentially is Codex. Codex is exactly that where oh like now these Codex agents are like orchestrating various tools, very complex patterns for getting a task done completely in the background. Nothing is running on your code, everything is being spun off, tested and everything for you on the background. You need a reliable orchestration engine to kind of tie all of these things together. So the way we typically talk about that is these models tell you what to do, but you need an execution authority. How does that work gets done at a very large scale. That's the next problem. So this is how Codex uses. This is exactly where doing millions of those Codex executions at any given point of time with using the right resources, handling spikes and failures and all of those. This is exactly how they leverage temporal essentially. So my belief is these eventually these coding Agents are starting to where you hand off longer and longer streams of work and whoever is working towards that future is probably going to be the winner.

25:13

Yeah, makes sense.

26:35

Absolutely.

26:36

There's probably a cost implication too. Right. Given how token intensive it is the cost of starting over. Right. If you're not managing state is extremely expensive retries. Exactly.

26:37

Yeah. So when we talk to your customers, besides coding, which by the way coding agents, I think you've sort of swept the field at least anecdotally. We also see a lot of folks using you for what would be conceptually equivalent of a deep research. Right. I have to go off and research the problem and customer success. So what are some of the common patterns that you're seeing there?

26:50

So I think especially for deep research is a very good example of a temporal adds a lot of value.

27:15

Yeah.

27:21

First of all, to your point, you burn a lot of tokens gently. It's expensive. And there are few characteristics where temporal kind of as a platform provides for deep research kind of use cases, agentic use cases. One of the characteristics is interaction with a human. Because a lot of those dgp. One of the things is awesome is like by the way I talk to AI all day and AI is asking me questions and waiting for my response essentially. So before it kicked off like other part of the research. And I think so that maintaining that context across multiple human interactions and managing that state so you don't lose the work before that's a huge value. Temporal provides the second interesting characteristics. If you saw deep research is that it kicks off things in parallel.

27:22

Yes. Yeah.

28:20

Where you start a complex topic. Oh, let me scrape this thing. Or let me look into like call and make an API invocation. So it basically does a lot of things in parallel and then collect all of that into the next sequence of steps that needs to happen with temporal so natural for coding up like any complex kind of orchestration logic for entire deep research agent to kind of conduct the research and consolidate the responses before reaching back to the user. So I think and especially provided with recoverability and state management for your entire thing. The other of course thing is these deep research is now can also getting. I've already seen like now I come in like give it my thing that I want to research, go make coffee and I come back and it's still doing some stuff.

28:21

Exactly.

29:16

So it is absolutely getting longer and longer lived and then all the tokens you are burning. You absolutely want recoverability of any point in research over there. Yeah, yeah.

29:17

I think by the way just going back to The Uber scale piece of it. What we've been amazed by as investors is just how quickly products like a deep research have gotten to enormous scale. And you really don't have anything out there other than temporal able to handle that sort of production and scale at scale.

29:26

So thanks Sarah for bringing this thing up. If you think about this is the thing which I'm super excited about is when we build durable execution like this is 15 years ago. What we are seeing is basically business transactions. We are in the path of code like every. For example every Snap. Actually Snap story is my second category of use cases. But like every, let's say Coinbase transaction or Yum Brands which is KFC Pizza, Taco Bell every time you place an order that's a temporal review essentially. Yeah.

29:46

Wow.

30:15

So it's. But still it's order of magnitude of how many business transactions which is happening. For example at Uber it was trip scale. How many trips was happening. Yeah. So it's awesome. It's a lot of. And since we are building a consumption based business, we care about the throughput of those transactions essentially. But then the next category is what we call as Internet scale. And Internet scale is things like Snap. Like every Snap story and you can imagine New Year's Eve, how many Snap stories being posted. Essentially each one of them is basically business transaction, temporal transaction. And so we got actually pretty excited about that. And this is where the last few years we've been actually focusing on can we run at that scale One of the awesome outcomes we have delivered delivered actually last year, which we take a lot of pride on our engineering team on. We actually already have a cloud system which can handle spikes of 150k actions per second on a moment's notice.

30:15

Wow.

31:20

Without any interactions or anything. Incredible. So we've been kind of focusing a lot on basically this Internet scale. Can we handle that? The funny thing is last year, a couple of years ago actually I was talking to Max and said Max looks like we have over engineered on scale. What's the use case where we needing this much scale? And then of course the whole AI agentic wave happened.

31:21

Now you're the agent scale and this

31:51

is where I think if you look at some of the largest labs use us and the scale that we are talking about, snapscale is peanuts right now.

31:54

Incredible.

32:03

So we are talking about a completely different scale of consumption which this is the thing which I'm super excited about. This is the first time I am out of my comfort zone. Okay. Now I'm having conversion with Max. Okay. How do we 100x

32:06

yeah, I never thought. I hear snap scale is peanuts. That's a good one.

32:23

Yeah.

32:27

Yeah. The other interesting side benefit and I didn't realize that until we started spending time together is you guys keep a record of every step of the execution. Right. As a workflow executes. So there is a lot of. We hear a lot of hype and popularity about execution traces, especially for agents. And anything that interacts with models, all of that becomes is a free benefit on your platform. Right.

32:28

And this is by the way we actually bring the entire companies to. Even before AI, what we used to see is like when an organization adopts temporal as their core orchestrator we bring companies together, the entire organizations in the company together. Because people literally share our temporal UI to their business teams to show oh this particular. Because they will map business like business transactions to temporal workflows and then they literally will send links to their workflow execution histories as the visualization of what that workflow was doing and if they feel something is wrong essentially. So like this event sourcing which is the mechanism we used for state management, essentially it has this advantage of now everything in your system is auditable and everything. You get so much visibility into your business transactions now just because of that nature. So what we actually saw spatially with AI and when people start modeling their agentic loops as a temporal workflow is because these agents are completely non deterministic and the moment you model an agentic loop as a temporal workflow and temporal workload is an awesome way to kind of have a durable agent. And suddenly now you have full visibility into everything your agent was doing essentially. And then people are finding that to be super useful.

32:58

Yeah. And it's useful in runtime. It's also useful in making the agent better. Right. In training and so forth. So it has dual benefit there.

34:27

That is actually is awesome. That Raghu you brought that up is I feel like the amount of data we have of the execution history of those agents. I think we can eventually spin up a completely different product surface area for analytics. Business analytics. Essentially it's a goldmine of data to kind of figure out oh, which of the my tool invocation is taking the longest time and which of my tool has the highest failure rate. And we can because we have all of that data and we can run all of those execution histories to an analytics engine to kind of power all of that business level observability is also

34:36

a natural byproduct of your system. Yeah, yeah. So it's interesting. I mean we Talk a lot about agent Stack and there are a lot of companies, amazing companies that come by saying we solve this part of the agent problem. Agent stack problem. You solve a massive amount of the agent stack, if you will. Right. Obviously keeping these agents up and running long running agents observability, all these traces, a good chunk of all the environments surrounding an agent get simplified on your platform.

35:13

I think by the way, this is the funny thing is suddenly everyone is scared of oh SaaS is dead or software is dead.

35:45

The public markets certainly are.

35:54

So I think there's certain dynamics which is going on which is things that we talked about is more developers. That's a reality that's happening explosion of applications because the cost of building an app software is going to go down. That's a reality. Where is the value moving? I think at the end of the day the way I talk about that is at least the way I rationalize that is a lot of value is going to move to APIs. Like yes, if someone wants to build the next Google Maps using agents, probably they can do it. But all of the data Google have essentially power those maps essentially. That's the value essentially same thing like someone, like Stripe is. Yeah, if someone wants to build their next generation payment system, maybe they can do it. But Stripe has all of the relationship with all the vendors, everything out there. So I think there is real business value which is going to be exposed through APIs and agents. And I think at the end of the day to my earlier example, like if someone is now comes in and have some idea about the next generation experience for booking a vacation, they will not go and reinvent the whole a line reservation API. They will still going to use something there.

35:58

So you don't think SaaS is dead. A lot of people very interested in that question.

37:18

Yeah, that's a definitive statement right there.

37:25

I think.

37:29

I feel I agree with you Balaving.

37:29

I feel on the contrary, I think all of this, the other core things that I talked about, more developers and more applications, we are going to automate the world even faster. This is going to drive insane. We are already seeing insane amount of traffic consumption coming in. It's not slowing down, it's increasing. And I think all of those applications going to drive even more. If you are providing the right valuable business outcomes through APIs essentially your business is going to skyrocket essentially. So now taking that question to the agentic stack, one thing which is a reality today is a lot of if you look at Agentic platforms today out there are not differentiated.

37:31

Yep Agreed.

38:19

And I honestly feel like, I think we are in this world where we are evolving so quickly, by the time you take a bet on one of them, within six months it will just disappear because they are not differentiated today. So that's why at least the approach we took at Temporal was we don't want to create yet another platform. There are already too many out there. I think every organization, based on their needs, they need to assess and find the right platform for them. We solve the problem of state management or we believe these agents are going to get more long lived, more mission critical, more asynchronous. We solve the problem of scalability, reliability and putting these agents in production. That's where we are differentiated. So the strategy we took was integrate with everyone.

38:21

Yeah, of course.

39:13

And this is where I think we did an integration back in August last year with OpenAI's AgentIC SDK. We partnered with Pedantic to provide like a temporal plugin to basically where we could be the state management underneath the COVID behind Pedantic as a experience, essentially. And then we are working with everyone essentially. We have a team who's kind of basically building these integrations with every popular platform out there, essentially. And I think for foreseeable future we continue to go on that journey. And the area that we are investing into is we need to simplify the experience of Temporal itself. Today one of the bigger focus area for us is the entire developer loop. Today we clearly see people want to build something, try things out, see run it and then iterate. And I think today that loop is today we have a very clear separation between application logic and the backend. And we want to bring that together, provide a fully closer developer loop of someone building an application, testing it, running it and then deploying it to production. So that is going to be a big focus area for us and this is where we could potentially partner with a potential winner out there.

39:14

Also, by the way, just to maybe follow up on your point that there's just so much evolving in this space right now, it's hard to know what is sand, what is stone. Right. And Temporal is making bets on what you guys see as more set in stone. But curious, what if you have a perspective on is there an emerging sort

40:33

of

40:56

minimal viable architecture or infrastructure, if you will, of hey, I want to build a long running agent, I'm going to have Temporal for state management. Maybe there's some sort of observability layer code sandbox. Curious if you're seeing any consistency in the tools that folks are using and Maybe there's things that people started with frameworks, et cetera. Are those even needed? Curious how that that's kind of shaping up even though obviously it's quite a chaotic and crowded field.

40:58

100%. So there are some patterns which have started to emerge. For example things like sandboxes. Especially as these agents have now access to tools it can destroy a company. So I think if any organization basically rolling out agents in production better be investing into a sandbox or either building one or buying something.

41:26

We are seeing that.

41:47

And same thing essentially is we you mentioned prompt management is a problem we see because the quality of prompts evals is clearly an area where we see value. People are driving value as they start implementing these agents at scale. Another big area you mentioned is observability is as these agents are fully non deterministic you better have a way to find out and have guardrails in the system and have observability into those systems essentially. So we are seeing this is where by the way observability has always been a problem especially with even cloud architectures or even pre agent era. But these agents are going to push the boundaries out in observability at a completely different scale. And I think there is a huge scale problem there which someone needs to come up with a creative solution of. Okay. How to even rationalize all of the observability requirements when this massive agent explosion and application explosion happens.

41:50

Yeah.

43:02

How are enterprises doing that now? Is it mostly built in house or they're using outside tooling?

43:03

People are trying at least. What we see is there are a bunch of solutions there which claiming that oh we are the agent take solution or observability. But I think this is an area where people at least I feel it breaks at certain scale. There is this impression that oh the entire SaaS is going to collapse into agent. Actually I don't share that view and

43:08

especially that's a hot take these days. Please continue.

43:34

Why? So yeah, if you go and talk to a traditional enterprise or a bank, let's say they have certain business processes which is very well defined.

43:37

Exactly.

43:49

Yeah.

43:50

And they don't want. There is no reason to move those business processes to into this completely dynamic world essentially and it actually introduces such a gigantic risk for those organizations where I don't think they are moving to agents anytime soon or there's no need to move them to ad.

43:51

There's a reason it's called system of record. You don't want to lose the record.

44:10

Exactly.

44:13

Yeah.

44:13

And so I think we are Going to see the split. There are certain business processes which requires an orchestration or this deterministic way of building those and have the right guardrails and system of record. Everything is auditable, traceable and stuff like that. But then there are of course examples where agents are going to take off. How big is each space? It's anyone's guess right now.

44:15

But the beauty for you is you

44:40

are applicable on both sides and. Exactly. And this is for us, this is why at least whenever we talk to executives on those traditional enterprises, we solve both problems for you. At the end of the day you want an orchestrator which gives you guarantee you have a system of record. Where's the execution authority? We basically giving them an execution authority.

44:41

Yeah, to your earlier point, I mean there are so many business processes in an enterprise that surround the system of records that are done by humans or not even done by humans because it takes too much time and money to do it that agents can now do all those become use cases for you.

45:05

Yeah, and by the way, another thing I would add there is, especially if you look at now coding agents now people can take any business process and generate a very strongly typed application for it with very little cost. And I think there is going to be this tension also where people are going to evaluate do we really need an agent here or you just built a new app. Because building an app is so cheap

45:22

now by the way, just to hit on this, you sort of made this broader point of creating real business value with, with agents. And you know, and you also, you know, touched on sort of context management. And I think it would be, it would be amiss if we didn't talk a little bit more about context management. Context is such a buzzword right now. We've probably seen dozens of startups talk about being the context layer, context graph, context engineer, you know, sort of the, the full gamut. But I think again you're in this unique spot to really talk about how to best give agents cont. You know, what, what do you see as the way for, you know, agents to actually build proper context management?

45:48

For me, this was the biggest surprise we saw last year. So both me and Max essentially we were a big believer that temporal is actually a pretty interesting platform for data orchestration.

46:34

Yep, yep.

46:49

And we ourselves didn't get excited about that space is because at the end of the day we are building a consumption based business.

46:51

Right.

46:58

And the volume there are super low. Like how many data pipelines you have essentially at any given point in time. And the thing which actually Surprised us the most is clearly one few things are happening. Traditional data orchestration solutions are not powering context engineering is because they are not designed for that scale. The other thing we are seeing is now the context is coming from such a broad class of sources or maybe it pulls from an API or a Slack message or I don't know, Google Docs or like there's so many kind of connectors where people are pulling context from. Essentially everyone has their own reliability characteristics. Yeah. And I think a lot of those like data orchestration solutions don't actually work well. They work really well that you have a data warehouse.

46:58

Exactly. Yeah.

48:00

And then you can then do all sorts of transformation on that data.

48:02

They were designed for a different flow.

48:05

Exactly. So I think what we are seeing is these agentic applications, the thing which is feeding them is real time context engineering. And I think that for the thing which was. We are surprised, that is super high. This is actually one of the big use cases for temporal people who have these different sources to pull out that data in a real time fashion, massage it, put it into rags or whatever to power context for your prompts essentially is actually like those entire architectures. Temporal is the orchestrator for those architectures. And it's insanely high throughput. Yeah. Wow. And so that's to your point essentially is we call these use cases basically collectively describe that as retrievals because it's retrieving data from a wide variety of use cases and to power these LLMs and the right context. And at least it's a pretty interesting or attractive category of use cases for us at tampo.

48:06

Yeah, people are worried about context length and packing the context and so on and so forth. So what you're saying is very relevant to that problem.

49:06

Yeah. And this is where I think one investment area for us in the product itself is large payloads today. So durable execution today is awesome at building control planes or orchestrating basically complex or sequence of steps. Essentially we are not a system to pass large amounts of data. And I think especially for these class of use cases, we clearly see a need for us to handle large payloads also.

49:15

Yeah, well that's actually maybe a good segue because you're already, you know, you're firmly powering some of the best agents today. And I think what you just described is actually core to why these agents are actually effective for, you know, for, for any individual customer. I'm curious, what do you think is ahead for long running agents? Like what are you starting to see or anticipating that you'll see like what do you think? You know, let's say we do another podcast at the end, you know, in 2027. How will the world of agents change by then?

49:42

What are your smartest customers doing, in other words?

50:14

Yeah. You know, like having a Magic 8 ball to kind of predict. It's super hard in this world.

50:17

Yeah, we'll buy that from you.

50:26

Yeah. But at least what we see is these agents are starting to do more and more meaningful work.

50:27

Yeah.

50:37

And which means they need to break out from their sandbox essentially. And then world is clearly moving towards a swarm of agents essentially. And I think this is where the thing which blows my mind is there's such a big massive opportunity is having a durable rpc. How do you stitch together the swarm of agents essentially to basically doing state management across that? And I think this is where at least I feel we have a project called Nexus, essentially where we're trying to drive an industry wide standard. Even MCP for instance. MCP originally started with tool invocation, which is very synchronous.

50:39

Yes.

51:17

And first of all, basic problem today. Now I have a tool which I make a request response comes back in three days. There is no standard solution for that. MCP started to look into asynchronous kind of defining the protocol at least how do you implement that? Right, Exactly. So I think temporal would be then awesome implementation for any asynchronous tool invocation, for instance. So I think these tool invocation first of all is very natural that's about to happen. It's already happening. But then I think the swarm of agents talking to each other basically we will see a lot of that specialized agents doing specific things and then people are orchestrating calls across dozens of those agents to getting the right business outcomes. And then suddenly now you have a distributed systems problem at a massive scale, essentially. So durable rpc. I feel it's a very key building block which is missing right now.

51:18

Yeah. There are others in the industry trying to introduce some standards around it. Are you guys part of that?

52:15

So we've been trying and this is where we have our own initiative right now with project Nexus. But that's actually we want to coordinate with because I think this is a problem industry is seeing everywhere. We would love to work with like minded people to drive. We don't want to do it just only for temporal.

52:21

Of course.

52:38

I think it's a very generic problem and I think it should be driven as an industry wide standard, essentially.

52:38

Yeah, yeah, absolutely. By the way, just, I mean this is more of a question based off your description, we're also starting to see, you know, maybe you call them sub agents or these specialized agents. Right. And they're more focused on a particular task. They might even use different models under the hood. Right. Because models have differentiation by use case as well. But curious for your take because this seems to be evolving a lot. Do you think a lot of this development happens with the labs themselves or third party applications that can maybe make use of the multimodal world or of course there's the third answer. It depends.

52:44

I think Jensen describes as the best way I have seen it is this five layered cake of this whole platform shift that's happening and there is value at different layers essentially is starting from like electricity to chips to cloud infrastructures to model and then the application essentially, by the way, at the end of the day, this whole stack, the value of that stack is going to be proven by how large the application courier gets essentially. Because that's the end value to a customer then essentially. And so I'm a big believer that this application we have already talked about like things like Abridge, which is doing amazing work essentially around where Dr. Needs to focus on patients not transcribing.

53:20

Exactly.

54:07

And I really love their mission there. And same thing like we talk about even up like legal and I think this application.

54:09

Yeah.

54:16

Someone like Harvey for example.

54:16

Right, Harvey. Yeah. And this is going to show up in a massive way and which is going to prove the value of all of us. What we are building is going to be proven by that layer, essentially.

54:18

It better know.

54:28

Yeah. And so I think that has already started to happen. It's in its infancy, but the whole bat is this layer shows up in a very meaningful way essentially.

54:29

Yeah. Let's switch to another interesting company building topic which is very relevant to a lot of our listeners. Originally Max started as a CEO and then you guys did a switch and you were the cto, you did the switch. Right. And so that's fairly unusual I would say. Right. What led to that and what is it about the cto, the CEO position that you guys discovered that led you to do the switch?

54:40

Yeah, so I think when we were starting the company, by the way, like me and Max have a pretty long history of working together. We've been like I think working together for almost 15 plus years. I remember when we were starting the company, we didn't even discuss who needs to be the CEO or who like or what is my role going to be when you are starting and so small that things need to get done and someone needs to do it. I always assumed, I always assumed, Max, you are going to the CEO essentially. And I think literally there was zero conversations that happened on who's the CEO or what my role is and stuff like that. Both of us are very product people. We have built our entire professional career writing code and building products and providing technical leadership. Funny things about both of us. We have never been people's manager throughout our professional career. Before starting the company. We are close to 400.

55:08

There you go. You can't escape that.

56:05

Yeah, it's landing on the job.

56:08

So

56:12

I think then I think roughly like two years ago, it started becoming clear. My role and Max role. We looked at how we've been operating even before starting the company is. Max is amazing. Taking any complex problem, breaking it down and then figuring out where the company needs to be five years down the road. That's Max skill set. I've always been the practical one. What steps we need to take to deliver on that five year vision. That's always been my role. Essentially what we noticed, especially two years as the company grew bigger and bigger. Majority of the problem was not that we lacked product strategy or where we are kind of headed. Majority of the problem is execution risk. As the company grew bigger, essentially. And that is when actually Max himself got to a point where he said summer, I think it's about time where I think right next step. The company needs a different flavor of leadership for the next phase of company building. We are in. And you are the best person to lead that. And literally that was 15 minute conversation.

56:15

Amazing.

57:19

Wow.

57:20

Incredible.

57:20

You hit upon the two most important questions that a CEO faces. One is what should the company do right and how do you get the company to do what needs to be done right? And it looks like Max was amazing the first part and you're amazing the second part.

57:21

Yeah, 100%. And I think one of the things which is really humbling thing for both of us actually is since we never were people's leader, we never relate to the challenges. We have a lot of respect for people's leaders now running organizations and managing people in heart.

57:40

Yes, yes, yes. There is no agent for that.

58:02

Yeah, exactly. Maybe soon, but maybe sign me up whenever. Well, there's the Rent a Human now. I don't know if you saw that. Yeah, it's so funny. Rent a CEO maybe just to end on a slightly spicy note, not. Not necessarily related to management, but still touching upon advice that you would have for founders today. And let me just set it up a little bit so I won't ask you the question that everyone seems to be asking right now, which is are we in an AI bubble? Right. Who knows the answer to that? But I will maybe draw some parallels between today to that 2021 period. And those parallels would be in the fundraising environment, a lot of companies raising maybe further ahead of traction than in other markets and both a lot of commercial activity that is positive, but also fear in the market that good times will end soon. And so we see a lot of this push and pull among our portfolio and founders Curious given temporal is one of the few companies that came out of that 2021 period winning, you know, full stop. Right. You have blown past from a revenue perspective, valuation perspective, but of course it was a journey. And so curious what advice you might have for founders who are building companies today. And you know, what do you think should be optimized for in partners that you look for rounds that you're raising and you know, any advice you have.

58:06

In all honesty, I think the experience post 2021, it's actually helped us be a better company. One very clear example which actually is our margins. I remember back in 2021 when capital was cheap, everyone was of this mindset is growth at no all cost essentially. And we ourselves were kind of guilty of that little bit. And we had like, like multiple negative hundred margins,

59:38

like many did. Yeah.

1:00:13

And I think that shift that happened actually it helped us build, become a more resilient company. And we've completely like turned. We have amazing margins right now essentially. And we are able to turn that company because of that experience essentially. But at the end of the day, the thing which has not changed for us is focus on solving customer needs and having the clarity on what value you are bringing to the customer doesn't matter like downtime or up market or down. Like I don't obsess about that a lot. Yeah, you definitely want to have cushion on capital. But our strategy for this raise is different. It's not about the cushion, it's about we believe there's a gigantic opportunity. Can we go after that opportunity in an aggressive fashion. And I think that's what by the way, like this capital raise is not going to change our plan for this year. It's not like we suddenly. But at the same time I think we are going to especially both me and Max, we are product builders. We are actually pretty happy at Uber building cadence over there because we feel solving problem pretty meaningful problem there. But at the same time, at the end of the day, the core reason for us to leave Uber and start a Tamporo was Uber's business is to get people in car building a general purpose workflow solution is not part of their mission. And we believe there is such a gigantic need for this thing in the industry and we believed it so long we were doing it even before Uber essentially. And that's when we decided to take the leap because we feel we could never get funded enough to realize the potential of what we are building. I honestly feel like, I think with temporal today a big portion of this fundraise is going into R and D investments essentially, and I think we are just getting started over there. We have so many ideas about the kind of value that we can bring in and especially this fundraise kind of sets us up and to capitalize on the opportunity which is in front of us.

1:00:15

Absolutely. Well, we agree with you that there's a gigantic opportunity and we're so excited to be a part of it with you. Thank you for joining us on the POD Summer.

1:02:27

Yeah, thank you.

1:02:35

Yeah, thanks for having me. Thanks for having me and really excited to be part of this journey with you folks, essentially.

1:02:36

Thanks for listening to this episode of the A16Z podcast. If you liked this episode, be sure to like, comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes go to YouTube, Apple Podcasts and Spotify. Follow us on x16z and subscribe to our substack@a16z.substack.com thanks again for listening and I'll see you in the next episode. As a reminder, the content here is for informational purposes only, should not be taken as legal, business, tax, or investment advice, or be used to evaluate any investment or security, and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see a16z.com disclosures.

1:02:44