Hi, listeners. We're running a short survey to learn more about our audience so that we can continue to bring you a podcast you find helpful. If you have a moment, please take the survey at mitsmr.com slash podcast survey. You'll receive a complimentary download of MITSMR's executive guide, How to Manage the Value of Generative AI. Please take the survey this month at mitsmr.com slash podcast survey. We'll put that link in the show notes, and thank you for your help. How can AI remove barriers and make more seamless, enjoyable customer experiences possible? Find out on today's episode. I'm Will Crosshorn from Wendy's, and you're listening to Me, Myself, and AI. Welcome to Me, Myself, and AI, a podcast from MIT Sloan Management Review, exploring the future of artificial intelligence. I'm Sam Ransbotham, professor of analytics at Boston College. I've been researching data, analytics, and AI at MIT SMR since 2014, with research articles, annual industry reports, case studies, and now 12 seasons of podcast episodes. On each episode, corporate leaders, cutting-edge researchers, and AI policymakers join us to break down what separates AI hype from AI success. Hi, listeners. Thanks, everyone, for joining us again. Our guest today is Will Kraushorn, product manager at Wendy's. Most of you are familiar with Wendy's. It's an international fast food restaurant. Will has a fascinating background, which I'm hoping we have time to get into. At Wendy's, he's been involved with the Fresh AI Initiative, which is using AI agents to take drive-through orders. Will, great to have you on the podcast. Thank you, Sam. I'm really excited to be here. How did you get interested in artificial intelligence? I was working for a company in Nashville. I'm just by nature a pretty curious human. And I remember the first time that I typed into ChatGPT, it was a really simple prompt. and it was like magic. I put in the sentence, I think it came back with like two sentences or something. This is back in early 2023. And I just remember thinking, wow, this is incredible. I'd never experienced anything like this. What also stood out to me was there are very few moments in human history that are this democratized. I was sitting around a table with the CEO and engineers and all our data scientists. And I was a product person and we were all figuring it out together. Literally every week we'd show up and be like, oh, I learned this or, oh, I tried this. And there's very few moments like that where nobody knows what to do with this technology. And I had the thought of, I want to help lead that. So fast forward, ended up getting in contact with the team at Wendy's, was absolutely fascinated by the work. It was still very, very early with Fresh AI. The thing that has driven me to this day is, yes, we can automate things. But the thing that's really fascinating to me, and that's helped drive the work that we've done with Fresh AI, is how do you do something different? How do you use this technology to serve customers in ways that just weren't possible before? For us, that looks like creating accessibility. It looks like creating experiences. But it all stems back to, wow, I've never experienced anything like this before, what are the cool ways that we can use that? Yeah, that's a great point because there have been no human species before that have ever had this experience. So we're all having to learn it together. And I like your perspective on it brought everybody down to level ground. I think about that with the students that I talked to in class. They're thinking about what they're going to do in the job market. And, you know, I like to remind them that nobody has more than three or four years experience in this max. So you mentioned Fresh AI. All right. Tell us about Fresh AI, how it works, and I think that'll lead to some of the discussions about accessibility and things that you alluded to. Yeah, so we started out with this question of, can you take an order in the drive-thru? Right now, 2025, moving to 2026, that sounds fairly simple, but at the time, that was pretty groundbreaking. Can you handle this? The drive-thru is a really complex place. There are something like 200 billion ways to order a Dave's Double. That's just in English. If you try to order our menu, it's like 67 trillion different ways to order in English. Everybody has this feeling of, oh, I just have a normal order. If I've learned anything in this process, there is no such thing as a normal order. Customers will pull up and say, remove that green thing or, oh, wait, that last thing, I don't want that. And we say one thing, but we intend something entirely different than the words that we use. And so it's really a giant, complex math problem with a lot of emotions tied up. But, you know, drive-thrus are complex places. Mom pulls into the drive-thru. Kids are screaming in the back. There's a line full of cars. She's tired from a long day at work. And she just wants her food and she wants it fast. There's just all of this pressure and stress. They're trying to budget friendly. And then somebody comes over the speaker box. You know, hello, can I help you? Sometimes friendly, sometimes not. And so we set out, can we take orders? And quickly realized there's so much more opportunity to use AI. With Fresh AI, a customer can pull up. They can order in either English or in Spanish. Currently, we're working on expanding to other languages. The agent completely handles the order. And so it'll say, welcome to Wendy's. What can I get you? And you can throw almost anything at it. The agent's going to be able to handle it. And then on screen, you're going to see on a digital menu board, you're going to see your order. We've done a lot of research to understand how do we help guests looking at the cognitive load order in an easier process. And so one of the ways that we do that is we actually will show you the transcript so you can see what the agent is saying. You can see what you're saying. And it does a couple of things. It gives you a little bit of trust to know, oh, that it heard me correctly. And then also for customers or guests that might have hearing challenges, it also just gives that reassurance of I know what's happening on the screen. I like your point there that we all think our order is normal, but everybody's normal. It reminds me of a study back in the 50s in the Air Force where they measured the average pilot. And the average pilot had arms of this length and a height of this length, et cetera, et cetera. And they found the average. And it turns out that there was exactly zero pilots that were average. You know, everybody was average, but there was no one that was exactly that average specification. And so, yeah, I think my order is normal, but I guess everyone else thinks that their order is normal, too. It amazing to see you know we started to see all of these regional trends There are mustard parts of the country There are ketchup dominant parts of the country How accurate are these systems at getting these orders right I mean, I'm not even thinking that the humans are accurate 100% of the time either, though. That's not a good benchmark. They definitely are not. Right now, we are handling about 95% of the orders that come through. It's about 150,000 orders every day, and we're continuing to add sites. What we see, it is extremely accurate. and its consistency. I feel really strongly like if we want to build the right thing, we have to understand the customers that are going through the drive-through and understand what are their needs, what are their preferences. I like the idea of the confirmation too. You're talking about the screen. You know, one of the things that's always a mystery historically is you order something, you come around, it's not what you wanted. You at least can get that confirmation that what has happened has happened. Talk about how the accessibility part of that works. Yeah, so we've done quite a bit of work around this. What drives me with Fresh AI is a customer should be able to pull up and order in a drive-through in any language with any need. And it just works. It should be an afterthought. I spent a little bit of time in the Middle East. And when I was learning Kurdish, I walked into the Dukan, the market, and I just needed one cucumber. I walked out of that store. I have a master's degree and I couldn't communicate. I want one cucumber. I walked out with two bags and it was humiliating. And there's deep emotions with that. And it stuck with me. And one of the things that I, you know, as we've been building out this platform is nobody should feel that if you pull up and you don't speak fluent English, that shouldn't be a barrier. So how do we widen the road? So we've done that in a couple of ways. we've built Fresh AI in tandem with Google Cloud. And so we've worked really closely with them to train out our Spanish model. And what's really cool with that is, you know, before, say, in rural Ohio, we may or may not have crew that speak fluent Spanish. And so that customer, they can choose to order on the app, or maybe they pull up and they only know the word for hamburger. And so they might get an order, but it might not be the order that they want. with Fresh AI, a customer can pull up. I actually saw this in Miami recently. It's a location that is Spanish dominant and a customer pulled up and was able to order in English. And so the flip of most of what we've been training them, what we see is customers are spending more. There's more to light and it's just easier. You've removed that barrier. So that's one of the ways we're continuing to expand that. The other area that I'm incredibly excited about, we've recently brought in a speech therapist. And one of the pieces of feedback was the agent's interrupting. And it is speech-to-text. It's a speech-to-text model. And so language is incredibly hard to guess. Where do you jump in? Where do you hold back? And so those are some of the things that we're still getting right. And so one of the pieces of feedback was the agent is interrupting too much. And because of that, Phil's rude. And so what we did was we said, all right, what's the edge case? Like, what's the most extreme that we probably should account for? Well, one of those is atypical speech. We have something like 10 percent of the U.S. population has some sort of stutter. And so we said, all right, well, can the agent handle that? And so we brought in a specialist and literally just spent a day driving through our drive through, driving through competitor drive throughs. And I'll tell you, one of the things that we saw at a competitor drive-thru is we pulled up and his name's Travis. Travis went to order and there's a privacy screen. And in order to get past it, you have to say, OK, I'm ready. But you have to do it within a certain time frame. Well, because of his stutter, he wasn't able to do it. And so he was literally barricaded from being able to order, which was a terrible experience. It was also just really bad for business. You know, Travis is like, I would never go back to this place because that was so bad. And so, you know, thankfully, we pulled up to a fresh eye location and it handled his order perfectly. But we spent literally two hours, him just going through the drive-thru, trying to test this as hard as we could. And what's amazing is we learned so much on pages and pages of feedback. And we go and roll out those updates and those updates are now helping somewhere around 10,000 people a day. Like that's the power of this technology to help the business, but also to do really good. And we're not even scratching the surface yet. One of the fun things that's come out of a lot of episodes, and I go back to one of our early episodes on DHL, the first day is the worst day of AI. What you've done is provided an excellent example of that. As these systems, if you can get a minimum in place, they can start to get better all the time. And you say English and Spanish now, but there's nothing conceptually that prevents you from going further than that. And as you said, that gets rolled out in the software update. That feels pretty huge. One thing that came out very strong is the idea that you can meet people where they are. And the idea that we could, for example, train a lot of workers to recognize different languages, recognize different regional differences, even my beautiful southern accent. And, you know, Will has a beautiful southern accent, too. We could do that, but it's really hard to do it at scale. And so what this is pointing to is the ability to use this technology to reach people where they are. And your examples of the impairment give a lot of hope that we can go a lot further with that beyond the drive-thru. One of the things that fascinates me with, you know, I think the drive-thru is just a starting point, but I see this technology, other technologies. I mean, retail is just changing overnight. And what is really cool is it allows us to do things. Again, you could never do this before at scale and consistency so that you can guarantee, hey, if I go to this location in Miami, it's going to be personalized to me. But it's going to be the same experience and consistent quality as, say, I go to the UK or Italy. It doesn't matter where I am. It's a consistent brand experience every time. And the way that we've tried to approach this is, what are the things that AI can do really, really well and let it do that? And then what are the things that a human does really well and empower them to do that exceptionally well? And so there's this ability to free up the human labor to really focus on the things that are really human and to have this consistent experience that was not possible before. So all this seems wonderful. What are the problems? What are the difficulties? And is it all just technical difficulties with speech to text? What are other problems that other people might face if they try to get inspired by doing a similar thing? There's been two big challenges. One is just data. Again, there's 200 billion ways to order a Dave's Double. If the Dave Double has a different ID in one location than another location it not real to the agent And so that takes a different way of thinking in how you structure that data and how you give it to the agent Drive are really complex. Real life is really complex. There's things like regional campaigns or, you know, oh, that snowstorm hit randomly. And so, you know, we're discovering new things every day. We go, oh, we need to account for that. Or, oh, that's something that doesn't exist in the data. But the other challenge I would say is, and I think this is every business will go through this. Change is hard. That change management is hard. I see this technology. I see where it's going. I see the good that it does. But it's almost similar. The first time I drove a Tesla and I put it on autopilot, there's a trust that you're letting go. There is so much emotion to trusting these agents and trusting these systems with your business, with your livelihood, customer satisfaction. And that change doesn't happen overnight. And so building the right platform is just half the battle. The other challenge is getting everyone else to see what is the vision, where is this going, and do I trust this car to drive me down the road? So how do you do that then? I mean, I agree that it's a hard thing. How have you tried to work to solve it? So it's looked like a couple of things. I mean, number one is just having a really great team. In this age of AI, I think the best skill set that anybody could have is a deep curiosity and hunger. I was meeting with some students last week and was talking to them as they're trying to navigate, like, what do I do with my career? And everything's changing so fast. And what's crazy is one of the things that I've started doing is, you know, when I'm going down the road, I'll put ChatGPT on voice mode and I'll just say, tell me something new. Help me learn something new. You have Einstein in your pocket. You have access to information that was not possible even two years ago. So the knowledge, the information, that's readily accessible. What's not as accessible or just apparent is the right questions or just that being hungry to learn. And so if you get a really great group of people around you that are all asking great but different questions or that bring different points of view, it makes that product, it makes that experience a lot stronger. The other thing that I would say that we've done is, you know, again, it's not pretty, but it's just sitting and watching, asking customers. I will get on TikTok and I'll just watch. Customers will post their videos of their experience of going through and using the platform. And then 100 people will comment and say, oh, that didn't work. Oh, that sucks. And they'll just post what they it's just unfiltered their thoughts and opinions. Sometimes great, sometimes not. That's really, really valuable feedback to go, oh, we hit it there. Oh, we didn't get it there. We need to go back and make that better. And so if you have a great group of people around you and you're constantly learning, you're constantly asking questions. My hope is that, you know, a year from now, we look back and we go, man, that was so primitive. that was so basic, we've come a really far way. There was a time where I would worry, oh man, that's a $100 order. I don't know if I can handle that. Or, oh, they said that weird. I don't worry about that now. We've moved well past that. It's that building that trust, again, that Tesla example of just letting it go and trusting that it can take the wheel. Has using artificial intelligence in the drive-thru, has it changed how you measure things? What is different about metrics now? There are those traditional KPIs, you know, check size, speed of service, et cetera. What has changed is the insight. So one of the metrics that we never would have looked at before is the number of sorries. As we're looking for friction, well, if the agent or a customer is saying sorry, that means something went wrong. And so that was a new one for us to go, oh, wait, maybe we should start tracking that. And so we did this whole analysis. we've reduced the number of times the agent says sorry by about 60% over the last few months. And, you know, that's one of those not obvious metrics, but it's made a huge impact in the customer experience. I love the idea of metrics on that because once you say that it seems so obvious, but it would have taken me forever to figure that out. How did you figure out that that was the key word to look for? Again, it just comes to so much of this is the more you spend time, you start to see these patterns that only are there from, I've just spent so much time in the drive-thru or interacting with customers or watching, the agent sometimes will say the same phrase over and over. And so you'll go, oh, wait, why did it keep saying sorry then? Wait, let's see, maybe that's a metric that we should dig into. What's fascinating in this process, and we've talked about this, there's no rule book for how you build generative AI products and platforms. so much of this is it's being curious and hungry and just like as we spend time you're just sort of making the playbook as you go you know sitting in a room with google and their engineers are oh we've never done this before and you're figuring out together like there's something really beautiful and energizing about that experience and things like the sorry metric come out of that you know yeah it seems like you're approaching this very much from a learning perspective when if we think about the benchmark i mean i hate to break it to you but human customer service people are not perfect. And, you know, we mentioned driving a couple of times. Humans killed 40,000 people last year in car accidents. So our benchmark on these technologies is often they have to be perfect on day one when the reference standpoint is nowhere near perfect. And it seems like you're really approaching that from an ability to learn and how do we learn from we'll make some mistakes, how do we get this right? Where do you go to get these ideas and to get innovative ideas and new things to try? I am so glad you asked. That is one of my favorite things. I am not looking to competitors. Obviously, I'm aware of what's happening in the space. But one of the things that we've done is we've spent time at Disney World. Disney Imagineering and Disney Parks have done an incredible job of getting guests in a line to enjoy it. You're interacting, you're having fun. And literally one of the things it's like, can I get somebody to say, I want to go to a drive-thru in the same way that they say, I want to go to Disney World. We're not quite there yet, but I think we- That seems like a pretty hard, pretty lofty goal. But I think it's very doable. Could you pull up an order from your favorite character, or we've already started playing around with Easter eggs, or if you say a different phrase, it'll unlock certain content in the drive-through. Oh, I like that. Actually, I hadn't even thought about that. I'm thinking about, oh man, can we get the order right? That just shows my engineering mind going down the path. Rather as, yeah, put it Easter again, or have them talk like a pirate today. Exactly. And that's just one of the many ways that you can just put in these little moments of delight that take this thing from being oh that was really frustrating to oh that was actually really fun She was kind of snarky there We were at the MIT Museum and one of the things there was research that was done on I believe it was called Neuroquery And it was understanding how typing could predict brain health, Parkinson's, et cetera. But it was that type-to-text and the sequences that you see, what does that tell you about? like how that person is functioning. And so while we're not quite doing that, literally that visit was an inspiration for, oh, wait a minute, we have 150,000 speech to text orders. Can you start to see patterns? Can you start to see, oh, they said chicken, they didn't say chicken. What can you learn from that to make this more efficient? One of the things we realized was, oh, it takes 12 steps, we call them turns. So, you know, I say something and the agent says it back. It took about 12 turns to order one of our main menu items. And we said, oh, that's not great. How do we reduce that? Because anytime there's friction, it costs money, it causes frustration. And so getting down to literally to the syllable level, we were able to reduce that down to three terms. And so it's a lot faster. It's a lot more efficient. So the best inspiration is like I call it this like what if mentality. It's like, go get out of your office, go get into the real world, go sit in that drive through, go to competitors, go to Disney World and just be really curious and hungry. And it's sort of similar to the iPhone keyboard. I was reading a book recently and was talking about how, you know, nobody had ever done, they'd done similar things, but not quite like Apple did with the first iPhone where you have typography and art and then you infuse that with this technology and then you get this glass touch keyboard. That's actually kind of fun to use, sometimes frustrating, but it was the first of its kind. And it was this mirroring of two different fields and you put it together and that's where innovation happens. And so, you know, for me, it's looking at what is Disney doing when I'm ordering, you know, one touch from Amazon or going to that museum down the street, you have to get out. That's where those ideas come from. How did you get into this? Tell us a about your background? I have the most eclectic background and it's something I'm actually really proud of. I did my undergrad in foreign policy. I ended up moving to northern Iraq and doing a startup. We started a school for refugees in the Middle East. Came back, was really interested in products I'm obsessed with. Just what is that customer interaction? How can you use these products and technology and software to create, again, a Disney-esque, oh my gosh, that was cool. I will never forget you've created that moment. So when I moved back to the US, I started getting more into the product and technology space. And I feel like we're in one of those opportunity moments right now where it's every person in every business is trying to figure out, what do you do with this? And so for me personally, it's my own curiosity on what this thing can do. And it's just led me down this really fun path. So we have a segment where I'm going to ask you a bunch of rapid fire questions. So just think about the first thing that comes to your mind. Okay. What are people doing wrong with artificial intelligence? They're applying it for the sake of adding AI. There's no real value add. Everything looks like a nail if you have a hammer. Exactly. So what do you wish that AI could do better? I think be more context aware. right now it takes so much work to, if I want it to respond to an email or I want it to respond to, hey, I have this problem. There's so much context I have to give it for it to give me the response that's actually helpful. I think that's a problem that in a couple of years won't exist, but in its current state, it's so siloed off that it's only so helpful. Yeah. And that probably ties to our own reticence to share with the technology the way we share with a human. You can't expect context if you're not sharing that context at the same time. Exactly. Has using artificial intelligence made you spend more or less time with technology in general? I would say I use technology more, but in different ways. I find myself on my screen less. I actually spend a lot more time talking and asking questions and learning in a way that I didn't do. It's so interesting to go and just see, oh, I have this itch. Let me go ask it about, you know, yesterday I was asking about the Cold War and things like that. Like, the sky's the limit on what you can learn. And it's right there in your pocket. It's fascinating. What's the biggest misconception that people have about AI? I think the thought is AI will solve all of my problems. And AI is the answer. I firmly believe the fundamentals don't change. So for UPS, I need to get packages from one point to the other as fast as possible, as cheaply as possible, and make it great for the customer. That has nothing to do with technology. That's just a fundamental value. For us, it's delivering fresh food consistently with a great experience every single time. AI is a new way. We can serve with accessibility or with the experience. It allows us to serve those fundamentals in a way that we absolutely couldn't do before. But that fundamental doesn't change. It doesn't matter if Gemini, there's a new model in six months or, you know, OpenAI does something crazy in the next year. It doesn't matter. That fundamental is still going to be there. One of the things I like about these episodes and about this podcast in general is that lots of times I talk to people and they open up ideas that I'd never thought about before. and certainly had not thought a lot about the drive-through experience. And it's obviously much more complicated than I ever dreamed. But I also like that you connected to the idea of being able to serve lots of different people in a way that's best for them. Thanks for taking the time to talk with us today. Thank you, Sam. It's been a pleasure. Thanks for listening, everyone. We'll be bringing you some bonus episodes this winter while we ramp up for season 13, premiering March 10th. Thanks for your continued support of our show and thanks for listening. Thanks for listening to Me, Myself, and AI. Our show is able to continue in large part due to listener support. Your streams and downloads make a big difference. If you have a moment, please consider leaving us an Apple Podcasts review or a rating on Spotify. And share our show with others you think might find it interesting and helpful. Hi, listeners. We're running a short survey to learn more about our audience so that we can continue to bring you a podcast you find helpful. If you have a moment, please take the survey at mitsmr.com slash podcast survey. You'll receive a complimentary download of MIT SMR's executive guide, How to Manage the Value of Generative AI. Please take the survey this month at mitsmr.com slash podcast survey. We'll put that link in the show notes, and thank you for your help.
