Hard Fork

A.I. Goes to War + Is ‘A.I. Brain Fry’ Real? + How Grammarly Stole Casey’s Identity

67 min
Mar 13, 20263 months ago
Listen to Episode
Summary

Hard Fork explores AI's role in the Iran war, including Claude's use in military targeting systems and Iran's retaliatory strikes on data centers. The hosts also discuss 'AI brain fry' - a new condition where workers feel mentally exhausted from managing AI tools, and expose Grammarly's unauthorized use of journalists' identities in their AI writing feature.

Insights
  • AI is being used extensively in military operations for intelligence processing and target identification, with Claude integrated into classified systems
  • Workers using 3+ AI tools simultaneously experience 'AI brain fry' - cognitive overload from managing multiple AI systems
  • Iran's targeting of data centers represents a new form of asymmetric warfare against AI infrastructure
  • AI companies are quietly removing military usage restrictions from their terms of service under pressure
  • Legacy SaaS products like Grammarly face existential threats from free AI chatbots that offer superior capabilities
Trends
Military AI deployment accelerating beyond intelligence to real-time battlefield decision supportEmergence of AI-related workplace mental health conditions as adoption increasesData centers becoming strategic military targets in modern warfareSaaS apocalypse threatening subscription-based AI writing toolsAI companies abandoning ethical restrictions under government and market pressureWorkers experiencing productivity paradox - feeling less productive despite AI assistanceSupply chain vulnerabilities in AI infrastructure exposed by geopolitical conflictsUnauthorized use of expert identities becoming common in AI productsShift from human-AI collaboration to AI oversight fatigue in workplace settingsGrowing legal challenges around AI companies' use of copyrighted content and personal identities
Companies
Anthropic
Claude AI model being used in classified military systems for target identification and battle planning
Grammarly
Exposed for using journalists' identities without permission in their AI Expert Review feature
Palantir
Built Maven Smart System integrating Claude for military intelligence and targeting operations
OpenAI
Signed Pentagon deal and removed military usage restrictions from terms of service
Google
Removed AI military usage prohibitions and approved for non-classified Pentagon use
Amazon
Data centers in UAE and Bahrain targeted by Iran, disrupting regional cloud services
Microsoft
Building AI infrastructure in Middle East and involved in Pentagon AI contracts
Meta
Removed military usage restrictions from AI model terms of service
Boston Consulting Group
Conducted research study identifying AI brain fry syndrome in workplace settings
New York Times
Published AI writing quiz that generated controversy and hosts work for the publication
People
Kevin Roos
Co-host and New York Times tech columnist who created controversial AI writing quiz
Casey Newton
Co-host from Platformer whose identity was used without permission by Grammarly
Julie Bedard
BCG researcher who led study identifying AI brain fry syndrome in workplace environments
Julia Angwin
Investigative reporter who filed class action lawsuit against Grammarly for identity misuse
Timnit Gebru
AI ethics researcher whose identity was used without permission in Grammarly's system
Kara Swisher
Tech journalist whose identity was used without permission in Grammarly's Expert Review
John Carreyrou
Investigative journalist whose identity was used without permission by Grammarly
Stephen King
Author whose identity was used without permission in Grammarly's Expert Review feature
Neil deGrasse Tyson
Scientist whose identity was used without permission in Grammarly's system
Quotes
"Claude has been integrated into Maven Smart system since 2024. And the reporting that I've seen over the past week said that this combination has already suggested hundreds of targets, issued precise location coordinates, and prioritized those targets according to importance."
Kevin Roos
"14% of people who use AI said that they felt this. And I was especially surprised by the extent to which they told us about it. People wrote things like 'Feels like I have 12 browser tabs open in my head.'"
Julie Bedard
"If you are a paid subscriber to Grammarly, you are paying a subscription to get Grammarly to hallucinate on your behalf. To make up a bunch of stuff that is not true."
Casey Newton
"All of these companies were run by people who at one point thought this was all a bad idea to be selling these very advanced AI tools to the military. And then they changed their minds."
Kevin Roos
"Iran put out a statement that said that they had gone after the data centers to identify the role that they played in supporting the enemy's military and intelligence activities."
Casey Newton
Full Transcript
3 Speakers
Speaker A

Taxes can feel confusing when you're trying to manage everything by yourself. But with Intuit, TurboTax help is closer than you think. Stop by one of their new state of the art store locations. There you can ask a tax expert to sync your accounts via the TurboTax app and automatically import your documents. With TurboTax Full Service, your expert looks for every possible deduction while keeping you updated every step of the way. In the app, eliminate the guesswork and file your taxes. With a TurboTax expert today. Visit TurboTax.com well, I'm having sort of a weird day.

0:00

Speaker B

How so?

0:30

Speaker A

Well, I woke up this morning and I, you know, checked my social media feeds and I saw messages like the following. You're garbage and I hope you lose your job and become homeless. God, what a waste of sperm you are. And if you have never seen a message like that before 8am you might not work for the New York Times.

0:31

Speaker B

Well, I suspect that I know what this was about, but tell the listeners what what made people so mad?

0:54

Speaker A

So my colleague Stuart Thompson and I recently published this quiz, which is basically a set of AI written passages next to unlabeled sort of works from masterful human writers. And it was sort of designed as kind of a blind taste test where you pick which one you liked better and then it would tell you which one is generated by AI and which one was written by a human. And Casey, people did not like this quiz.

1:00

Speaker B

Well, what were the findings of the quiz?

1:26

Speaker A

Well, so the big headline finding is that like, it's basically a coin flip. Like a slightly more people, at least so far, have preferred the AI written passages. But when you tell them that they prefer the AI written passages, they get

1:28

Speaker B

very mad because they think that they are too smart to fall for a writing. Yeah.

1:42

Speaker A

Or they just don't like the way that the test was constructed or they just, it makes them uncomfortable. Or they think, you know, we're, we're cooked now that I can write passable versions of this thing. Or they just start saying, you know, oh, it's just, it's because it was trained on all these books, so obviously it can sort of mimic them. So I think there's a lot of different emotional reactions, but mostly the emotional reaction has been to get mad at the people who made the quiz.

1:49

Speaker B

I have to say you seem excited about this. Like, whenever a large group of people gets mad at you, you experience a glee that I have, I rarely see in people.

2:12

Speaker A

It's not a glee, it's just like, yeah, you're right Maybe it is a little bit. Okay. I'm Kevin Roos, a tech columnist at the New York Times.

2:21

Speaker B

I'm Casey Noon from Platformer. And this is Hard Fork. This week, how AI is reshaping the war in Iran. Then researcher Julie Bedard joins us to discuss the discovery of a strange new condition they're calling AI brain fry. And finally, I was turned into an AI editor against my will by Grammarly, here's how I stopped. It involved overwhelming physical force. All right, Kevin, let's get into the biggest news of the week, which is the war in Iran. Specifically, we want to talk about what we know about how AI is being used in this fight. Yeah.

2:34

Speaker A

And I think the reason to talk about this is not just because it's happening. It's the biggest story in the world, but also because. Because I think this is really a turning point in the use of AI in the military. We've been hearing for years and reading science fiction books and listening to people talk about the use of AI in military applications. But now I think we are starting to see exactly how these tools are being used on the battlefield and what kind of effects they might be having.

3:18

Speaker B

We are. And I'll say up top that anytime you're talking about the use of technology in war, there is always the risk that you are just passing along propaganda. Right. Both the military and the contractors have a vested interest in telling you, hey, we have some real gee whiz, new stuff and it's totally changing the game.

3:46

Speaker A

Right.

4:06

Speaker B

Everybody has an incentive to tell you that. And yet, as you and I have dug into it, we do believe that there are some notable ways that AI are being used. And I think it is worth mentioning them, if for no other reason than I think it's been the experience in the United States over the past couple of decades that tools that are deployed abroad during times of war sometimes come back home after the war and wind up being used against American citizens.

4:06

Speaker A

Yeah. So I think we should tease apart a few things here, one of which is like let's talk about how the actual AI tools are being used by the military, what the tools are, what the kind of ramifications of using them this way are. We should talk about how Claude in particular seems to be a key part of the war in Iran so far, and at least from what we know, seems to be behind a lot of the strategic decisions and operations that the military is making. And finally about how this conflict is or isn't going to reshape the future of AI by doing things like taking Aim at data centers by interrupting the supply chains of things like semiconductor materials. All the larger questions about how this conflict is playing out.

4:31

Speaker B

And before we get into it, let's briefly do our disclosures. My fiance works in Anthropic and I

5:12

Speaker A

work at the New York Times, which is suing Open AI Perplexity and Microsoft over alleged copyright violations.

5:17

Speaker B

Okay, Kevin, so where should we begin?

5:22

Speaker A

Well, let's talk about how AI is actually being used in the war in Iran and what we know about the actual deployment of this stuff. Casey, what do we know?

5:23

Speaker B

Yeah, so I read a great overview this week in the Wall Street Journal by Daniel Michaels and Dove Lieber, who goes into good detail about what we know about how the United States and the Israeli militaries are using AI. They're upfront about the fact that the military is trying to keep a lot of this secret. They are not apparently going into a lot of detail, but there are some things that we One is that Israeli intelligence for years had been monitoring traffic cameras in Tehran that they had hacked into and also eavesdropped on senior officials communications. And this is a big theme, Kevin, that runs through all of the coverage of AI in the war in Iran, which is that the military is saying that it is very effective, as you would probably imagine, at processing large quantities of information.

5:34

Speaker A

Yeah. So you've got all this data coming at you. If you're, you know, running a military in the year 2020, got data from drones and sensors and maybe security cameras that you've found a way into, and you can kind of use AI to process all of that, to put it onto some kind of like a real time dashboard so that you can just like open a screen and kind of see where all your supplies and all your troops and where all the enemy combatants are and like use it to sort of make sense of this wave of information that is coming at you every day.

6:25

Speaker B

Yeah. You know, recently on the show, as we've been talking about the conflict between Anthropic and the Pentagon, we've been talking about the potential eventually to have autonomous weapons out in the battlefield potentially killing people without human intervention. And the big message that I'm reading in the coverage so far is we are not there yet. Right. That the AI tools that are being used, we're seeing them in fields like intelligence, mission planning, logistics, actually pretty far away from the battlefield doing things like helping to find a target to send a missile at and then after an attack, trying to do some kind of quick analysis to see, hey, what exactly did we hit and maybe what should our next target be?

6:57

Speaker A

It's also really clear that what's happening in the military is what I would call like, shrinking the haystacks, where there's. There's sort of these massive troves of data where it's like we have, you know, hundreds of thousands of phone calls or audio recordings or emails or intercepted traffic to Iranian websites. And we can, like, use that to AI, to kind of narrow down the bits of that that might be useful to us. Because in all intelligence gathering situations since the dawn of eternity, like 99 percent of what you're collecting is totally useless. And there have been, you know, entire divisions of humans who have been employed to, like, dig through all that stuff and find the stuff that's actually useful. And now I can do that pretty well.

7:39

Speaker B

Yeah. And military leaders are saying that there are many, many missions that just never happened because they didn't have the manpower to do exactly what you just said. And now they do. And I would point out, Kevin, that again, you know, in our whole discussion of anthropic versus the Pentagon, we were talking about, you know, the risk of this technology being deployed against Americans and how effective that could be and, you know, all sorts of surveillance operations. So I think it's important to highlight, like that exact thing that we were talking about, like sort of like a bad scenario in the United States, if the government was doing it to its own people, is just sort of absolutely happening right now in Iran. Yeah.

8:25

Speaker A

And we probably won't know the extent to which it's happening because most of it is classified. And, you know, nobody in the military wants to, like, give away their secrets to, to any potential adversaries. But my best guess, and from the people that I've talked to who have been working on this stuff, is that this is happening pretty rapidly. That we are seeing many, many divisions of the military that are essentially using this stuff every day.

8:58

Speaker B

Yes. Now, one question that is coming up a lot is to what extent, if any, is the military starting to offload decisions to AI? Right. Is it the case that there is some military commander that is typing into a chat bot, hey, should I send the missile here or there? And the military's public statements are that they are not doing this right. They are. They are sort of taking care to say no, like, humans are in the loop here. We are relying on human judgment. But there are other experts that are saying, you know, at some point, if you're going to be consulting with a chatbot and the chatbot is getting smarter and Smarter before too long, it's probably not going to feel very different from the AI actually just making the decision for where to shoot a missile.

9:20

Speaker A

Yeah, I think that's a really good point. I think there is a difference between a fully autonomous weapon that can sort of do everything from selecting the target to, like, firing the weapon all on its own with no humans in the loop. But I think what you're talking about is sort of a system that can do everything except fire the weapon. It can sort of select the target. It can tell you the right timing. It can, like, identify all the objects in the surveillance footage, and it can kind of give the military officials the confidence they need to go ahead and push the button. And there's some worry that this is starting to happen with the help or the encouragement of AI. There was a missile strike in Iran that hit an elementary school the other day, and according to Iranian officials, killed over 175 people, mostly children. Horrible thing. And people have been wondering if that was related to Claude or some other AI system telling the military, maybe erroneously, that this was a legitimate target. Now, we should say that particular incident is still under investigations, and initial reports from the military have said that it was unlikely that AI was responsible in that case. But I think this is the kind of thing you're going to start seeing more and more of, is like, when there is an attack that, you know, kills civilians or doesn't hit its intended target, people are going to be asking, oh, was that a human who made that mistake, or was that an AI system?

10:00

Speaker B

Yeah. And I have to imagine, Kevin, that there is just going to be more and more pressure within the military to more fully defer these decisions to AI systems. Right. Because at some point there will at least be some contingent in the military saying, these systems are more trustworthy, they can make decisions faster, and let's do it. So I think that's just something that we need to be very much on guard for.

11:22

Speaker A

Yeah.

11:41

Speaker B

So that is what we know about how AI systems have been deployed so far. But, Kevin, as you mentioned, there's also been a lot of discussion about, well, what some particular models may or may not be doing during the war.

11:42

Speaker A

Yeah. And I think Claude and Anthropic have come up a lot in recent weeks for obvious reasons. They had this big fight with the Pentagon. But it's also the case that right now, in this war in Iran, Claude is the only AI model that has actually been deployed inside classified military systems. So to the extent that AI is having an effect in Iran, it is Probably Claude.

11:54

Speaker B

Yes. And the Washington Post had a story about AI and the war in which they said that Claude was so essential to operations that if for some reason Anthropic said, hey, we want you to stop using Claude, the military would push back and say, we're actually going to force you to continue to use this product. So just again, the continued strangeness of the situation, the Pentagon has now formally declared Claude and Anthropic to be a supply chain risk. This week, Anthropic sued over that.

12:14

Speaker A

Yeah. And there's also been a lot of reporting coming out over the past week or two about the actual ways that Claude is being used and deployed in the military. There's been some reporting on this system built by Palantir called Maven Smart System, which from what I can tell, is kind of a real time dashboard for intelligence that basically allows you to pull in a bunch of drone footage and sensor data and track a bunch of, you know, supplies and troop movements and things like that.

12:44

Speaker B

And by the way, this is the system that caused a huge controversy at Google in, in the late 2010s and, you know, Google's like, quit over this. They did not want the company involved with Project Maven and eventually Google dropped the contract. When they did, Palantir stepped in and eventually brought on Claude.

13:13

Speaker A

Right. And so Claude has been integrated into Maven Smart system since 2024. And the reporting that I've seen over the past week, including in this article in the Washington Post, said that this combination of the Maven Smart System built by Palantir and Claude has already suggested hundreds of targets, issued precise location coordinates, and prioritized those targets according to importance. And according to this same article, it says that the use of Maven and Claude has turned weeks long battle planning into real time operations. So this is not just like a kind of tool that people in the military are using for handling, like, routine office work. This is actually sort of a core part of their strategic decision making process.

13:32

Speaker B

Now, Kevin, do you know if this is a, like, specialized model of clot? Again, I'm thinking back to our conversation with Amanda Askel where she talked about all these efforts to make sure that, you know, Claude is really good. I'm sort of imagining that version of Claude being told, like, hey, analyze all this footage and decide, like, where to send a missile to kill a bunch of people. It's hard for me to imagine that version of Claude being like, yeah, yes, sir, right away. Right. So do we understand at all how that, how that is working?

14:16

Speaker A

So my understanding is that it is largely the same model that consumers and enterprises would use, but there may be some additional fine tuning to make it work inside these classified systems on these sort of military applications, that it may sort of refuse different prompts or fewer prompts than a model aimed at consumers, and that there may be some additional kind of changes around the edges, but that it's basically the same cloud that you and I have.

14:45

Speaker B

I see. Well, so this appears to be a very temporary phenomenon. We know that OpenAI has signed a deal with the Pentagon and presumably its systems will be onboarded onto classified defense systems soon. Gemini was approved for non classified uses at the Pentagon. So I think pretty soon the Pentagon is to have more options to choose from as it deploys these systems. Yeah, so that is how AI is being used offensively by the United States and Israel. Kevin. But we should also talk about what Iran is doing offensively against some of these AI systems.

15:14

Speaker A

Yeah, this is a part that I have not spent as much time looking into. So tell me what you're seeing.

15:53

Speaker B

Well, so as you know, there's been this huge build out of AI infrastructure throughout the Middle east over the past several years. We've seen these multibillion dollar projects being signed and built in Saudi Arabia and United Arab Emirates and Qatar. And these deals involve basically all of the big American tech giants, Amazon, Microsoft and Google. And I would say there's sort of like two major pieces of infrastructure that are relevant here. One is, is data centers. Right. Which are, you know, being used to run AI systems and also just provide basic cloud hosting and storage services to all sorts of companies. And then you have fiber optic cables which connect those data centers to the rest of the world. So let's maybe talk about the data centers first.

15:57

Speaker A

Sure.

16:44

Speaker B

So the Guardian reported that on the morning of March 1st, which was the day after the initial US attacks in Iran, Iran responded by striking a couple of Amazon data centers in the uae and they also damaged a third one in Bahrain. And in the immediate aftermath of that, people in those countries were opening up their phones and they couldn't check their bank balances, they couldn't order a taxi. It seems like a lot of services in those countries were being hosted on aws and they just didn't have access to those services anymore. Afterwards, Iran put out a statement that said that they had gone after the data centers to identify the role that they played in supporting the enemy's military and intelligence activities.

16:44

Speaker A

That's so interesting. So they were basically targeting data centers rather than, say, troops, because they thought it could actually be more disruptive if it turned out that the US Or Israel or any of the other allied nations were running their services on data centers located in the Middle East.

17:30

Speaker B

Yeah, well, I mean, and also, like, data centers are a great target. Like, they're just sitting there. They don't have any defenses. Right. So you can just send a few missiles over there and do an asymmetric amount of damage. And so now, Kevin, people are starting to question the logic of doing all these multibillion dollar deals in the Middle East. They're saying, hey, should this really be a linchpin of global AI infrastructure if it's just kind of a rough neighborhood and all of the investments that you're going to build there are just going to be kind of perpetually at risk?

17:46

Speaker A

Yeah, I think that's a really interesting sort of tactical shift that just speaks to how central all of this AI stuff has become in military conflict. And then you have all these other risks of disruptions to the supply chain. And right now, there are lots of ships stuck that can't get through this trade of Hormuz because it's been blocked off. And we now have people and companies saying that some of the raw materials that you need to make things like semiconductors might be delayed for weeks or months or however long this conflict lasts, and that prices might go up and it might get harder for companies to build new data centers here in the US So all of these ripple effects we're starting to see are, like, downstream from the fact that we're at war with Iran.

18:17

Speaker B

So that's what's going on with the data center infrastructure. Kevin, you're also probably wondering what is going on with these undersea cables, Right? So there are very important fiber optic cables that run through the Strait of Hormuz that are responsible for transporting Internet traffic from that region to the rest of the world. As of press time, as we record this, these lines have not been attacked or disrupted, but everyone is keeping a really close eye on it, because were they to be disrupted, there is just simply no obvious way to fix them in the middle of a live war.

18:59

Speaker A

Casey, how does this all make you feel that AI is playing such important and central role in an ongoing war in Iran?

19:36

Speaker B

I mean, this, to me, just feels like the frog is being boiled. Right? Like, when I think of all of the potential violent uses of AI, data analysis is not among those that gets me most nervous. Although, of course, I do have concerns about, you know, domestic surveillance. But I also know how rapidly these systems are advancing. I know the pressures that are quite apparent in our military to use AI for ever more things, I worry that there aren't going to be appropriate safeguards on those things. And so, yeah, I just have a high degree of concern about where all of this is going. I'm open to idea that AI systems could be used to wage war more safely and to maybe even prevent casualties, but I am not sure that we have built systems that will actually do that.

19:44

Speaker A

Yeah, and I would just say, like I keep thinking about how all of the companies that are building frontier AI systems today at one point in their existence had decided that they didn't want their stuff being used by the military. You know, back in 2014 when DeepMind was sort of little known AI startup in London, they sold themselves to Google. And one of the major sticking points in those negotiations, one of the reasons they sold to Google and not what became Meta and was at the time Facebook, was that Google had allowed them to have this prohibition on using their technology for military applications or surveillance. As recently as a couple of years ago, Google's AI principles said that we are not going to allow our technology to be used for the military. And in 2025 it quietly took that language out. OpenAI same thing. They had language in their terms prohibiting their models from being used for military applications. They took that language out quietly in 2024. Meta, same thing. Anthropic, interestingly, is the one sort of frontier AI lab that never explicit prohibition on military applications, but they did have a bunch of language in their original terms that they have amended to make it more possible for the military to use this stuff. And so like, I understand strategically why you would make the decision to sell your AI tools to the US military, but I just don't want us to forget that like all of these companies were run by people who at one point thought this was all a bad idea to be selling these very advanced AI tools to the military. And then they changed their minds. And they did that because of some combination of pressure or just maybe market opportunity to get these big military contracts. But they did at one point have a principle that involved we don't want our stuff being used to kill people. And I would like them to at least reflect on the fact that that has changed.

20:36

Speaker B

Yes. And for everyone else, the next time one of these companies tells you about some unshakable principle that is the foundation that the entire company is built on, it should make you wonder whether that can hold up to pressure as well.

22:40

Speaker A

Yeah. When we come back, are you experiencing AI brain fry? If so, you may be entitled to compensation. We'll talk to researcher Julie Bedard about this strange new AI psychological phenomenon.

22:53

Speaker B

The thing about AI for business? It may not automatically fit the way your business works. At IBM, we've seen this firsthand. But by embedding AI across hr, IT and procurement processes, we've reduced costs by millions, slash repetitive tasks, and freed thousands of hours for strategic work. Now we're helping companies get smarter by

23:32

Speaker C

putting AI where it actually pays off,

23:54

Speaker B

deep in the work that moves the business. Let's create smarter business IBM.

23:57

Speaker A

This podcast is supported by BetterHelp International. Women's Day is this March, time to celebrate all women, the leaders, the caregivers, the hype friends, the how do you do it? All types. Therapy can be a space to reflect, reset and reconnect. And better help makes it simple by matching you with a qualified therapist based on your needs and preferences. No pressure, no guesswork. This month, visit betterhelp.com newyorktimes for 10% off, that's betterhelp.com newyorkTimes when you use the trusted investing and savings app Betterment to help grow your money automatically, you have more time for new niche hobbies like collecting miniatures. The joy that brings helps you sleep better at night and even motivates you to always use your PM moisturizer. Now you've got a dewy glow and a sense of balance to match, not worrying where your money is growing. That's the Betterment effect. Get started today@betterment.com investing involves risk performance not guaranteed.

24:02

Speaker B

So Kevin, I feel like there is this new genre of blogs and social media posts all devoted to the idea that using AI is making people feel completely exhausted.

25:05

Speaker A

Yes, and insane.

25:16

Speaker B

There's a spectrum and it starts at exhausted and it goes all the way to insane. Siddharth Carr, who's an engineer who builds tools for AI agents, wrote a blog post that I saw all over social media recently, recently called AI Fatigue is Real and Nobody Talks About It. And he said that on one hand he felt like he'd had the most productive quarter of his entire life as he uses all these new agentic coding tools. But on the other hand, he said he had felt more drained than ever before in his career. Yeah, I think people are starting to

25:19

Speaker A

sort of use these tools more and come to grips with not only the effect it's having on their productivity, but also like on their brains and on their ability to kind of make sense of how quickly things are shifting. I really liked this essay that a venture capitalist wrote a few weeks ago about what he called token anxiety, which was this feeling that, like, if you don't have a bunch of, you know, Claude code agents, like running parallel tasks for you while you sleep, like you're feeling like you're missing out. And people at dinner parties in San Francisco are now talking about bragging about how many agents they have running at all times. So there's like something psychological happening to the people who are using this stuff a lot at work.

25:47

Speaker B

Absolutely. And recently we have begun to see some actual empirical research on the subject. So last month, researchers at UC Berkeley published some findings in the Harvard Business review from an eight month study observing workers at one 200 person tech company. And they found that AI was just making work a lot more intense. Workers were having to multitask a lot more. They felt like if they were not using a lot of AI tools, they were not keeping up with expectations and that they used to have little breaks during the day where, you know, you go to the water cooler and talk about, you know, what's going to happen on Survivor this week. Well, that doesn't exist anymore, at least not at this company. And then last week, a group of researchers at BCG shared some similar findings in the Harvard Business Review. And this one really caught our eye because they found that under certain conditions, workers are experiencing what the researchers are calling AI brain fry.

26:25

Speaker A

And to be clear, that is different than AI brain rot, which is what you get on TikTok when you start looking at videos of ballerina cappuccino.

27:23

Speaker B

That's right. You know, and actually they thought that Emmanuel Macron might have this, but that turned out to be AI French fry. So anyways, here's what AI brain fry is, Kevin. They're defining it as mental fatigue from excessive use or oversight of AI tools beyond one's cognitive capacity, which I think is kind of a funny idea. It's almost like you got a new co worker and they're really, really smart and it's sucking your life force out of your body. Yeah. So we want to know more about this study because I think it gives shape to a conversation that we're seeing Ripp out across the economy. As more and more managers are telling their workers to start using AI tools, it is clear that not all is well out there. People are starting to feel kind of bad and they're maybe going to be less productive and likely to leave their jobs as a result. So to learn more about the findings in this study, we've invited the lead author, Julie Bedard. Julie is A managing director and partner at Boston Consulting Group, as well as a fellow at the Henderson Institute, which is an internal research group and think tank at bcg. So let's bring her in.

27:31

Speaker A

Let's do it.

28:43

Speaker B

Let's get fried.

28:44

Speaker A

Julie Bedard, welcome to Hard Fork.

28:51

Speaker C

Thank you. Thanks for having me.

28:53

Speaker B

So let's talk about the study. You surveyed 1488 workers in January of this year from all different disciplines, lots of different companies. What kind of questions did you ask these workers?

28:55

Speaker C

Yeah, we asked them all kinds of questions around how they use AI, how they feel at work, you know, traditional burnout metrics. We asked some, you know, sort of proxies for cognitive ability. And we did throw in a question about AI brain fry. We said specifically, like, what do you think about this thing that could be AI brain fry? Like, are you feeling that?

29:10

Speaker B

And tell us how you define AI brain fry and what the survey results told you about.

29:30

Speaker C

I mean, we defined it as really like a type of cognitive strain. So we said it was mental fatigue. It was related to excessive use of, interaction with or oversight of AI, and it was about being beyond one's cognitive ability. So it's sort of like I'm using the tool, but it feels beyond my ability to process it. So 14% of people who use AI said that they felt this. And I was especially surprised by the extent to which they told us about it. We asked, you know, free ended like, just tell us, what is this thing? What does it show up? How does it feel to you? And people wrote a lot, right? Like, they wrote all these things about. Feels like I have 12 browser tabs open in my head. Or it feels like I'm working so hard to manage the tools. I'm actually not really doing the work. Like, I'm not actually managing what I'm supposed to be doing.

29:37

Speaker B

I thought this was so interesting because on paper, if you told me, hey, we're gonna give you a brilliant new assistant, they can answer all of your question questions. They can do many of the tasks that you prompt it to do. That would sound very exciting. You know, sometimes I think, what would it be like to have, like, a really great podcast co host? You know, somebody who kind of came in really prepared, asked a lot of great questions, had a great energy.

30:26

Speaker A

You'll never know.

30:48

Speaker B

You'll never know, and I'll never know. Okay, but some of these people at work are now having that experience, but what you're saying is that that is not an energizing thing for them. It's draining them in some way. So what do you think is the mechanism by which people are coming to feel so exhausted by working with these systems?

30:49

Speaker C

Yeah, Well, I do think it's particular to these two things that we found, which is the oversight of the tools and the intensification of work due to AI. And what people reported specifically is they put in more mental effort, they felt more fatigue, and they felt information overload. And, you know, we need more research. Right. Like, this is new and we're learning. But my hypothesis, right, from working with a lot of different companies on this kind of thing is it is fun and exciting combined with. We feel more pressure. Everybody's talking about AI, AI productivity. Right. And I think it's just nature. Okay, one more thing. Let me just sort of try this out, see what I can do. And we're not re. Centering on, like, what was I actually trying to achieve today. Right. We're not getting focused on some of the most important aspects of our work.

31:05

Speaker B

Yeah.

31:53

Speaker A

I'm curious how much you think this really boils down to fear. Because when I talk to people who are anxious about using AI at work, they circle around this issue that, like, maybe it's materializing as burnout or feelings of overwhelm. But, like, at. At its core, what they're nervous about is that we now have these systems that can do parts of their job and they're worried about losing their jobs. Did anything in your study sort of get to any of the economic or sort of survival anxiety that these workers might have been feeling that might have been registering to them as burnout, but deeper were else?

31:54

Speaker C

Yeah. So this is probably a good time to separate the two, because the brain fry is the cognitive piece. Burnout is, you know, physical and mental exhaustion. It's more emotional. It's more about how I feel about work and, you know, do I feel like I'm doing a good job at work? Burnout. We did not find a correlation with brain fry. So I just want to be really clear. It was very interesting. I thought we would. We did not. Brain fry is distinct. And then what we found is actually you could use AI to reduce burnout. So there's a lot of nuance. Maybe the last thing I would say is we did look at how positive or negative you feel. But typically the people who are afraid are not the people who are doing heavy oversight work, in my experience. Right. So they're sort of the people who are leveraging it more like a search tool. Right. They're not necessarily getting up that learning curve. Curve to more of the intensive Interactions.

32:30

Speaker A

In your study, you found that people in certain industries tended to experience AI brain fry more frequently. I was struck by marketing seems to be the place where people are feeling it the most. And people in areas like management and law and compliance reported significantly less brain fry. Do you have a theory on why that is?

33:26

Speaker C

Yeah. So the short answer is, unfortunately, our survey, at least scientifically, was not designed to answer that question. But I have my theories based on other work that I've done. And, you know, three years ago, I worked with some of the models to try to predict skill disruption. I was trying to figure out, like, which jobs will change the most. And one of the jobs that changed the most from a skill perspective was marketing management. A marketing manager was 90% disrupted from a skill perspective. So that's sort of the first fundamental piece about marketing is like, they've tended to adopt. And it's a really different way of working because of the power of the tools. The next thing, if I really just think about, like, what is brain fry? Like, it's about the iteration, it's about the oversight. A lot of marketing lends itself to that. Like, in the field, we see stories of folks who are doing image creation. They're doing synthetic consumer panels, right? They're spinning up a bunch of campaigns at the same time. And it really lends itself to that definition of, like, when do they know they're done? When do they know the image is ready? Like, have they defined those success thresholds for themselves? I'm guessing they haven't yet, Right? Like, they haven't figured out how do you do all the things to the right level of quality based on the outcome that you're trying to drive for.

33:48

Speaker B

It makes sense to me that, like, the more your job is changing, the more kind of vertigo you're going to be experiencing as these new tools are introduced into your workplace. You know, Kevin, you just observed that managers seem to be experiencing this less. One of my theories was that, well, the reason is because they're already used to overseeing a bunch of digital abstractions since they're human employees, right? They're mostly just sending them Slack messages and sending them emails, you know, hopefully meeting in person, you know, fairly regularly. But I think if you're a manager, you've already been used to sort of overseeing a bunch of stuff, and those people just sort of may have skills that people who have not yet been in management roles don't have.

35:07

Speaker A

I. I think there's a. There's something to that. And I also wonder, Julia, if you think there's any. Anything that is sort of inherently isolating about these tools. One thing that I've found with using AI for my own work is like, it's a single player video game, right? You're going back and forth with a machine. Very rarely am I in a room with other people using AI with them. And I wonder if part of the brain fry is sort of this siloing effect that these tools tend to have in the workplace, where it's like everyone is chatting with their chatbots and their agents and no one is talking to each other. Other.

35:45

Speaker C

I'm glad you brought that up, Kevin, because back to this point around, there's ways to use AI that actually reduce burnout. The people who were using it for repetitive tasks, they actually were doing those types of things. Like, we found that they felt more socially connected at work. And so it's interesting, like, in. In all the companies that I go to, I do various types of, you know, AI enablement and workshops. And one of the questions that I always get a lot of engagement on is what could you use AI for? Which is like the three worst things on your to do list, like the procrastination things. Like the things really wait and do. I mean, people love to talk about using AI for those. And my hypothesis is sometimes that's probably the repetitive work. And when you use it for that type of repetitive work, you actually reinvest the time in things that give you energy. So more work needs to be done. But I think I've seen that a bit in the field, and that's what our data would suggest as well.

36:18

Speaker A

I want to ask about the three tool cliff, which was a funny part of your study. Basically you found. Found that the sort of number of AI tools that people are using at work has some sort of bearing on their productivity or their feelings of productivity. And that actually when you switch from using three to four AI tools at work, there's something that happens where you all of a sudden start experiencing these things is not like a productivity enhancer, but actually just more of a stressful thing. Do you have a theory on why that is or why there seems to be this threshold?

37:09

Speaker C

Well, I mean, classically, multitasking is not very productive. Right. Like, we all are, you know, seduced by the idea that we can do more and more and more.

37:45

Speaker A

Yeah, Casey's playing Balatra right now.

37:54

Speaker C

Exactly.

37:57

Speaker B

I am not.

37:58

Speaker C

So, yeah, no, I think multitasking is part of that. But it's back to this point of like, I'm overseeing more things. Like, I'm actually doing more things, I'm starting more things, I'm stopping more things. I have more output to govern. Advice for leaders and managers are to help people understand this. One of the things I'd love to see is AI fluency right now mostly was defined by technical skills. Maybe in the last six to nine months we've started to talk about the human skills that persist. I actually think cognitive sort of health should be part of defining AI fluency as we go forward. So both again individuals, I can start to work differently with the tools. Tools but also again managers and leaders can, can help protect against that.

38:00

Speaker B

Let me ask one objection that some people might have to the research. You work for a consultancy, consultants have an interest in making AI seem difficult so that companies will hire them to help manage it. Is there any chance that we're over pathologizing what is going on here or sort of, you know, giving a scary sounding name to what might just sort of be a temporary adjustment process as as well people start to use AI tools in the workplace.

38:44

Speaker C

Yeah, I'm glad you've asked that. Maybe what I would say just first about kind of how I look at this and why I'm doing this research. So I am a consultant. Yes, I do advise companies. It's sort of the bread and butter of what I do. However, I'm also a researcher and I care really deeply about the data. And what's been very hard is our clients have wanted answers, answers that we don't necessarily have all of the playbook for because it's so new and is changing so rapidly. So I'd say just, you know, we really designed this to be a data driven intervention. But beyond that I think I've been for, like I said for the last three years at the rock face, like I've talked to more than 100 companies. I've actually trained teams myself. I've been in the room with software developers, marketers, etc. Trying to use these tools. And I see that like there's something there, like there's a real strain brain where I'm trying to do the right thing but something's getting in the way of me being productive with the tools. And we need to redesign work hopefully and particularly within teams to do that better.

39:10

Speaker A

And if you're a worker out there, if people are listening to this and saying, yes, I am a worker, I am using AI tools at work, I am feeling the brain fry that you are describing. What can they do do to help themselves? What has shown itself to be effective in your experience?

40:13

Speaker C

Yeah, so if you're an individual worker, I think first just acknowledging that this is a risk is the first thing. The second thing is really focusing on what you're trying to achieve. It's like back to that outcome piece. I mean, I know this is really basic, but if we were very clear about we're measuring outcomes, not output and we're trying to get to the right answer and what those steps to help me get there. And so from our data we would say the things you could do is one, engage your manager. So managers who engaged in questions, we saw brain frag go down and I think it's about creating that sort of open dialogue about how should I use AI, when is it valuable. The other thing is to engage your team on this. So interestingly, when teams were using AI together and they had better integrated it into their workflows. So like how I hand off work to Kevin and Kevin does to Casey, we also saw brain fraud go down. And you know, I don't have the data to say exactly why, but my hypothesis would be is we're not bottlenecking work in one person and we're creating actually like a much more effective system where we're getting the work done with the right outcomes together.

40:32

Speaker B

It seems tricky to me though because I think there is just so much thrashing around in organizations right now. Now I think that the amount of knowledge that any given manager or worker has about AI right now is highly variable. Whether their knowledge is, is like keeping pace with the capabilities of the latest models, that seems like an open question to me. So I have to say like in the near term I actually feel quite pessimistic about this. I'm sure there are going to be individual managers and teams that are like doing a great job, but at a like economy wide level. I think people are just absolutely all over the map on this.

41:41

Speaker A

Yeah, I think so too. And I think it's also not clear to me that people are going to feel comfortable talking to their managers about how they're feeling about AI because I think a lot of people have these reasonably well founded fears that like if you tell your manager like I'm using AI to do this part of my job, the manager's first thought is, well, going to be, well, maybe I can

42:18

Speaker B

lay you off, maybe I don't need

42:37

Speaker A

all these humans anymore. And I think we're seeing enough of that happening at big companies now where they're laying off big percentages of their workforce and attributing that to productivity gains from AI that I Think people are sort of feeling like, well, if I discover how to use AI for my work, I'm going to keep it to my damn self.

42:39

Speaker B

Absolutely. Or, Kevin, I think we also see the reverse of that, which is you go on social media and you see people bragging about the insane lengths that they are going to to be using AI at all times, to have their, you know, clod swarms up and running and coding, you know, while they sleep. And I feel this sort of deep insecurity embedded in that, which is if I'm not constantly telling you how much AI I'm using, you know, I might sort of be next on the chopping block.

42:56

Speaker C

My reaction to that is this is why leaders play a really important role. Because I think, Kevin, your point is well taken. I think there are things individuals can do. There are absolutely things managers can do, but this is about systemic redesign of work. So, Kasey, to your point, like, I don't think AI brain fry is going away unless we tackle it head on. Like, I don't think this is something that we can sort of just democratize and let everybody figure out it out. Although I think there are things they can do to mitigate. But I'm really interested in actually, like, okay, let's rethink how we get the job done. Like, you know, we are really bad at stopping work. Is all work valuable? Like, if we had leaders engage more meaningfully in these questions, that's the work we need to do if we really want to address some of this.

43:22

Speaker A

Julie, I'm wondering how much you went back and looked through sort of historical precedent here. When I was researching my last book, I. I was doing a lot of reading about the 1970s when a bunch of manufacturing workplaces, like auto plants, were getting all these new automated robots to help them do things like assemble cars. And there was this whole sort of nationwide panic about this. They called it Lordstown Syndrome because the first sort of GM plant to have this level of automation was in Lordstown, Ohio. And Congress held hearings about this, like, sort of new wave of worker alienation that was happening in these blue collar manufacturing workplaces for a lot of the same reasons that that to me seem like they rhyme with at least this AI brain fry idea. Workers were just saying basically, like, I don't feel like a human anymore. I feel like I just push buttons and the robots do all the work. I don't talk to people at the office anymore. My managers have all these crazy productivity expectations of me. And I think what was interesting in that, beyond just the Parallels to what people are feeling in white collar workplaces today. Today was that the way that they sort of got out of that was through striking and through organizing and unionizing and getting a bigger share of the profits that these companies were making from all this productivity. So I guess I'm just wondering if you could riff on maybe the, some of the historical parallels before and where this may all be heading.

44:07

Speaker C

Well, I always get the question around Excel and accountants, right? Like, did the rise of Excel lead to more or fewer accountants and, or even if you think back actually to the Industrial Revolution, one thing I actually think is a really interesting parallel. There is, you know, the rise of technology at that time. In many cases, it wasn't until there was actually a re architecture of the shop floor did we actually see the productivity gains. And to me, that's an interesting parallel to what we need to do with redesigning work.

45:32

Speaker B

Julie, one of the questions I wanted to ask you was like, like, you know, it is the role of the consultant to come in and say, I have talked to people all across this land and I understand the best practices and I will bring them to you and you can redesign your shop floor so that you can get back to being maximally productive. But I feel for Kevin and I, we feel like the ground never stops shifting under our feet anymore and that every few weeks some new model comes along where the level of capability goes up and, and maybe even something that I would not have been able to do in November. I actually can now. And before too long, maybe that's going to be a core expectation for me. That is part of my job. So part of me wonders, like, is this actually even a good time to be redesigning your workflows if, you know, three months from now, six months from now, the landscape might have completely changed all over again?

46:03

Speaker C

Yes. And I have tackled this question many, many times. Here's my take. For companies who didn't do anything two years ago ago, they would have said the exact same thing to me, Kasey. They would have said, the tech is going to change. I'm going to wait. I want to be a fast follower. And honestly, there is some smart truth to that, right? Like, pick your bets. Like, I definitely wouldn't be doing this everywhere, but I think this is about learning a new capability and muscle as an organization. Like, this is about teaching us how to change. So I would say, like, if you're not, if you're on the sidelines, yes, it's, it's just going to keep moving. So you could have that Excuse a year ago, two years ago, two more years. But you're also gonna be missing out on that opportunity to build capability as leaders, to build that in your teams, to start upskilling people. I think there's actual things that you can do to support your talent, to go on this journey with you.

46:57

Speaker A

Yeah. And I would say also if I could add something to that from 1972, which is apparently where I love going on this subject. There was this sort of team at GM when the Lordstown Syndrome was taking over that had to figure out how to bring back the striking workers. And one thing they did was that they set up these new humanization councils where basically workers, people from the assembly line were invited to give their thoughts on how the robots were being used and how the machines were set up and how the assembly lines were laid out. And feeling like they had some input and some control over their situation and were not just like passive bystanders actually seemed to help. So I don't know whether that's directly applicable to white collar workplaces that are going through this today, but I do think that having some of the energy and ideas come from the quote unquote bottom from the actual workers doing the individual contribution seems to matter.

47:50

Speaker C

Yeah, I mean, Kevin, that's absolutely right. Like how do we have more agency in this? And if you do that, you're going to be really user centric. You're going to think about like, what work do people enjoy doing? Doing what work do they not enjoy doing? What are some of the barriers, cognitive or otherwise, to getting actually that work done? I think that's exactly right.

48:47

Speaker A

Well, Juliet, thank you so much for giving us a lesson. Now if you'll excuse us, we have to go deal with our AI brain fry.

49:08

Speaker B

I actually have AI brain freeze. It happens if you use ChatGPT while you're drinking a Slurpee.

49:13

Speaker C

Well, as long as it's not AI brain rot, we're fine.

49:19

Speaker A

We got there a long time time ago. Thanks, Julie.

49:24

Speaker B

Thanks, Julie.

49:27

Speaker C

Thanks.

49:28

Speaker B

When we come back, the worst AI feature we've ever seen.

49:31

Speaker A

Makes you more like Casey.

49:36

Speaker C

Most all in one HR systems are a patchwork of disconnected and manual table tools. Rippling is totally automated. If you promote an employee, Rippling can automatically handle necessary updates from payroll taxes and provisioning new app permissions to assigning required manager training. That's why Rippling is the number one rated human capital management suite on G2 TrustRadius and Gartner. If you're ready to run the backbone of your business on one unified platform. Head to rippling.com hardfork and sign up today. That's R-I P P L-I N G.com hardfork to sign, sign up Adobe Acrobat

49:50

Speaker A

Studio, your new foundation. Use PDF spaces to generate a presentation. Grab your docs, your permits, your moves, AI levels up, your pitch gets it in a groove. Choose a template with your timeless cool. Come on now, let's flex those tunes Draft, design, deliver, make it sing. AI builds the deck so you can build that thing. Do that, do that, do that with Acrobat.

50:25

Speaker B

Learn more@adobe.com Dothatwith Acrobat.

50:50

Speaker A

When you use the trusted investing and savings app Betterment to help grow your money automatically, you have more time for new niche hobbies like collecting miniatures. The joy that brings helps you sleep better at night and even motivates you to always use your PM moisturizer. Now you've got a dewy glow and a sense of balance to match. Not worrying where your money is is growing. That's the Betterment effect. Get started today@betterment.com investing involves risk performance not guaranteed. Well, Casey, I heard you got an exciting new job last week.

50:54

Speaker B

I did, and it was the sort of job, Kevin, that I didn't even know that I had or was doing.

51:28

Speaker A

So you had this crazy experience by being selected against your will and without your permission as one one of Grammarly, the AI kind of writing assistant. They have an expert network of people whose voices they have borrowed for the purposes of, I guess, making people's writing better. So a congratulations.

51:34

Speaker B

Thank you.

51:58

Speaker A

I assume the royalty checks are just overflowing your mailbox, but what actually happened here? You had a fascinating newsletter about this this week.

51:59

Speaker B

Well, thank you. So this story I first learned about from the Verge, their reporter Stevie Bonifield wrote, and it turned out that last summer Grammarly had added this feature called Expert Review. I had not actually used Grammarly until this. Have you ever used it?

52:06

Speaker A

No.

52:24

Speaker B

So I decided, you know what, why don't I sign up for the free trial and see what what Grammarly can do for me. And if you go to the support page for this feature, it says that Expert Review quote is designed to take your writing to the next level with insights from leading professionals, authors and subject matter experts. That sounds pretty cool, right? Well, scroll a little further down, Kevin, and you see the following disclaimer. References to experts in Expert Review are for informational purposes only and do not indicate any affiliation with Grammarly or endorsement by those individuals or entities. And so I read that and I thought. When you say that these insights come from leading professionals, what does the word from mean to you? Because it sounds like what you're telling me is they don't come from those experts at all.

52:25

Speaker A

Yeah, it's like when you see like a tub of margarine and it's like, you know, it's like butter style product in very small type.

53:18

Speaker B

Yeah.

53:25

Speaker A

They have sort of an expert network with an asterisk. None of the experts were actually consulted and we didn't actually hear from them in any way.

53:25

Speaker B

Absolutely. So Stevie over at the Verge put a bunch of writing through expert review to see what sort of expert names would pop up. I was one of them.

53:32

Speaker A

Congratulations.

53:43

Speaker B

Thank you. You know, as you might imagine, Grammarly also picked a bunch of like actual famous people. So Stephen King, Neil DeGrasse Tyson, Carl Sagan. And I decided to put this thing through my own paces and loaded up some recent columns that we published in Platformer and pasted them in to see what sort of experts it would suggest. And while I was never able to get my own name, Kevin, I did see a succession of people that sort of felt like if you made a list of people who would hate this idea the most, that is who Grammarly had picked up. So Timnit Gebru, a very vocal critic of AI systems, the way they are built and deployed. She showed up as a quote unquote expert. So did Julia Angwin, who is an investigative reporter. She writes for New York Times Opinion and it used her writing, even though she has written a lot about how tax systems are used for privacy and surveillance in ways that are contrary to how we probably want them to be used. Julia, by the way, filed a class action complain complaint against Grammarly's parent company on Wednesday, seeking to stop them from, quote, trading on her name and those of hundreds of other journalists, authors and editors and to stop them from, quote, attributing words to them that they never uttered and advice that they never gave.

53:43

Speaker A

Wait, can I ask a question about the mechanics of this? Okay, so you're writing in Grammarly, which I gather is sort of like a bolt on to like a word processor.

55:04

Speaker B

Yes.

55:11

Speaker A

And it sort of detects the topic you're writing about and then pops up a little like, clippy thing that's like. Would you like Julia Anguin to edit this for you? Would you like Casey Newton to give this one a pass?

55:12

Speaker B

Exactly. I'll actually show you an example here. If you want to look at my laptop, you can see that here is the text that I wrote and Then in this little left hand column, in this case, it just says Kara Swisher, Cara Swisher, my good friend, Pass Hard Forecast, legendary Silicon Valley journalist and podcaster, and someone who has absolutely no involvement with Grammarly, but her name just sort of pops up there with no disclaimer at all, right? And then when you sort of click in, it will offer this sort of Cara inspired advice. And this is the point, Kevin, where I would like to talk about the kind of advice that this thing actually gives, please. So you might expect, given that they were, you know, allegedly trying to borrow the expertise of real humans, that that expertise would seem like, incredibly specific to that person. Right. Instead, what you're getting is just a bunch of very generic advice about something that you might do. So I noted, for example, that, you know, we public Mike, my colleague Ella Marchian sort of story in Platformer last week where she went to a protest at OpenAI and there was a suggestion that Grammar Lee had said was inspired by John Carreyrou, the legendary investigative journalist who brought down Theranos. And the advice basically boiled down to try opening with a colorful scene and use a lot of rich details and characters. Right? Like sort of the most absolute generic advice that you would ever imagine getting. And nothing like I would imagine the actual experience of sitting down with John Carrey Roo and saying, like, hey, how did you write Bad Blood?

55:25

Speaker A

Yeah. How did it say that Kara Swisher would edit a story?

57:01

Speaker B

So I will just read you the piece of advice that it gave me. This was also a piece of advice about this protest story, the fake AI Kara said. Could you briefly compare how daily AI users versus AI skeptics articulate risk? Creating a through line readers can can follow. A synthesizing sentence here may tighten the narrative arc.

57:05

Speaker A

I'm laughing because that is the exact opposite of how I imagine Kara Swisher would edit someone. Yeah, it would just be like a string of like four letter words and like, you know, this sucks. Do it over again.

57:29

Speaker B

Yeah. It would say, stop wasting my time. You know, like that. That would be the advice, the thing that I just read, I just wanted to acknowledge. Like, it is word salad. Do you know what I mean?

57:40

Speaker A

Totally.

57:50

Speaker B

Like, you can tell I don't know what underlying model they're using here. I'm guessing it is not a frontier one. Right? It's reading very like GPT2 to me, you know, so this advice is so bad, but let's bring this into what I actually find upsetting about this, Kevin.

57:51

Speaker A

Yeah, let's make this about You?

58:05

Speaker B

No. Well, here's the thing. I'm actually not going to make it about me because I have sort of just long since accepted that all of these companies have stolen all my intellectual property and are having their way with it. Where I really feel bad is for the subscribers to Grammarly. These people are paying $144 a year to be able to use this glorified spell checker, okay? And they load this thing up and then Grammarly gives them this service. And so if you are a paid subscriber to Grammarly, you are paying a subscription to get Grammarly to hallucinate on your behalf. Right. To make up a bunch of. Of stuff that is not true. Right. This is not the actual sort of advice that any of these experts would provide. And you are paying for that service when you just as easily could have taken whatever text you had written and pasted into a free chatbot and gotten generic advice that is just as not great as what you were getting here.

58:06

Speaker A

Right. And the, the truly crazy thing about this is that despite charging all this money for people to use this substandard AI product, they are not, to my knowledge, passing any of this along to you or Kara or John Carreyrou or any of these authors whose identities they have purloined for the purposes of selling this product.

59:03

Speaker B

No, they're not. And you know, look, I think that all of the AI companies just have a huge entitlement problem in general. You know, I think that they think, look, if it's the. If it's on the Internet, it is in the public domain and it belongs to us. And they don't spend enough time thinking about how they are destroying the incentives for anyone to create a public, open Internet. Right. If you feel like you're just kind of get screwed in this way. So I do think that that is really unfortunate.

59:22

Speaker A

Yeah. So what did Grammarly say when you started writing about this?

59:50

Speaker B

Well, when I reached out to them, they thought about it for a while and then finally came back to me on Monday and said, you know what, We've thought about it, and if you're one of our experts who we didn't consult and we're not paying, you can now opt out of this feature.

59:54

Speaker A

How nice of them.

1:00:11

Speaker B

So you can now send an email and I don't want to be a part of this system anymore. And so, you know, I wrote the story and got a lot of comments on social media like, you know, geez, that really seems like the least they can do. But Kevin, as we record this, I actually have some breaking news. What's that? So I got an email from the spokeswoman over at Superhuman today. Superhuman is what Grammarly now calls itself. They did a rebrand last year and they're now sort of a bundle of mediocre products. And they sent me a note and said that after careful consideration, we have decided to disable Expert review as we reimagine the feature to make it more useful for users while giving experts real control over how they want to be represented or not represented at all. Thanks for holding us accountable. We're committed to getting it right next time and we'll be transparent about how we improve from.

1:00:12

Speaker A

Wow. So results. Newton gets results.

1:01:07

Speaker B

Newton getting some results. I mean, look, it's clear to me that they are embarrassed about this, but this is one where the whole time I was using this thing, I was like, who was the product manager? What were the meetings? Imagine the meeting. Imagine, was there a lawyer involved in that? Who was the lawyer that signed off and said, yes, feel free to misrepresent that. You are getting inspiration from all of these different editors. So the thing is such a, like, spectacular misfire. And it really made me wonder, like, what is the future of a product like Grammarly? And like, that's kind of where I want to end this. You just finished writing a book. You presumably could have used some sort of AI writing assistance. Did it ever occur to you to use Grammarly?

1:01:11

Speaker A

No.

1:01:56

Speaker B

Why not?

1:01:56

Speaker A

Because I don't know anything about it and I don't need it and I have other tools.

1:01:57

Speaker B

Well, so talk to me about these other tools, because this is what I think the real story is, which is like, in 2009, when Grammarly launched, you didn't have a lot of options for writing assistance. Right. You had like whatever spell checker was in Google Docs and like, that was, you know, probably going to be the best tool available. Fast forward to today though. You got ChatGPT, you got Gemini, you got Claude. There are free versions of these services. If you want a quick grammar check, you can get it. My guess is that's. That's the experience that you just had.

1:02:01

Speaker A

Yeah, if I want a grammar check, I'm just copying and pasting into one of the AI models. I'm not using like a purpose built thing for that. Or it's now built into, you know, Google Docs.

1:02:27

Speaker B

Yeah. And. And to, you know, emphasize a point, when you're using Claude, as you did in your book, you're using the latest and greatest version of Claude. Yeah. If you are using some Sort of startup that is like using the API of Anthropic. They're not actually incentivized to give you the frontier model most of the time.

1:02:36

Speaker A

Right.

1:02:54

Speaker B

Because that's going to be very expensive. So they're going to give you a model that's a couple generations old because they can get a lower price and their, their margin is going to be better on it. So we've talked a lot in recent weeks about the potential for a SaaS apocalypse where these companies that are selling these sort of, you know, business prosumer services are going to get crushed by the fact that there is now just a cheaper way to do it. I wonder if you think the Grammarly might be one of those.

1:02:54

Speaker A

No, I think it's going to be part of the ass apocalypse, which is for software that absolutely sucks that there is no reason to be using in the first place place. And I think that, that that software has a hard roadhead.

1:03:19

Speaker B

I just do not think there is a future for this product. And like, like when I saw this, yes, I did have the moment of like outrageous. Too strong a word. I felt supremely annoyed. Okay. I did feel like very annoyed that this was happening. But again, it's like, I know all these companies have like all read my stuff. You know, you could go into clot today and say, draw inspiration from Casey Newton and edit my piece. Claude is not going to refuse and say I don't have the rights to his intellectual property. It's just going to do it. And it's not going to notify me and it's not going to. So I do think that there is a distinction between what these companies are doing. But I just want to point out that in some way, like the violation is the same. The bigger thing to me was this really feels like desperation, you know. And I think that more and more of these consumer sort of Internet services that have been able to get away by offering a pretty subpar product and selling it to you for more than $100 a year. I think the rude awakening is shocking. Showing up, you know, where all of a sudden if you have a subscription to your Claude or your Gemini or your ChatGPT, you're probably going to be able to get more from that and do more things and you're just not going to need the subscription anymore. It's exactly like when we were talking about Vibe coding and being like, why are we paying Squarespace all this money? Right. I think the why are we paying Grammarly all this money moment is coming.

1:03:32

Speaker A

Yeah. And I, I should say, if you want to rip off Casey Newton's editing style without his permission or without compensating him, you should just do that in a free chat bot. His advice is not worth that. Trust me. I have seen his edits and I would not pay $140 a year for them.

1:04:54

Speaker B

I'm a great editor. Okay, ask around. You should really ask around. I have a very detailed, thoughtful, thoughtful feedback. No, this.

1:05:14

Speaker A

This is horrible. I'm very glad you exposed it. I'm very glad they went back and said, we're not going to do this anymore. But I think this kind is going to keep happening, unfortunately, because there is money to be made and if you can get away with it, you're going to do it.

1:05:23

Speaker B

Yeah. You know, an interesting question might be like, is there a good version of this feature? And what would that be, do you think?

1:05:36

Speaker A

So if they had come to you and said, hey, Casey, we're starting this new expert review feature, and every time someone edits their emails to sound more like Casey Newton, we're going to give you 10 cents, would you have have done that?

1:05:44

Speaker B

I mean, I don't know. In general, I am in favor of AI companies trying to strike deals with creative people that say, like, we are going to give you some sort of, you know, we're going to essentially share the revenue that is based on the creative work that you have done. So certainly I would like to see some kind of explorations like that. But, you know, I think about some of the editing I've done. You know, I can remember, like, working with one writer once and she was working, like, on a kind of narrative feature story. And it just made me think of Catherine Boo, the. A great features writer for the New Yorker for a long time, wrote this incredible book, behind the Beautiful Forevers. And I was like, go read Catherine Boo. Like, go read Katharine Boo pieces in the New Yorker.

1:06:00

Speaker A

She's the goat.

1:06:41

Speaker B

And see how she, like, evokes characters and see how she kind of structures her narratives. And like, so can I imagine an AI tool that, like, you were having a conversation with that also said, like, you need to read some Katherine Boo. And click here. And hey, if you already have a New Yorker subscription, maybe you can log in right here and we'll sort of bring up some of the relevant passages. So, yes, I do think that there is value in sort of guiding writers to actual excerpts. The key is you have to guide them to the actual expertise, not just what your LLM is hallucinating.

1:06:41

Speaker A

Right. That would be my worry if they had come to me, which they did not. Which I'm a little bit offended by, frankly. They put you in the feature. Was I not worth ripping off? I'm right here. Grammarly, I'm pretty good. But no. Had they come to me and said, hey, we want to make you part of this, I would have said, well, how good is your model? Because, you know, my worry about something like that would be that someone would, you know, open up their. Their word processor and start writing their business memo and say, make it sound more like Kevin Roos. And then it would make it sound terrible and generic, and people would blame me and I would kind of get the bad rap for allowing my reputation to be laundered in this way.

1:07:14

Speaker B

I was texting with my friend Matt Honan, who's the editor of the MIT Technology Review, and he found that he was also being used as an expert. And when he clicked on the exper that he was allegedly providing the user of expert review, he looked to see what source they were citing. And it was a speaker bio that he had submitted from an event. So it's like, based on Matt's speaker bio, you should have used to work at Wired. Like, I. I don't even know what it said, but again, it's just like that. They just did not think this will. Yeah.

1:07:52

Speaker A

Well, now, as a result of Grammarly pulling back this feature request, if you want your emails to sound like Casey Newton, you're going to just have to put a bu of typos and random punctuation in yourself manually.

1:08:29

Speaker B

And if you really want to know what I think of your writing, it's that you should start a podcast. That's where the future's going.

1:08:40

Speaker A

Okay, Casey, I'm glad you escaped Grammarly Servitude. Foreign.

1:08:46

Speaker C

HR systems are a patchwork of disconnected and manual tools. Rippling is totally automated. If you promote an employee, Rippling can automatically handle necessary updates, from payroll taxes and provisioning new app permissions to assigning required manager training. That's why Rippling is the number one rated human capital management suite on G2, TrustRadius and Gartner. If you're ready to run the backbone of your business on one unified platform, head to rippling.com SL hardfork and sign up today. That's RIP P L I N G.com hardfork to sign up Adobe Acrobat Studio,

1:09:09

Speaker B

your team's home base, collaborate within a shared PDF space. You've got your docs, your plans, your specs, and then invite the crew to build what's next to talk off the teamwork. They think that this design could be a contender. When somebody wonders what's the next steps AI helps you finish the rest. Bolts are tight now your plan's refined. Run a smoother business when you're on the line. Do that with Acrobat learn more@adobe.com do

1:09:44

Speaker A

that with Acrobat when you use the trusted investing and savings app Betterment to help grow your money automatically, you have more time for new niche hobbies like collecting miniatures. The joy that brings helps you sleep better at night and even motivates you to always use your PM moisturizer. Now you've got a dewy glow and a sense of balance to match, not worrying where your money is growing. That's the Betterment effect. Get started today@betterment.com Investing involves risk. Performance not guaranteed.

1:10:11

Speaker B

Hard Fork is produced by Whitney Jones and Rachel Cohn. We're edited by Veran Pavic. We're fact checked by Caitlin Love. Today's show was engineered by Chris Wood. Our executive producer is Jen Poyan. Original music by Mary Lozano, Diane Wong, Rowan Nimisto and Dan Powell. Video production by Sawyer Roque, Pat Gunther, Jake Nichol and Chris Shot. You can watch this whole episode on YouTube@YouTube.com hardfork Special thanks to Paula Schumann, Pui Wing Tam, and Dalia Haddad. You can email us@hardforky times.com with whatever fake advice I just gave you in Grammarly and I'm sorry for it. Holy. They just disabled Expert review in Grammarly.

1:10:44

Speaker A

Whoa, you're free. ADP knows anything can change the world of work.

1:11:48

Speaker B

Like what if a blocked shipping port caused a worldwide coffee shortage?

1:11:55

Speaker A

Suddenly, businesses are scrambling to manage an increasingly drowsy workforce. Productivity slips 20% just due to the extended time it takes to conduct a job interview while yawning. Anything can change the world of work. From HR to payroll, ADP helps businesses take on the next anything. ADP always designing for people.

1:11:58