The AI Lie: ‘It’s not just misleading it’s dangerous’ | House of El: AI

This is where calling something inevitable becomes actively dangerous rather than just misleading. The big elephant in the room is the fact that the technology is just, in my personal view, generally not really ready for a lot of the things that it's being sold for. I don't think that this is talked about nowhere near enough as it should be talked about because I think we're having like another social media 2.0 crisis. The most sure way to lose a battle is to not know who your enemy is. Like you have to be able to identify what exactly you're mad at. You have to identify what exactly is generating this problem irrespective of feelings. Like what is the causal effect? Like what exactly is going on? Because you have to be specific so that you can actually tackle that problem. The technology is sold as transformative, but the lived experience right now of the general human population appears to be like, I now do my job plus the AI's job. I mean, it's really not surprising that people's feelings about AI are sort of getting worse. This episode of The Tech Report is sponsored by IG. If you're investing in crypto or diversifying your portfolio, hidden fees are still a problem. Why give away your hard-earned gains? You're doing the research, taking the risk, and then quietly handing a chunk of your profits back to the platform. This is where IG comes in. For investors who are curious about crypto, you can now invest in Bitcoin, Ethereum, and Solana, commission-free. You can buy, sell, and swap across 150 cryptocurrencies cryptocurrencies alongside 12,000 global stocks and ETFs, all from one app. IG, trade, invest, progress. Search IG.com to find out more. IG charges zero commission. Don't invest unless you're prepared to lose all the money you invest. This is a high risk investment and you should not expect to be protected if something goes wrong. Take two minutes to learn more at IG.com slash UK slash crypto. Hi, I'm Isaac. And on the tech report with me today is Elle from House of Elle AI. Thanks for coming on. Thank you for having me, Isaac. Really good to be here. When we met up a couple of weeks ago to talk about doing this video, we were trying to figure out what exactly the topic would be first, first kind of thing like this. And one thing that kept on coming up was the reality of AI sort of versus the illusion that is being solved, as well as the different types of AI that there are. I mean, the technology itself isn't malicious. The corporate resource grab and attempts to sort of sell it as something that it's not to many, it kind of is malicious. And what some describe as an inevitable step towards sort of the future of humanity, for a lot of others, is experience is more of the same old sort of corporate greed being placed above that of people. So what would you say is the biggest sort of misunderstanding people have when it comes to AI? Yeah, I think those are a lot of very good questions. But let's start with the biggest misunderstanding. So from what I've seen so far, because like I work in the field, but I also have a YouTube channel, so I tend to interact with the audience quite a bit. From what I've seen so far, it seems to me that the biggest misunderstanding is just what the word even means in the first place. You know, I think when people say that they're pro-AI or anti-AI, I think that they're collapsing a huge amount of nuance into a sort of like a binary framing that doesn't always work. I mean, sometimes it does, but not really. To me, with a computer science background, artificial intelligence is just a branch of computer science. It's like the equivalent taxonomy level as basically saying cardiology is a branch of medicine or abstract algebra is a branch of mathematics, that sort of thing. And AI is a lot of things as well. You know, it's not just generative AI, you know, the AI that flags fraudulent transactions on your credit card, for example, filtering spam, recommendation systems, that sort of thing. All of that is AI, but I'm not really hearing anyone protesting stuff like that. What is happening, I think, is AI has become colloquially synonymous with generative AI, specifically large language models, chat, GPT, and the likes. And that collapse, in my view, is doing a lot of damage to the conversation because it means that when somebody says, I'm against AI, what they might actually mean is, I'm against unchecked corporate data harvesting or against replacing human workers without evidence that it actually produces better outcomes. or against a chatbot confidently presenting false information as a fact. Like all of those, by the way, are completely legitimate specific concerns. I'm also concerned about those things. But the umbrella terms flattens them into a position that is very easy to dismiss. I, from the bottom of my heart, am very excited about this technology. I'm looking forward to seeing what it can do. But I'm also deeply critical of how it's being deployed right now. And I don't necessarily think that those two positions are contradictory. They seem sort of contradictory only if you accept this pro and anti style framing. But you know what the phrase is, life is but shades of gray, right? in terms of, but, you know, one could also argue that a lot of the framing is sort of driven by the way that marketing is, the way that these companies are sort of marketed. You know, it's, I wouldn't necessarily call it irresponsible because that kind of implies that they deliberately set out to be responsible. I don't want to imply any sort of intention, but I do think that a lot of the time marketing has sort of been like quite imprecise in a way that happens to serve the companies that are doing it, but doesn't necessarily pay that much mind to how the consumer is doing things. These days, every single product launch is like AI powered, regardless of whether the underlying technology is something very sophisticated, like a serious large language model, or just maybe a basic rules engine. I mean, like a calculator, technically, I suppose, is AI. So the term itself is sort of starting to lose a little bit of value. That benefits the companies, of course, because they can generate all of this hype and they can ride it for as long as possible and they have other incentives as well. But the consumer, I'm not exactly sure. I think there was like a paper that I was reading basically that was suggesting that corporate mentions of AI have actually exploded across earnings call and company filings as well. But like if you actually speak with a lot of these businesses, many of them kind of struggle to explain precisely what the technology is doing. Like, yes, your product is AI powered. I love that. Well done. But like, how, you know, like, what is it actually doing? So this is, this is something that I, that I think about quite a bit. I think they're oftentimes they're saying AI more frequently that they're explaining what the product is actually doing. And when I say this, I'm not just having in mind, like the big ones, like Anthropic, OpenAI, Google, I'm thinking about like the general corporate environment, like everyone basically like running the actual hype train. You asked a bunch of questions and I may forgotten some of them please remind me i'm sure we'll get into lots more let's start with this one the what do you think that ai oversellers are missing and maybe what do you think that the the detractors are missing as well i mean it was pitched as a tool which we could use to offload all the busy work too but then instead it's created a whole new type of work that is demoralizing verification of ai slop and then how do you think the adoption of ai has influenced how people feel about it as well. No, that's another bunch of questions as well. No, no, that's totally okay. It makes a lot of sense. I think it's a very complex environment, you know, as a whole. I think to start with the oversellers, I think the big elephant in the room is the fact that the technology is just, in my personal view, generally not really ready for a lot of the things that it's being sold for. I'm not saying that it's not going to get there. As a computer scientist, I'm rooting for it. I really hope that it does. But right now, today, while we're having this conversation, the gap between the marketing and the capability is just currently very significant. There is a survey from Gartner that I read that large organizations piloting or deploying autonomous systems, they were studying that. And they basically found that workforce reduction, so firing people, do not necessarily translate into better ROI. And essentially, stronger conclusions, sorry, one of the conclusions that they came to was basically that stronger results, stronger ROI came from essentially investing in the people and their roles in the operating structures needed to basically govern the technology. So basically helping people utilize it in as fantastic a way as they possibly can. I don't think this is an anti-AI finding, but it is definitely an anti-hype finding. And I think we don't have any shortage of hype. In my personal view, I think the technology works best when it's deployed very thoughtfully alongside human judgment. So augmentation, I'm probably going to keep coming back to this idea of augmenting people. But you know, the phrase like thoughtful augmentation, we're doing thoughtful augmentation, it doesn't really generate the same hype, the same investor excitement as like, this is probably going to replace your entire workforce, you're never going to have to pay for anything ever again. no one wants the first one you know like everyone wants to not pay for anything and not do anything we all just want to chillax and have a martini right in terms of the detractors I think there is a bunch of those people as well I think what they're not admitting is probably the fact that some of this technology is genuinely extraordinary and I'm not just talking protein folding stuff I'm also talking generative AI some of it can be very helpful in the right directions and I mean the ability to process, synthesize vast amounts of information, identifying patterns across data sets that no human being could hold in their head simultaneously. I mean, we live in an ever-growing body of knowledge, both in science and in the world. We have no shortage of data. I pray that we get even more. That is all real. And I think that dismissing it wholesale because of how it's being deployed right now is a bit like throwing the baby out with the bathwater. It's just, we can be more nuanced. We can be better at that. I mean, we can simultaneously be furious about the deployment decisions and still acknowledge that the underlying capability is significant. And I think a lot of people sort of get this, I mean, from what I've observed anyway, they kind of feel like they have to pick a side, like, are you pro or are you anti? And I think that's a failure of the framing of the conversation, not really a failure of the technology itself. and I think your last question was on the verification specifically I think in my view I think this is one of the most corrosive things happening right now I think you know AI was supposed to free people from the tedious work you know like somebody is doing my dishes that sort of thing but instead for a huge number of workers it sort of like created a new brand like a new category of new things that they now have to do so checking whether the AI's output is wrong I mean there are so many examples of this right like teachers are now spending hours extra extra hours trying to figure out whether somebody was cheating with a large language models. Many studies exist with like chat GPT style cheating as well. Developers in the software development community are reviewing AI generated code line by line because it looks plausible, it looks legit, but it might actually contain some subtle bugs there, some subtle errors. There's countless such examples, like we don't need to get into all of them. I don't think that the busy work disappeared, it just kind of like shape-shifted a little bit, it morphed into something arguably a little bit worse because verification itself requires that you know it requires like more expertise in a sense like it requires you to know what the actual answer was and only then can you verify whether this thing that the that the ai outputted is actually legitimate or not i mean if if you have like an excellent expert who is just hired to basically check the output of AI, what was the point of doing it in the first place? You know like they could have just done it anyway There I suppose there some argument there about scaling and that I would entertain that But still a lot of the time we are just generating like a different kind of busy work I think this feeds directly into why AI has also become quite economically divisive as well. I mean, you've got a very small number of companies deploying an enormous sum of capital all over the place, often with questionable returns, as we've seen so often with Anthropic, OpenAI, and the likes, while most people are sadly watching their cost of living continue to climb. I mean, communities in the US are having their power grids strained, their water supplies tapped for data centers they didn't really ask for, they didn't really vote for. I think Uber, I think, was one of the companies that burned through its entire annual budget just in Q1 this year. I mean, we're talking like ridiculous sums of money at this point. And I think this is very prevalent among the corporate world as a whole. We're not just talking these couple of companies that I just mentioned. But at the same time, I think people are just sort of sick and tired of just being made to use Gen AI in the workplace. Like my audience regularly says stuff like they're just doing more work for the same pay and that in their workplaces, they're sort of being like required to use more generative AI. But now they're not only having to do their job, but they're also having to babysit an AI basically into trying to make essentially the same style of output, but also sort of with the underlying narrative that, oh yeah, by the way, you're training this AI who's going to replace you one day. I mean, what is the incentive? Again, the technology is sold as transformative and perhaps it is going to get there and it will be there one day. But the lived experience right now of the general human population appears to be like, I now do my job plus the AI's job. I mean, it's really not surprising that people's feelings about AI are sort of getting worse, not necessarily better. And of course, I don't, I think all feelings are valid. Of course, I really don't think that this kind of resentment is an irrational standpoint to take. I think it's a pretty reasonable response to have to a gap, you know, between what is continually being promised and what is at present, at least being delivered. Do you agree with that? Yeah, there's some words that I hope I never forget from one of the Oracle employees that was laid off earlier this year. And they said when we were told AI would make work easier, we should have known that that meant that it was just going to mean we would do more work in less time, which kind of does make sense. You're not going to be working less than eight hours if they can get you to. I do want to move on. How much do you think the industry relies on the narrative that AI is inevitable in a way to sort of explain to people that it's just not another corporate resource crafter. Kind of is inevitable. It's going to have to happen anyway. You kind of got to move aside. And then would you say that that narrative of inevitability is getting in the way of the technology being treated as what it is, which is a consumer product that is, at the very least, concerningly under-regulated? I love this question. I think it's a super important framing to engage with. And I think the inevitability narrative is probably the culprit of so many unfavorable and suboptimal things that are happening right now. Because when you frame something as inevitable, like it's definitely going to happen, you kind of remove it from the very domain of choice. Like you don't even have to justify the employment decisions. You don't have to justify infrastructure costs. You don't have to justify energy consumption, labor displacement, because it was all going to happen anyway. So like the only question is whether you're just going to get left behind. And I think this kind of framing, because it's not actually inevitable, you know, something happening maybe like a decade later is not necessarily a good reason for making all sorts of reckless decision right now. You know, AI as a field has existed since the 1950s, 1960s. Okay, the technology developing further was always likely, dare I say inevitable, because I'm just a computer scientist. But the specific way that it is being deployed right now, I'm talking the speed, the scale, who benefits right now. None of that is inevitable. Those are choices made by specific companies, specific people in a specific economic environment. You know, these companies, just to be a little bit merciful to them, they operate under enormous commercial pressure from investors, competitors, the cost of building the technology there. It's not even between a rock and a hard place. You know, it's between a rock, a hard place, and maybe a black hole. I'm just trying to be like generous to their interpretation here as well. I mean, it's also important to mention that like OpenAI and Anthropic, for example, they also have very unusual public benefit and mission-oriented governance structures. So it would technically be inaccurate to say that their only formal responsibility is maximizing shareholder returns, as is the case for other companies. But those structures do not magically align every single commercial decision with the wider public interest. You know, the incentives currently can still favor rapid deployment, market share and fundraising over caution, right? Of course, if they do choose to prioritize responsible deployment, great, better plan for all of us. I'd love that. But there is nothing actually requiring them to, you know, and again, the tech is not the villain here. It is the incentive structure in the environment within which this technology is continuously being released into. The system right now, and when I say the system, I'm talking about like the system in which we live and not necessarily like an AI system. That does not currently have adequate guardrails. We're talking about regulation here. And again, this is not the fault of AI. It is a governance failure right now. And I think going back to your inevitability question, this is where calling something inevitable becomes actively dangerous rather than just misleading. Because it's also sort of getting in the way of treating these products as what they legally are. We're talking consumer products, right? If something is inevitable, you don't regulate it. You sort of just kind of like accommodate it. You work around it. But ChatGPT is a product the same way a bottle of Pepsi Max is a product. You know, it has terms of service, a pricing page, a corporate entity behind it. It should be subject to same or similar consumer protection standards as any other product on the market. Maybe the analogy with a Pepsi Max bottle was a bit of a stretch. I get that. But a general purpose chatbot should not necessarily be regulated, of course, like identically to like a pharmaceutical product. But when an LLM is deployed in medicine, in finance, employment, safety critical systems, another high risk setting, whatever, it should meet the reliability and accountability standards appropriate to that domain. Again, we're going to keep coming back to regulation and human augmentation and conversation. Like I generally feel that we need so much more regulation right now. You kind of brought it up on the first question that some of the friction around the AI debate is that artificial intelligence is being used to describe very different technologies and some very useful technologies are being sort of discarded because of the association with LLMs and generative AI and chatbots and that kind of stuff. So how should we really be categorizing these tools, do you think? I love this question. Pardon me, because, you know, as a computer scientist, I do get to see like firsthand actual implementation of these things in domains that are super useful and nobody's complaining about them. Like I said, AI is like an umbrella term, right? Like under it, you've got tools doing completely different things. I'm talking numerical analysis, time series forecasting, pattern recognition, alpha fold that I mentioned earlier, solving protein structures, radiology scans, catching things human eyes can miss. In finance, for example, fraud detection and risk modeling have become so embedded that people forget it's actually AI at all. You know, nobody's protesting any of that. I haven't seen any protests about it anyway. And they shouldn't be, you know, like because it is useful technology deployed again, back to deployment in ways that makes human expertise better at what it already does, you know, and probably skills better as well. The friction that I'm recognizing that I'm observing in my community and on my channel as well, and also in the experts that I speak with is entirely about generative AI. And there are so many issues with that as well. The problem is that the backlash against chatbots and image generators is catching all of those other tools in the crossfire as well. I think that if people as a whole categorize more precisely distinguishing analytical AI from generative AI, from robotics, from recommendation systems, because these are all like tiny little like subfields of the whole thing, then the conversation, in my view, would become a little bit more productive overnight. You know, there is in one recent video that I mentioned that the most sure way to lose a battle is to not know who your enemy is. Like you have to be able to identify what exactly you're mad at. You have to identify what exactly is generating this problem irrespective of feelings. Like what is what is the causal effect? Like what exactly is going on? Because you have to be specific so that you can actually tackle that problem. You know, instead of gesturing a broad field that has existed for the last like 80 years, we need to do better than that. I think there's already fantastic taxonomies for the separation of these products and technologies. But I would personally advocate for just greater tech literacy among all of the humans. You know, I'm usually not advocating for stuff like that. But now that AI has become such a prevalent piece of everyone's life, I think it's time that we can only fight fire with knowledge, at least on this occasion, I think. What do you make of using the word intelligence to describe things like generative AI? Do you think it might be fundamentally misleading the public about what these softwares, what these technologies are, what they can do, what they're actually capable of? I love this question so much. I get this a lot in my comments as well, in my videos as well. They always say stuff like, not always, but like frequently enough, they say stuff like, stop calling it artificial intelligence. It's not really intelligent. Okay, fair enough. But like, what is intelligence? You know, because I don't think humanity has settled consensus on the definition of that specific word. Like my phone, for example, if you shine a super bright light on it, it's going to increase the brightness on the screen. Like, is that intelligence? I mean, it's responsive. It's adaptive. Sounds like a form of intelligence to me. Large language models are passing the Turing test, which for decades was considered the holy grail of computer science. For those who are unfamiliar with the Turing test, we're talking about essentially a situation where a computer, sorry, a person is sort of trying to figure out whether they're talking to a computer or they're talking to a human. And in most of the cases, the computer convinces the human that they're indeed passing as an actual human. So this is the level of technology that we have right now. I think LLMs, in both generative AI and also like AI in general, they can perform tasks that were explicitly designed to require intelligence. Like that's the whole reason why we made them in the first place. So are they intelligent? Absolutely. In a lot of measurable ways, 100% yes. what they don't appear to be and what I think the people in my comments are often hinting at is they don't appear to be sentient and I think that's where the confusion lives people hear artificial intelligence as a term and they think that it implies something that not only thinks but has awareness and experiences and even its own thought processes are very akin to ours to some small extent maybe it does have those things but definitely not fully like humans do Let not get into the whole like anthropomorphization of models and all of that But intelligence and sentience are not at all the same thing And I think the bigger misleading element isn't the word intelligence necessarily. It's just collapsing the whole field into like a specific product category. So they are intelligent, for sure. I'm Shannon Maldonado, the founder of Yowie, a cadeau with a focus on art and handmade objects. From all platforms that I tested, I found Shopify absolutely the most useful. I found it important to think about where we are in the future. All tools to look at your sales numbers, like the planning, you just find on your dashboard. Start your free trial on Shopify.com. more specific, I think it's fine. It's just how language works. You know, that the shorthand isn't really the problem. The problem is when the shorthand replaces understanding, then we're arriving at something more specific. I would like to see better general technological education so that people understand not just what the tools are or how to call something, but they understand where the capabilities start and where the capabilities stop. You know, what are the advantages and disadvantages? Where are the weaknesses? Where are the strengths? What can you actually trust as an output? I think you can use the shorthand, you can say whatever you like, I mean, within, you know, there are some exceptions to that. But right now, for most people, this understanding is like a little bit lacking. And I think this is what we should be promoting more and more, we should be providing this kind of education as in as an accessible way as what we can possibly manage, which is what I try to do on my channel as well. Because understanding is exactly, you know, not, oh, sorry, not understanding is exactly where the hype lives. This is, this is the breeding ground for all of these kinds of like hyped up marketing strategies where people, you know, don't, are not necessarily equipped with the ability to be, with the knowledge to basically say something like, really though, like, what does that even mean? Like, I know this and this is not really checking out. That's really, that's really what I'm getting at. I suppose whether you think generative AI is a powerful tool or a waste of energy or you're somewhere in between. I think one very harmful and obvious impact that we're seeing at the moment is on our brains, whether that's AI psychosis or cognitive offloading, potentially eroding people's intelligence and skills and that kind of stuff. Do you think it's too early for this technology to be so widely used? Okay, this is one of my favorite questions. I don't think that this is talked about nowhere near enough as it should be talked about because i think we're having like another social media 2.0 crisis i use that term loosely the honest answer like as a scientist i'm just always going to go back to refer to the to the literature uh the honest answer right now is that the research is inconclusive and i think anyone who tells you definitively that ai for sure makes you smarter or dumber they're probably overstating what we as a human race right now know definitively I did a very deep dive into the literature for a video on this. And what I found is studies that basically point in both directions, depending on how the technology is used. So this is the most important part. You know, on one hand, you basically have very early studies showing reduced cognitive engagement and poor recall in particular conditions. To put it in the simplest terms, it makes you dumber. There is an MIT study basically where participants wrote essays. They were just writing essays. That was a test. And they basically found that people who use chat GPT show the weakest EEG connectivity of the group study. that basically i mean the study is essentially like they put like a bunch of things on your skull i forget what they're called and they're essentially measuring the how the different parts in your brain are essentially communicating to each other so it's not necessarily a you're smart or dumb sort of thing it's more about is your brain communicating with the different parts within it like in a in an effective way and essentially they found that people who were using generative ai try gpt specifically had substantially more difficulty as well recalling quoting their own work. So like they did it with chat GPT and then they couldn't really like quote what they had been writing about. And there is another study done by Carnegie Mellon and Microsoft. It was like a survey predominantly of knowledge workers. And they found that the more they trusted generative AI, the less they thought critically, which of course led to a reduction. There is many more studies that are not really coming to mind right now, but you can check them out in the video if you want. But importantly, all of these studies have their own limitations. We're talking small samples, less than 500 people, which is not a very big study at all. Self-reported measures of people telling what they felt they may have recorded, which is fine, but it also carries a subjective element to it. These are also preprints, which have not necessarily been replicated yet. In some cases, they haven't yet been peer-reviewed. All of these are limitations to their studies. That's not to say that I'm dismissing them in any way. I think they're incredibly valuable. but as a scientist i have to call for a lot more in this direction still the directional signal of what they're giving us is deeply concerning still i wouldn't call it settled science yet then we have the other side if i didn't bore you with the first side um we actually have like a research basically showing that when ai is used in a socratic fashion so we're talking when it's designed to push back on you ask you questions rather than just like hand you answers like here's a photo of exactly what you asked for, then learning outcomes actually improve. And there are studies supporting this also in like small children as well. Like there was one study basically found that students who critically engage with AI performed on average better than those who didn't use it at all. I mean, so in some cases it can literally like help you think better, right? There is another one as well that I read just a couple of months ago by L.F. Theriot and colleagues. I'm really sorry if I butcher the pronunciation of that. I can apologize to the authors. they basically found that standard use of chat GPT produced what they called confidence without necessarily a lot of competence. It was a very catchy phrase in their paper. But when the AI was redesigned with pedagogical friction, so like what I discussed earlier with the Socratic style of learning, then learning gains were actually very significant, statistically significant as well. So the tool itself is not inherently good or bad for your brain. It depends entirely on how it's designed, how we use it. Right now, most implementations, of course, are optimized for convenience, for hype, not necessarily for making you think harder or better, which again, leads to regulation. But I feel like I'm preaching to the choir at this point on the psychosis point. Yeah, psychosis is a very complicated field and I would be careful about speaking too definitively about this. I'm not a psychologist. So I have read a couple of papers though. I think the mechanisms are definitely real. People are, of course, forming attachments to systems designed to be responsive and agreeable, you know? And for certain people, unfortunately, with certain vulnerabilities, that can become genuinely harmful sometimes. But again, as a scientist, I want to see a lot more clinical research before making a sweeping claim about this. I think it would be inaccurate to say something like, Chad GPT will in 100% of the cases give you a psychosis. I don't think that's the case, but also in some vulnerable groups, it definitely could be. What I will say is something that I hinted at at the very beginning and basically just highlight that we literally have spent a decade learning the very hard way what unregulated access to social media does to developing minds. I'm talking predominantly small children, right? like it would be very good, please hear me guys, not to repeat that same mistake with something that captures thinking itself, you know, not just attention. I think thinking is a part of like, you know, what makes us human beings, you know, we cannot afford to lose something as precious as that. I think a very acute area that at least I see where these sorts of problems collide is with AI overviews. Oh, yeah. Yeah. Whatever Google says, it kind of actively redirects people away from human sources of information towards AI sources of information. That's true. Generative AI has the potential to bring people closer to the information that they need faster than it ever has before, but instead the authority with which it's being presented is, I would argue, misleading and potentially intentionally misleading. What do you think of this kind of use of LLMs? Yeah. Yeah. You know, this, everything that you said just like right now reminded me that, you know, about a month ago, there was a German court that ruled that Google can be held liable for false statements generated by AIO reviews. I think that ruling actually gets to the core of the problem, which is probably more the authority gap than anything else. Like when you type something into Google and a beautiful, perfect blue link comes up, you sort of understand implicitly that Google is pointing you in a specific direction. Like out of the vast nothingness or rather the vast darkness of the internet, Google is basically saying you can go in these directions, like take your pick. So you click through, you read the source, you evaluate it yourself. Google is essentially like a database, like a directory, right? But AI overviews fundamentally change that relationship. Now Google is giving you the answer directly, presented in a beautiful, clean, neat little paragraph at the top of the page with the confidence and formatting of a literal factual statement. This is a very important piece here. Like most people do not scroll past it. And honestly, why should they? It looks authoritative. It looks legit. It looks like it's been verified. A lot of the times, actually ever, it hasn't. And to be fair, most of the time it is accurate as well. Like I'm also definitely guilty of just reading something there and just be like, done. I don't have to do anything else. The problem, however, is that Google has insane volumes, which is a fantastic thing. But also when we're talking about this kind of thing, even tiny, tiny error percentages end up being significant misdirections of people, especially when we're talking about very critical queries that people engage with. The tech behind it is called retrieval augmented generation. We don't need to go into like the specifics of it, but just for like some general basic understanding, also RAG. And I'm not trashing RAG, by the way, but I think it's important to, I think it's a fantastic piece of methodology. and I think we should all keep doing it more, but of course it's very important to understand the pros and cons of everything. I think it basically has like three distinct failure modes, right? Like it can retrieve the wrong source. That is one specific wrong thing that can go, one thing that can go wrong. It can retrieve the right source and synthesize it incorrectly. That's the second one. And the third one is it can present the result basically with a level of confidence that bears no relationship to how certain the system actually is. And that third one is the most dangerous because it's the one users have no way of detecting. Like a confidently wrong answer looks identical to a confidently wrong answer. Like if Google basically told you the sky's green, okay, that one is like pretty too, just way too obvious. Like you're probably going to know. But if we're talking about something like less obvious, like a specific company earnings last quarter were X and they were in fact like X plus 10 or whatever you wouldn necessarily know that as confidently So basically this is what I talking about You also talked about like misuse of large language models. Maybe. I think it's more likely that it's mostly a business model problem. You know, like Google's entire revenue model depends predominantly on keeping people on Google. That's how they make money, right? Human sources of information, we're talking blogs, journalists, researchers, those entities live elsewhere and their data lives elsewhere as well. I'm talking like different parts of the internet. They're not on Google specifically. Every single time somebody clicks through to a different source, Google loses a pair of eyeballs. That's just how it works, right? AI overviews, however, solve that problem partially for Google by making the click through sort of unnecessary. Like you got the information, you don't have to go anywhere further. Just stay here. We can serve you another ad. Perfect. And whatever Google says publicly about supporting the open web, and I'm not saying they're not supporting the open web, by the way, they probably do, but also multiple things can be true. The product design, I think, tells us a little bit about where their priorities actually are. You know that phrase, all roads later roam? Is it? Yeah, I think in tech, all roasts lead to the concept of incentives basically like everything ties back to the same thing like who wants what and why we've talked a little bit we've kind of danced around it a little bit as well the the what exactly regulations might look like and i mean is anywhere ready to legally handle this because i mean even if we look at what happened last week with open ai they admitted to felony of accidentally hacking Hugging Face last week. Yeah. So first and foremost, I think I would personally be a little bit careful about taking the whole Hugging Face story at face value because OpenAI kind of has form with us. When they released GPT-2 back in 2019, they, if you recall, made a super big show of saying that it was too dangerous to release fully. And I think while everyone hears this is dangerous, oh my God, Terminator, I think investors hear this is powerful. You know, we should inject more money into this. This is serious technology. Maybe it misfired now, but look at all of the capability that it has. Announcing that your model accidentally hacked another platform is maybe an ad dressed up as a bit of a confession. I mean, no press is bad press, right? Of course, I'm not claiming that I have tapped into the collective mind of OpenAI and Sam Altman. I have no idea what they're actually doing, but this is where my mind goes on the specific question. So I would personally want to separate the genuine regulatory question, which I want to address from like the theater shenanigans around it. On the actual regulatory readiness, I can definitively say no, I do not think we're anywhere near ready on this one. And I think the problem is a lot more fundamental than most people realize on this one. On the regulatory readiness side, no, I definitely don't think that we are nowhere near ready, basically. I think that the problem is so much more fundamental than most people realize as well. So it's not just a matter of like making a couple of laws and just figuring out consumer protection laws that we discussed earlier. I think it needs to be so much better than what it currently is. But better guardrails for models are definitely necessary, but also like not sufficient, nowhere near sufficient. Regulation needs to address model training as well, not just deployment. How the models are trained, what data they're being trained on, what behaviors are being optimized for? What kind of benchmarks are we going to have? What kind of testing environments are we going to be allowing? All of these things are super relevant. And even then, you know, let's say that we, you know, definitely 100% want to go in that direction. We run still into generally hard philosophical problems, you know? Somebody in my comments very recently when I was discussing the whole GPT 5.6 Sol problem, all of the problems that were coming up with that, they basically asked, can they just tell the model not to lie? And I thought, what a fantastic question. Definitely that this can be set to the model. But in reality, what is a lie? You know, like what counts as truth for a system that doesn't really have any beliefs? You know, like teaching a model, a moral framework requires us all to first degree collectively as humanity, what this moral framework is. And to my knowledge, humans have not really like agreed on a consensus on this for the better part of the last couple of millennia. So when I hear people say regulation is lagging 100%, I completely agree with you, my friend. But also, I think we should be honest about why. This is not necessarily a case about regulators just being slow, regulators just being lazy. Some of these problems are genuinely hard and difficult problems. And pretending that they kind of have quick technical fixes is a different flavor of hype. 100%, I'm not saying that we should wait until we settle this millennia-long debate. Regulators can still set concrete requirements around testing, security, disclosure, discrimination, consumer protection, human accountability, all of these things. This can happen even while the deeper questions remain unresolved. So back to your original question, we 100% need so much more regulation. We just need to be mindful about what is feasible, what is possible, and how are we going to arrive there? There's an old IBM manual from, I think it was 1979, it said a computer must never make a management decision because it can never be held accountable. I would personally argue whoever sort of sets the command and the company that designs the model should and selects the data set and that kind of stuff that you're just saying should be the ones that are jointly or different, fortunately, but jointly responsible for the negative consequences of what happens. but where does that actually sit and where do you think it should sit especially if we consider the fact that uh people can be well maybe not people can end up thinking that ai has the ability to judge things to say if something is right or wrong to go back to your point about the the moral dilemma i love these questions they're so loaded so first of all i like the ibm quote very much because it capture is something that's kind of been true since the 1960s and we're still in part ignoring it you know a computer cannot be held accountable the same way a pen cannot be held accountable like it doesn't have legal personhood it doesn't face consequences it doesn't go to prison and fundamentally this is not a unique thing to AI humans don't have legal frameworks for other species or other entities of course if we're being pedantic the law does recognize non-human legal entities like firms, corporations, but AI models do not currently have that kind of status. Laws govern human behavior. That's it, right? So even if a model does something catastrophic, and we've seen that it can do that, there is no framework for holding that model accountable. And honestly, I am not even sure that there should be one. Where responsibility should 100% sit is with the humans in the chain, but not necessarily in equal proportions. Again, And we're talking about a chain because this is a sequence of how we even got to the situation where a specific model does something unfavorable to us. The company that trained the model, there is also the company that deployed it, not necessarily the same company, by the way. And yes, of course, the person who gave it the command without understanding the limitations, all of these people and teams, usually teams, are accountable. And by the way, the model's limitations are usually publicly documented stuff. This is not something that companies usually hide. every major AI company has them on their website. If you hand an agentic system, the keys to your production database without reading the safety documentation, some of it is on you, my friend. Like I'm sure all of these people don't go into the situation waiting. You know, they probably go into it with a lot of hope. Look at this model. Everything is great. They probably don't expect that the worst is going to happen. But I mean, come on, hope is not a strategy. We have to, you know, mitigate for a variety of risks as well. But also not to put all of the blame on the person running the command, the companies definitely have a responsibility not to ship products that they know are seriously dangerous. You know, as we discussed with the GPT 5.6 Soul saga, that line sometimes can be a little bit cross. But I think I always come back to the same thing, and I'm going to keep talking about this forever probably until it actually happens. We have a unique opportunity as a human race for augmentation. Neither humans nor AI, in my personal view are better off alone you know ai in its current form cannot and should not replace human judgment i will address your judgment question in more detail in a second just want to say that like humans cannot do a lot of things that ai can do like i wish that i could parallel process in my brain but alas here we are the most productive path i think is like a partnership between where human essentially a partnership between like humans and machine where the human can remain responsible and accountable, remain in the loop throughout the whole process. That, of course, requires so much better regulation than what we're having right now. So much better training, release, rules around all of that, of course. But it also requires all of us to stop framing this whole debate as like humans versus machine and start treating it as human and machine, you know, where the human, of course, never actually lets go of the steering wheel. You mentioned also a question about judgment specifically, and you tied it back to the intelligence point specifically. Usually, whenever there is any sort of a machine learning model, large language model included, when these things are tested and their outputs are evaluated properly, there is, of course, a statistical methodology included. And here we can have like confidence intervals. We can say with this kind of a certainty, we can make these kinds of claims. We're 80% sure about this. And the model can be trained to output these kinds of confidence intervals, confidence bounds can tell you like, I'm X percent sure about this kind of claim. This is usually embedded within the whole process. And when I say AI here, I'm not just talking large language models, just AI as a whole field. This is everyone wants to, every computer scientist needs to know, like, I built this beautiful shiny thing, but like, can I trust this judgment? Trust is like the core of the whole thing, right? So I think that while we can use rigorous statistical methods here to basically tell us it is sure with X percent probability, fantastic. But at the end of the day, am I as a human being going to trust this probability? am I going to still decide to take the shot because if somebody tells you there is an 80% chance that this is going to happen that means that there is a 20% probability that that thing is not going to happen and I as a human need to figure out okay if I do this off the back of this model's calculation I still need to prepare I need to be prepared for the 20% probability that it's just not going to go my way you know and this is where the human judgment comes in you know this is this is the whole like holy grail of how we make decisions and why we cannot just disconnect because all of these companies keep repeating stuff like, we just want to replace the whole workforce. No way, humans are not going to be needed. You're never going to have to pay for anything. Fair enough. But also, who's going to be responsible for the whole thing? What's going to happen when somebody needs to actually take a call? That's where I stand on that one. Well, Elle, thanks for taking the time. Thank you so much, Isaac, for having me. This has been great. If you enjoyed today's episode and you want to hear more of the tech report, please consider liking and subscribing. Also, you can get episodes of The Tech Report wherever you get your podcasts.