We're never wanting to automate away the instructor. We want to automate using AI some of the workflows that take a lot of instructor time. There's a difference between wisdom and artificial intelligence. Human wisdom is irreplaceable. Artificial intelligence is also irreplaceable. It's this partnership where we don't automate away the human because the wisdom comes from their expertise. We've got to combine that, the human touch, with the automation that AI brings. Welcome to the Expert Intelligence Podcast. I'm your host, Paul Estes. Today, we're joined by Peter Riehle, founder of the UWeb incubator, one of the initial investors in Discord, and a tech industry veteran. His mission is deeply rooted in his personal story of growing up in poverty, with two parents who believe that education would transform their children's lives. And it did. Peter is working to pay it forward with one of his latest projects, working to give students around the clock coaching for those make or break math courses in their career. Peter, welcome to Expert Intelligence. Thank you, Paul. Really a pleasure to be here. And we've known each other a long time, so I'm looking forward to this. Well, I want to start with where we first met. I was interested in the gig economy. I'll never forget the morning that I woke up and was reading Product Hunt. And I saw, it wasn't even AI, it was God at the time. It was a unique approach. Like I hadn't seen it before, I'd been in tech, and I reached out and I remember flying down to meet you and the team. And it was a tutoring service where you were connecting people in real time with tutors and education. What did you see then? Because I think it's really important to understand sort of a thread in your thinking. What did you see then that we're going to talk about later with MathGPT? So yes, mathGPT.ai next topic, but let's go back, rewind, and look at why we got here. So when we started on the project of what we call conversational microtutoring, the idea was that the iPhone had taken off by the mid, you know, 2010s for sure. And, you know, well before that. And people were text messaging and SMSing. And we felt it's really important to engage in not just Uber-like on-demand services and DoorDash and so on, which are very much part of life today, but try to give learners an opportunity to have a conversation just the way they would with a friend on demand, just like with a good friend, they would be able to just get them or reach them. But in the learning space, and the goal was, and it still is, that human wisdom combined with artificial intelligence. Now, at that time, it was human wisdom first, we call them experts, and that's why expert intelligence is what you call what you do. Assisted by AI, with the goal that we would, at one point, flip it, where it would be artificial intelligence assisted by human experts. So, we've come today to mathgpt.ai, which is exactly what I just said. You know, we started out with humans as the primary sort of resource you could get to with AI in the background helping them to mathgpt.ai, where you are interacting with an AI first in learning and education, starting with math. And there's a human, it's an instructor-led platform, so there's a human in the background who has overseen and set up this whole interaction that you as a learner are gonna have with this AI. So, to me, it's just the question of when we flipped it over, waiting for this moment, but starting with a kind of a, you know, who's primary and who's the backup. I had the opportunity to hear you talk and really learned about your passion, not only around math, and I wanna hear your story about being a math tutor, but why math? Why is math such an important subject and what problems specifically are you trying to solve with the technology that's acute that you see? So, broadly speaking, and we'll get to how I even discovered math, but broadly speaking, what I feel is a mass scale, and that somebody at Wellesley College has written a nice article. We have a bit of math in numeracy. We call something illiteracy when we can't read or write, and there's this new term, innumeracy, when we can't figure out lots and lots of things which have some very basic math behind them. So, that's the first broad macro trend. If you look at COVID and pandemics, if you look at results of vaccines, if you look at demographic data, if you look at GDP data, if you look at tariffs, which as we all know, there was this equation that came up that people were like, what is this equation, right? Life, too much of life is governed and we need math, literacy or numeracy, a comfort with all these things going on in our world that we may not have because too many people believe they're not good at math, right? So, that's a fundamental problem. In particular, there is this notion of pathways and gateways when you get to the first year of college or even dual enrollment, where what I found, and this goes back to my math tutor days at UCLA when I was just a 19-year-old kid, but I was tutoring other 19-year-old kids, and I discovered that a lot of people were giving up on majors that could have made their life and careers a lot better or different anyway because of a midterm or a quiz that they just didn't feel they could get through. And so, you do find that math is extremely important in majors and careers when you first get to your college math courses. If you want to be a nurse, you want to be a sports journalist. Do you know that sports journalism? You wouldn't think it would be blocked by, well, you got to take statistics to be a sports journalist, right? You want to be an electrical technician, you've got to take some math. So, it turns out it's a real gateway blocker and the correlation between not passing math and not causing effect but correlation and not getting through college in six years is very, very high. You're right, every single career now is computer-based and it's kind of like the argument of when people are introducing AI and they think it's a computer science course, when actually AI is a general-purpose technology. And I think the framing of math is the same. There's probably not a job out there today that some enumeracy is not needed and not having it is detrimental. When you think of bringing this technology into schools, so you've got this math product, it can help any student at any time, yet needs coaching. And I think one of the things that when you were talking, resonated with me is I was fortunate to have tutors and I actually had a math tutor growing up. You had to schedule it, we had to drive there, we had to sit down, we had to do those things. And more and more college kids are studying the middle of the night or in the morning and might need on-demand expertise on-demand coaching. When you go to bring this technology into schools and it's AI-based, there's kind of a resistance that we're all seeing that says, hey, I get the AI, but today AI in a lot of regards is responsible for systemic cheating inside the university. Yes, yes. What is the framework or the approach that you have that helps address that? And it helps not only address that, but educate people and administrators and faculty that, hey, there is an approach that is additive and it's not cheating. It'll lead to a numeracy. Higher numeracy, because you have real learning. Yeah, that's a great point. So today, madgipt.ai is being used, and I always say madgipt.ai because there's a lot of madgipt websites out there which are just solvers. And that's the ones that are allowing people to cheat. But what we do is we instructor led. So we're using hundreds of classrooms today, instructor led in community colleges mostly and state colleges. And why? Okay, the fundamental tension we discovered was between the propensity of a student to just want to get through the class. And therefore, if a convenient tool is available to help them, just give them the solution and they can type it in, that's what a student's gonna do. It's just life. Let's accept that. On the other hand, the instructors are worried that if they do bring AI into the classroom, they cannot trust it for accuracy. As you know, there's been a lot of discussion of hallucinations and the lack of guardrails and trust and so on. So this tension has existed with one caveat, which is the students continue to increase their use. Which if we didn't address by saying, hey, what if instructors, we gave you a guardrailed, trusted, accurate product, not just a raw chat GPT that you can give to your students with some discount, but an actual product built around these language models that added guardrails that checked everything for accuracy that was cheat proof. So we often show them a demo where in the chatbot, MagiPD is a product, it's got a great UI and then there's a side pane where you can chat with the chatbot and you ask it, give it a problem, just give me the answer. It says, I'm not here to give you the answer. I'm here to help you understand. Do you know how to do the first step? It's called Socratic Tutoring. So once we showed instructors the power of an application built on top of AI that gives them the tool and the product they want versus the raw technology, which is very powerful, we know, but has hallucination issues and guardrail issues, then they were more comfortable bringing it to their students, right? Because now they feel I can take charge of AI in my classroom versus just kind of trying to figure out who cheated and who didn't. And then the student still gets a tool that is very close to what they would have had if they just went to chat with one exception. With the chat GPT or other tools that are language models alone, a student is not a good judge of whether the solution or whatever the thing gives you is correct. They just don't know enough to make that judgment call. So they don't know. So that's why they prefer MagiPD.ai because we guarantee it's accurate and we've had bug bounties for years on this. So all the pilots we did were well over a year ago. So we have guaranteed the accuracy for certain subjects that we support. And second, we don't allow cut and paste into MagiPD, which is where their assignments and practice problems come in. So even if you were using a tool externally, you can't just copy that over. So you'd have to go through a fair amount of effort to use an external tool that gave you the answer and bring it in. You may as well just work with this chat bot, which kind of helps you understand and then you're finished. So we really brought the student and the instructor together as opposed to this divide that existed with this trust on one issue and the propensity to just want answers on the other side. One of the big trends that you made me think about is personalized learning. We all approach a topic like math or science from a certain starting point. What are instructors able to get from the way I learned? So I'm using the product. Is there any feedback loop back to the instructor that helps them understand students better? Because they're on the front line. It's not just this product. They've got a bunch of other coursework and projects and in-classroom learning. What are the instructors able to get out of this? Yeah, this is great. So we are not just a student chat bot tutor. It's a complete solution. It really has three parts. Magip.ai has a course manager. You bring in your course, your materials, your PDFs, your lecture notes, your slides, your textbook. We allow you to bring all of those in and get your course organized into the sequence in which you want to teach. Now that's expected because how else are we going to know what to tutor? So that would be what we call table stakes. But it is an instructor co-pilot kind of experience where they get to actually set up the course and say, here's the best part then. Once that's set up, and it takes about 30 minutes for them to upload their PDFs and this and that, they hit a button and it will generate the entire set of assignments, quizzes, and practice problems for the course, for the entire course. You just choose the chapters you want and what sequence and it says, here's, and these assignments, quizzes, midterms, practice problems are all auto gradable. That means all they have to do is set the due dates and grades that they would give to each assignment, et cetera, the number of marks. And suddenly they have a full teaching assistant now who will dish out the assignments, a tutor sitting on the other side of the student who will help them with all the different things. They can control how much tutoring is given in the quiz or midterm, no tutoring, you have to take it. In homework assignments, maybe some difficult problems, tutorable, practice problems, all tutorable. These are some of the operational practices we're seeing. And now all the grading occurs. And what they get is a dashboard with the analytics for the entire classroom in real time as the class makes progress. That analytics, because you're working inside a real time system, is just invaluable. They can see, typically what they see is three levels of a class above average, average, and below average. And they can quickly group those students and even drill down to a single student to see, how did they do a lot of practice problems? Oh, wow, they did, and they stood there right. Let's give them this particular student five more. Or they can have remediation materials and say, oh, you're doing a calculus course. You don't look like you got the pre-calculus, trig stuff figured out. And then they can add those PDFs and textbooks and recommend certain practice problems. It's incredible if you think about that level of ability to personalize learning. And that's just on the instructor side. I can talk about the student side in a bit. One of the interesting things when you were doing, got it, the tutoring, you were out talking to schools. I mean, you've been at this. This isn't like you woke up one day and said, hey, I got this thing called AI. Here's my latest AI app. And one of the reasons I've inspired by you and just love the work is because everything starts with human expertise. And then you figure out how technology can help. But tell me one story from mathgbt.ai on the ground where there may be a student or a teacher. There's some anecdotal story. Yeah, I bring a couple to mind. So the first pilot we did was with a statistics professor, adjunct instructor at De Anza College right here in the Bay Area. It's one of the biggest colleges we have in California. California is the biggest community college system in the country, two million students. A lot of them, two year degrees, a lot of them go to transfer to the UCs and state schools. So we're talking to statistics instructor. Here's an incredible story. These people, we hear about, as you know, what's in the news today, large universities, research universities, Harvard, Penn, all the big ones are getting their budgets cut. Why? Federal research budgets. Those are research universities, right? There are researchers there who teach. Here in this community college, it's a teacher. And we said, okay, here, look, we have to pay the costs of this product. He just loved the product. He said, this is what I've been waiting for. This is something where I have to answer emails at 9 p.m. or 10 p.m. or 11 p.m. Why? I said, but don't they have a tutoring center? They do, but it closes at six, because the tutoring center is other students and people who are hired to come in and do their job. And what we found is most people are doing their assignments at eight, nine, 10. And so they send emails or they don't do them. And then they come to my office hour two days later, because I can't hold office hours every day. So we saw this instructor just like fell in love with this tool. He's now onto his sixth quarter of using that GPT.AI nonstop. But here's the interesting part. So when he used it, I said, look, it's $25 per student per course, right? And I said, we just have to pay the bills. We've got these very expensive language models running in the background that we have to pay for. And so compared to most of the other Pearson or McGraw Hill tools that they usually buy for their students, which are $50 to $100 per student, this is a very good price. And you know what he told me? It's a beautiful story. He said, I care about these students so much, I'm going to pay for it. So his class of 40 students, he says, I'll pay the $1,000. And what we did then is we just waived it. We said, we can't let you pay the $1,000. He said, it doesn't matter. These students are going to benefit so much from this. When I do printouts, I pay for the copying in the paper. This is well worth it. So it was a beautiful example of technology meeting human who cares and is the expert on the topic and is doing the work with MagiPT because it's not like the students would just adopt it, right? He's the human intelligence who took MagiPT, applied it to his classroom. I want to talk about broader just AI and education. I was fortunate last week I got to attend a hackathon for Education Trust in the UK. And it was eye-opening to me how early in the adoption or AI literacy journey, most of the secondary schools, primary and secondary schools are, especially in the wake of what China just announced, saying, hey, every student starting today from first grade all the way through college will have at least eight hours a year, which doesn't sound like a lot, but it tops down all students. Oh yeah. How do you think about helping educators, faculty and administrators, how do you think of approaching them to start their journey of AI literacy so that they can better have experiences like the statistician that you spoke about? So there's a whole field in education as you know called PD, professional development, whether it's teacher development or professors or instructors, right? And one of the things that I am sponsoring this year, in fact, more on the nonprofit side is the notion of a nationwide tour that I will personally be conducting with a couple of colleagues who are in the NGO, where we will be talking about AI literacy and how AI can help instructors, why? And one is because we want to educate the instructors on how to use these kinds of tools, not just magic, but generally speaking, AI and what it can do for you. But here's the really important part. What I find is that if we were to call people about AI literacy and the use of AI, et cetera, the students are actually further ahead. They're actually aware of prompts and by the time they're a little more grown up, but they're not as well, I would say, educated on the more what I would call productive uses of AI. You've heard of character.ai, where these people are coming in, and they're playing with AI. You've heard of all the chat GPT weird messages and where they try to hack through. So there's a lot of pop culture that is building up around AI that I think students are very familiar with. How it actually helps learning is a whole topic and the goal is to actually get instructors to A, understand how it works and not just math across the board and B, get them comfortable with the idea of how to deploy it and how to co-opt the student. Because if you look at calculators, a lot of people give the example of calculators, right? We succeeded in both sides getting comfortable with that technology, right? So one of the things we do is we do a lot of hacking testing. So the idea is if the student does get their hands on it, I'm at GPT.ai, and the instructor is giving it, the student is gonna try to hack it. This is just nature, somebody's gonna try, right? Not saying every student. So we want that. We actually offer a gift card if you can hack it and try to make it say something that it shouldn't buy. It co-ops the student. So I am a big believer in AI literacy for all students, not just college students across the board, and I'm a big believer in a joint understanding, not just the instructor's understanding and then the student's colloquial understanding or cultural understanding, but a joint understanding. I think YouTube was very good that way, right? Both sides co-opted it. I think calculator is very good. We've gotta make AI that way, and that's gonna be a mission for the next, you know, many months, two years of bringing a joint understanding of AI in education. I think the thing that is interesting to me in your approach is that it comes from a deep experience in the topic. You sort of got to this place from a lot of education, the math focus and tutoring. It feels like a... It's in my DNA. It's in your DNA. It's in my DNA. You're not a Silicon Valley tech executive coming and sharing some shiny object. It comes from a very deep understanding of not only the problem, but the earned wisdom and things that it took to kind of get to this point, which is exciting in not only your story, but in product as it sits today. And especially at this time, I think I was with a bunch of faculty and we were talking, in fact, even this morning, I talked to the National Institute in the UK, a bunch of teachers on their professional development. It was a professional development session. We had a conversation and they were asking, help me understand where technology isn't cheating. I mean, they were open to the idea of wanting to learn more and really embracing that curiosity, which was really exciting. Which is great, yeah. And another point I wanna make on that, a lot of people do ask me, so everybody's talking about automation and the way of all the jobs, right? And we won't have any need for anybody, robots and AIs and everything. This is where your topic always intrigued me, expert intelligence, human intelligence. I always say, we're never wanting to automate away the instructor. We want to automate using AI some of the workflows that take a lot of instructor time, setting up the assignments, grading the assignments, trying to understand the classroom and collect all the data and do the surveys and what we're gonna do all of that through AI. There's a difference between wisdom and artificial intelligence. Human wisdom of a teacher, like this person I was describing in the statistics class, is irreplaceable, all right? Artificial intelligence is also irreplaceable for doing some of that stuff. I'm looking into the future. It's this partnership where we don't automate away the human because the wisdom comes from their expertise. As you were describing mine, I was thinking of, this gentleman who did our first pilot statistics, he's like, my age, well, 30, 40 years of experience doing this stuff. The ability to read a student's thoughts and mind around when they are in a classroom or when they are struggling with something is very different when a human being can read that and can draw on a vast amount of experience having taught millions of students, right? And have an intuition, it's in their DNA, like we were saying. We've got to combine that, the human touch with the automation that AI brings. So the vision is never to automate away the instructor. You may find that the instructor can run out three times as many classes, right? So you'd say, well, they're not really running the class, but the reality is like, why do we call it a co-pilot? Well, who's the pilot? Right? The word co-pilot has taken over, why? Because you were at Microsoft, right? It's like the biggest product out there right now being used by, I don't know how many people. Cause there's a pilot. And the idea of a co-pilot is you don't automate away the instructor. And there's a big trend around coaches, AI coaches. And I just, I think that language, I think people are starting to understand how to frame it. I have the meta glasses, I just got a couple of weeks. And I'm finding myself, I go on a walk and plays music. And I'm finding myself kind of as a coach, like asking a question, it's about my day, it could be about my relationship, it could be about parenting with my kids. I'm finding it as a good on demand coach, not to tell me some top secret really complicated things. It usually just tells me the obvious. Yeah, but it's useful. It's like a sounding board. But ultimately you will still use it as a co-pilot. Cause if it, whatever it is, you're gonna be, it's not gonna exercise for you. It's not gonna eat for you. It's not gonna, right? Have your relationship for you, hopefully, because of the human nature of certain things. So exactly, we talk about moving from AI tutors to AI coaches now. Why? Because we think they're smart enough to actually go beyond the particular session you're having. You understand your whole session history. We're talking about things like personalized knowledge graphs. We're talking about more personalized, how do you make a linear equation, Y equals MX plus C, interesting to somebody who's a swimmer, okay? Or somebody who's a runner and is going up a hill. If you just think back to everything you and I learned, how much of it was made relevant to me personally in my life? And how is it possible for a single instructor to have 40 students in class and know each of their interests and say, oh, okay, let me give you a practice problem where you're running up a hill. The slope is defined by these two coordinates at the bottom of the top of the hill. What is that slope and how do you express it as Y equals MX plus C if you start at the base with a certain amount of distance. That is what AI can now start to do. Is really be your coach. There's a difference between saying AI tutor, which I think we all understand the subject, a coach understands you and the subject. That I think is where we're heading to. You say heading as if it's very far into the future. No, it's this year, next year. Yes, exactly. Hey, Peter, thank you so much for taking the time. I again continue to be inspired and really learn from how you approach technology starting with human expertise. And I think that's really missing in the market. I think it's a different view. And I think in the long term, that mindset and that approach along with AI is going to make a difference in education. So thanks for your time. And as you say many times, how we amplify human connection and that potential in humans is all that matters. So thank you for sharing your vision with us and to our listeners. Until next time, always stay curious. Thank you, Paul, and appreciate being here.
