Operationalizing Real-Time Voice Intelligence for FinServ and CX - with Ken Morino of Modulate

Welcome, everyone, to the Emerge AI in Business podcast. Today's guest is Ken Marino, Director of Marketing and Behavioral Research at Modulate. Modulate is an AI-powered voice integrity platform that helps financial institutions and enterprise contact centers detect fraud patterns, manipulation cues, and synthetic voice threads in real time. As AI-generated voice cloning tools become widely accessible, contact centers and financial institutions face a growing gap between the sophistication of fraud attempts and the capabilities of legacy detection systems. Ken examines how enterprise leaders can close this gap by deploying voice intelligence that works alongside existing infrastructure, identifying behavioral risk patterns and synthetic voice markers without adding friction for legitimate customers. The conversation addresses how organizations should define clear success metrics before deployment, structure ownership across fraud, CX, and compliance teams, and why specialized AI model ensembles offer greater accountability and adaptability than large general purpose systems in regulated environments. Today's episode is sponsored by Modulate. Position your brand alongside the Fortune 500 leaders defining the enterprise AI roadmap. For the opportunity to showcase your solution to the executive's currently funding and scaling global initiative, start with Emerge to reach the decision makers holding the strategic mandate. Secure your partnership at go.emerge.com slash partner. That's G-O dot E-M-E-R-J dot com slash P-A-R-T-N-E-R. Now the conversation with Ken. Ken, welcome to Emerge's AI and Business Podcast. Thank you for having me, Yolanda. I'm excited to talk about the topic. It's dear and dear to my heart. Yes, I'm also excited. I think voice is where we see many high-stake incidents happening when we think fraud, social engineering, customer distress. And I was reading recently that even AWS and Microsoft have actually pulled out of voice biometrics entirely, which signals to me that the old playbook is obviously broken. And I guess that's where we need to start the conversation. I want to know, Ken, if you had to walk into a large FinServe institution today or a big contact center, what that is starting to think about or scratching the surface on voice intelligence, what is the very first workflow that you would prioritize and why that specific one? I would come in, I would be asking questions. I'd be doing a lot of listening. I found that every organization has awareness of where their pain points are. And there are a lot of different things in voice intelligence that need to be tackled. But the right one for your particular organization is going to vary. Where are your losses? Where are your vulnerabilities? What are your concerns? Having said that, there's a couple they're always going to be in the back of my mind when talking to clients, prospective customers, the ones that usually crop up and kind of the things that they have pressing needs on are either fraud, especially when you've got a customer facing fraud and you're seeing high volumes or you're seeing upticks of detected instances with the technology evolving as quickly as it is for every one detected instance you're seeing. There are going to be many, many that are not detected. And so I think that people are starting to realize that that is an area of vulnerability. They aren't even recognizing the amount of losses that they're seeing, oftentimes until after the fact. So that's a huge pain point for people. And that's something that voice intelligence can really help with. Another one that people usually don't think about until it hits them is something like an IT help desk or an HR line that is meant to be more internal facing oftentimes and just has built up voice as a assumed trust. So you're going in, you're thinking, oh, I know who this person is or the person I'm speaking to is who they say they are. And there are there's just rife for manipulation. It's rife for vulnerabilities. And that's something that people are starting to harden. But it usually doesn't happen until after that first major incident. One other area that I've heard a bit about, and I think it is emerging as more of an area of growth than kind of a pressing need, but it, again, depends on what your business is, is that customer service relationship, and is there room for improvement? There's been a ton of tech out there, multi-channel, either through email, looking at reviews online, and then voice more traditionally using through a survey or a post-call response. But now people are realizing that they can do things more in real time. They can start capturing situations before they escalate. And then that's a much better situation to be in than reacting to a situation that's already gone bad. That's great. So two things that I really want us to get into on what you just said. First thing is you just mentioned real-time detection. And it sounds, if we talk to guys in the fraud ops teams in different organizations, it sounds as if they are drowning in alerts already. So how do we think about signal quality versus signal volume when we're wiring this in for the very first time? Yeah. So it really comes down to what is the useful signal for you? Is it when you have a definitive case? Is it when you are in a high leverage situation where any pain, any additional amount of friction is high stakes because of either the amount of money involved or the reputational risk? So you really want to be able to tune the noise that you're getting to the situation. You don't want a one size fits all where you're saying the call with the customer who's trying to return your $5 handbag is getting the same level of attention as, oh, I'm talking to a vendor partner on a multimillion dollar deal. There's different levels of scrutiny. There's different types of things that you want to look after. So being able to tune to the situation is really, really important. And you also never want to be interrupted when you're saying, okay, there's some warning signs going on in this high leverage conversation. But then we've got like 50 customer calls coming in. You want to make sure that you can separate those. You can put them in separate queues and you can give them kind of the attention and they warrant and they're going to the right people. I love that you explained it like this because it actually melts in perfectly with what I wanted to ask about. Next is you just said it's not a one size fits all. every industry and every organization might have unique needs for this. But obviously, there should be meaningful differences between what you would prioritize at a bank versus a large telecom contact center. So we'll have different priorities, but will the starting point be the same? Or are we going to start differently for the two different organizations? I think it really depends on what your goals are. So before you get started with voice intelligence, I think it's really important to understand what are you trying to address? What are your goals in trying to address it? And most importantly, I think the thing that often gets overlooked is how do you know whether or not you're successfully addressing that? So you need to have a really strong sense of metrics coming in to say, okay, are we looking at reduction in fraud? And how are we measuring that? Are we looking at increase in customer satisfaction? and are we considering things like the customer who's coming in calling super, super angry, and then they get to a neutral place at the end, being as much a success as someone coming in neutral, getting upset, and then de-escalating. There are different ways that things can express themselves, and you need to really have a really strong grounding for what are the use cases, how are we measuring success, and then how do we make sure that we are either adapting when we're finding things that we aren't expecting, or when the metrics that we kind of first set out for don't necessarily align with the goals that we were trying to accomplish in the end. I like, so we've established that it's not a one size fits all. The goals will be different, even though the starting goal for every organization will be defining the goal. The starting point will be defining the goal. And that makes sense for a starting point. And it reminds me of something that I came across in preparation for our conversation today, that I saw that a retailer that only figured out that they had a problem when they noticed thousands of tiny refund requests, each one with just a small enough amount that it doesn't trigger anything and no one needs to escalate it. So no one saw it coming because there was no single call looked at that they found suspicious So how does the leader even structure their operations to catch something like that We just said that returning a purse will not be dealt with the same way when we have a multi dollar investor on the line How do we focus on catching small things that could lead to big things where the threat might be invisible until you zoom out and you actually see what's happening? I think two things come to mind. I think the first thing is even if you have a really robust kind of process in place for that real time, things don't end there. The data doesn't necessarily go away. The insights that you can find don't stop as soon as that call is over. Looking at trends can be especially enlightening. So when you're talking about thousands of little tiny returns, there's probably two things that I would say were signals that you probably weren't looking for, but had you been looking for, you would have found. The first is there is probably some sort of reconference saying, what is the threshold that people are not going to be suspicious about? So you'll often see organizations do testing. So they're going to say, okay, did we get additional scrutiny at a $20 return? Did we get additional scrutiny at a $17.50 return at a $12 return? And you're like, okay, they didn't even blink when we tried to do like the $6.50 return. That's our sweet spot. And then you don't want to be like every return is 650. So they're going to do a little bit of variance in there, but it's probably going to go right up to that threshold and maybe a little bit below because they're still trying to maximize per exchange to kind of get that return on their side. And so you're saying, OK, are we seeing a little bit of testing and then like a sudden spike? And then the other thing you can look at is trends and volume. So if you know that you've got a standard return rate and you're seeing a spike in a return rate, that's going to be like, okay, why are we seeing that return rate? Do we need to dig a little bit deeper? Let's look at the audio. Let's look at the circumstances. Are we seeing patterns in the topics? Are we seeing patterns in the language being used? There's oftentimes like a scripted element in there that people aren't thinking about. So even if they're like, okay, we're going to do a generic, get a little bit of a warm rapport to get some emotional appeal, and then they go into a very automated scripted thing. And they're like, we know these are the buzzwords that we have to do to avoid getting escalated to the manager and still getting the return. There's kind of that precursor. Okay, we're going to get the reconnaissance. We're going to figure out what buttons to push. And then we're going to automate that system. So if you're looking for either one of those or ideally both of those systems, you're going to be able to pick things up before they spiral. Okay, so it's going to be, obviously, in the beginning, those trends and those patterns need to be established. Is that something that we can establish at implementation stage with data that we already have? Or will we have to start from scratch? It really depends what your system is and how sophisticated it is. So there's a lot of stuff that you can do to get a baseline. If you even just have base transcription and you can kind of feed that in and start looking for keywords, if you know the right keywords, then you can pick that up. if you have a more sophisticated group targeting you, that's not going to fly. You're going to need something a little bit more sophisticated. So you might say, we've got kind of a good baseline. We have a decent methodology, but now we need to get a little bit more sophistication in our tool set to augment that because we've caught the low-hitting fruit and now we've kind of got that more sophisticated group. But again, it varies from organization to organization. what your risks are, what your dangers are. You are probably, as an organization, very aware of that. And so you need to say, what are the attack factors that we are most vulnerable to? And how do we make sure that we have the most coverage so we stay informed when something new comes up? That absolutely makes sense. And then I guess these organizations will have their existing infrastructure, their existing workflows. How do we implement voice AI in our workflows without ripping and replacing an existing infrastructure? Integration may look a little bit different, but every solution out there really should be able to kind of work into your existing workflows. So you shouldn't be changing what you're doing for AI because you know what works for you. You should be thinking about how AI works with your workflows. That's kind of the part partnership angle, like finding the right partners to work with, they will meet you where you're at. They're not going to ask you to come to where they're at. There may be a little bit of, oh, this is our strong recommendation. This aspect is better. And that might be a slight variance in your workflow, but it shouldn't be a disruptive element to your workflow if you've got a good partnership. And all the technology, as much as it's different in the minutiae, is generally the same when you're talking about voice. You'll all have an audio stream. There are ways to access that audio stream. Even if you're working with like an old PBX system, there are ways to get those streams out real time up through the most complicated VoIP system out there that's got all the bells and whistles. You can get the audio, you can analyze the audio in real time or at worst near real time while the analytics are happening on the call. And then you can get it escalated back to the workflow that your security team or your agents or whoever you need to escalate to where they're already at. I agree with you. I think if we don't have to make too many changes to the workflows and the infrastructure, we also won't have to deal with the culture shock when employees now need to start working with that. There will obviously have to be some changes made to the workflow in the handoff. If I think about it correctly, and you're obviously allowed to correct me here, I see it in my mind as if in certain cases, a suspicion or suspected fraud might have to be handed off from our voice AI to the fraud department. What does that handoff actually look like? And what will the fraud analyst actually get? Will they get a timestamp and a risk score? Or is it something much more complicated than that? Let me take a step back, because I think that that's kind of an important point to talk about, where should there be a handoff? I think that that's the first question you need to ask, because there is a risk when you're introducing new things, especially when you have potentially a fair amount of complexity on what is the increase in cognitive load. Even if your interface isn't changing, even if kind of the process is not changing, you look at something, you kind of make a decision, you push a button, there is increased cognitive load there. And I think that that's something that is oftentimes missing where you're like, okay, you are a customer service agent. Your number one goal is to make sure that you're keeping the customer happy, but also make sure fraud doesn't happen. That is a massive amount of cognitive load. So when you have kind of the right tooling, you can decrease that cognitive load by saying you do not have to be a behavioral scientist expert to try to identify that someone is using an advanced manipulation technique on you. We have a system that already does that. But then you have a real question about who owns responsibility for that and what does that kind of responsibility and flow look like? That could mean alerting the agent. That could mean giving the agent a decision point because they are a specialist and they may have additional knowledge. Or that could mean setting up a workflow for your fraud team in real time. And then maybe you'll have some sort of notification that says, hey, the fraud team's doing some stuff. Just continue to act like something is normal. But if they ask to start a wire transfer, stall. So there's a lot of different ways that could look. And that is kind of organizationally dependent. But I think that there is a really good exercise to sit down and say, who owns this? what is the appropriate kind of responsibility structure and what is the actionability structure that you have in place to make sure that you're putting the decisions into the hands of the people that should be making those decisions and the people making those decisions also have ownership and responsibility tied to it. You just triggered something that always makes me wonder when we read about changes in technology and advancements in AI as a whole who owns it, the accountability part of it. And I'm wondering here, when something does go wrong and a fraud manager needs to explain why a certain decision was made to a regulator, to an auditor. What does accountability actually look like? What does the accountability trail look like? I think that that's a really, really important point. And one of the biggest things that you see when regulation is done right is that there is a auditable source of truth that you can say this person made this decision at this time and this is the information that they made it based off of If you can explain a decision then that outcome may not have had the best possible outcome but you can dissect why that happened, and then you can work to fix it or change it or increase the process. You can take accountability for that decision. If a decision is made kind of either in a black box or with a limited set of contexts, it's really hard to dissect that decision making process because you might be saying, well, I didn't have good data, so I just had to go off my gut. That is a really, really horrible place to be at. It's something that you don't ever want to say. So you have to go back to say, OK, do I have all the available information that I need? And can I point to that and stand by my decision, even if that decision turned out to be wrong? Because then you can say, okay, this assumption was incorrect. You did the right process. We had the right process in place, but we made some incorrect assumptions. Let's fix those assumptions. Or this process isn't working. This is why that process failed. And so now we can put a better process. It makes sense. And you'll always get those departments that will just by default feel like they don't want the accountability. And now the great thing about voice AI is that it's quite versatile. Realistically, it could sit in fraud. It could sit in CX. It could sit in compliance and they probably all have legitimate claim on it to be in control of it for the organization. In your experience and what you've seen, who should actually own it and how does a leadership team make that call without it becoming a turf war? Yeah, I think it can vary. Where I have seen success where you have multiple organizations owning different aspects. And I've also seen success where you have, we are reporting to one central authority that owns everything, and then they delegate certain aspects. And it comes down to what's right for your organization. I think that's going to be my mantra that you're going to hear over and over again is, you know your organization. You know what works. You should hopefully also know what doesn't work. And if something's not working, you can always change it. But you should start, hey, let's define super clear ownership. let's define super clear success metrics and that then we know yes this is working or maybe we need to tweak it a little bit or no we need to change this um but i think the the the kind of through line between who is successful and who is not successful is not oh we decided to put ownership under one organization or we decided to put ownership on multiple organizations it was hey we've got really, really clear ownership here. And so everyone knows this is who owns this segment, this is who owns that segment, or this is who owns everything. And so if something needs to change, if something's going well, they know who to float that up to. And if something is like, okay, this is a failure in the process. We did all the process correctly, but there was a failure in this process. We needed to own this thing. And that other group owned this thing. And because we didn't have control over that situation, something failed. We didn't have a critical piece of information. We weren't able to take control of the situation. Then that's a conversation where you have kind of a clear set of identifiable issues or a clear set of identifiable demarcations that you can say, let's try something different. Let's change this. And is there any point where we see the ownership actually having to change within the organization? So if we think about an example, if we look at something like account takeover, where the signals, they often start small. It's something small like changing an address or a phone number or something like that. Have you seen the ownership shifting when voice intelligence flags a voice mismatch during a routine administrative call? Does it ever change from this being a fraud alert to being just a normal CX red flag? Do we see the ownership changing? The most common thing I've seen is something starts out as this is a CX issue and wait a second, this is actually fraud. I think that that is the most common case you see. And having kind of a preparation for that can be the difference between, oh no, fraud happened, but the fraudster was happy and we have stopped fraud. So I think making sure that you kind of have clear handoffs and you don't necessarily silo the information. So someone can say, I'm owning this call from a CX perspective, but that doesn't mean they can't share out some of that data. And if fraud says, hey, maybe this is just a yellow flag, we're just going to put a yellow flag on this account. And the next time they call in, let's see if there's another yellow flag. Let's see if there's a red flag. And then there can be kind of a parallel because the data is not, oh, we have this kind of finite and tangible object that must live in CX only. It's audio. It's digital. That means that you could have multiple people looking at different angles and they can do the things that they know how to do properly. And you have to have kind of, again, that ownership, that specific things where fraud can come in and say, hey, CX, you're doing a great job at CX, but this is actually fraud. We're taking this over. That makes sense. So it's just, it will basically be a monitor thing before it becomes a change of who is in charge of the specific case. So monitoring, and then it could lead to change of ownership of the specific case. So it is something that is, I would say, flexible in some senses. What I'm also thinking about, and it's just because we were using words like share and everyone looks at it, is when you're in a regulated environment and voice data needs to be stored for audit trails and to prove how decisions were made, how do you balance the evidentiary value of that data deletion policies? or let's rephrase that. We need the data to make sure that our voice intelligence learns from it and that we adapt from it and that we have framework to go by. But then we also have deletion policies and privacy policies and privacy expectations in regulated environments. So how do we make those two worlds meet? This is so dependent on your industry and your country and the regions you operate in. I will say, in general, what we have found to be true is that if you have a very concrete, specific, often evidentiary reason to hold on to a piece of data, there is a way to do that in a legitimate fashion that does not step in any regulation. There may oftentimes be additional security measures or siloing and classifications. So it's really important to understand that. your vendor partners will often be able to assist you with that. So they'll say, hey, you want to delete this every 30 days, 90 days, six months, or kind of the common cadences. But if you know that this is going to be used for evidentiary proceedings, you need to hold on to it for seven years. So don't necessarily hold on to that in your big bucket of audio or your data warehouse, have a secure silo. You say, hey, this is evidentiary data. It can still be in the same format, but it's siloed. It is marked. It's not going to get touched. You're not going to accidentally delete it. That's kind of the best practices that we see. But again, there's a lot of variance between countries, regions, organizations, industries. So you really have to say, what are the rules that we're operating under? What are the things that definitely apply to us? What are the things that could apply to us? And make sure that you're following those rules, but also understanding it's not always about making sure you delete all that data. It's about knowing when you have to delete the data and when you don't have to delete the data and when you shouldn't delete the data. And just make sure that you have a good enough reason for when you do not delete it. There needs to be evidence like we just said. Okay, we're kind of heading into the last part of our conversation. and I think we should focus on how do we know if this is actually working instead of just using a giant all-purpose LLM type AI with some instructions why is it better to use a team of smaller specialized AI models and then how do we know if this is actually working yeah so let me start with kind of the the large black box versus smaller models and why we see it the way that we see it is there's kind of three bold big reasons and I can dig into specific aspects of them if you want me to but number one is cost so if I'm going to a steak restaurant I could pull out a Swiss army knife but I would much rather have a really nice steak knife it's going to work better and it going to be less clumsy and it kind of to to bridge the analogy it going to cost a whole lot less because you don have all that overhead to work with So if you working kind of with the smallest model possible that is going to accomplish this task, you know it's going to do that task really, really well, and it's going to do it at the most cost effective rate that you possibly could. Reason number two is the smaller the signals that you're looking at, the better understanding you have of why something happened. So if you are looking at, for example, a deep fake model that just says, hey, this is the probability of deep fake. We are looking on these signals. You know exactly the logic that happened. If you are looking at, say, three deep fake models, each one looking at a different signal and then averaging, then you have the signals from those three different models. You have the outcome. If you're looking at something that is more of a black box that says, hey, we think this is deepfake, you don't necessarily know the decision process. You don't know what signals could be used. And depending on the training data set, if you had someone training off of every single deepfake data had a song playing in the background, it could be keying off that. So every time that song's in the background, it's deepfake. If that song's not in the background, it's not deepfake. And that's no way to make a decision based off that because you just don't know what that training data set's going to be. And then the third thing that I would kind of call out is kind of a combination of different things, which is you don't know what your problem is going to be tomorrow. You know today that maybe you're solving for a specific solution and the black box works just as well as two or three different models. But when you face kind of the new crisis, the new problem, because nothing's ever static. People are always evolving. Things are always changing. And it worked really well for the thing yesterday. And you knew it worked well based on kind of an iteration. But then the next thing, you're like, well, this seems to be working well, but how do I know? What is it working off of? It's those unknowns. You're still working on the black box. Whereas your signals are relatively clear and you're saying, okay, these signals are still working. This third signal is not working. They have figured out how to get around this. then you can just say, okay, I just need something new. And you can drop that third signal and you replace it with something new and you can validate that piece. And that's a relatively lightweight change to make because all these models are small. They tend to be relatively quick to stand up and relatively inexpensive to stand up. And comparing that to, okay, we've got the latest model, it's 10 trillion parameters. That's really, really expensive. and we don't know if it's going to get defeated tomorrow or not. So having that flexibility, having that auditability is kind of really crucial with making sure that you can adapt and adjust to the curve as it changes. And that is so important because I think history has proven that a fraud script or whatever is trending, it basically goes viral inside a criminal network within not even days at this point. It could go viral within hours. So having that learn as you go and be able to adjust it as we go, that's going to be crucial for this to be effective and to show that it actually works. I want us to kind of end the conversation with thinking about the future. We've moved from keywords to voice biometrics, and now we're seeing the big cloud players are walking away from biometrics entirely. Where does voice risk sit in the next 12 months? I think we're going to start seeing people increasing adoption or at least kind of running some proof of concepts and realizing that they don't have adequate metrics to kind of figure out like, hey, it plays really well in the system, but maybe we're not measuring it well. So I think people are going to start honing in, hey, these are the metrics that really matter. Here's how we know if it's working or not. It's not a matter of, oh, is this useful? Is this important? Is this interesting? It's how do we know that we're getting the most ROI on that? And they're really going to start getting back to kind of your basic business metrics that say, okay, this is how we know we're successful. This is how we know that our spending is getting the most bang for our buck to make sure that we can justify this. I think there's this paradigm that we are starting to slowly slip out of, which says, hey, if we don't adopt something, we're going to fall behind. And we saw a lot of failed projects. And so now we're going back to, OK, we need to adopt something. Things are shifting. We know things are changing. We don't want to fall behind. but we also need to say does this make sense is the money that we're putting into it less than the money that we're getting out of it are we getting a return on our investment are we seeing a positive net value from technology adoption and if we're not then we shouldn't go down that pathway but there's a bunch of different pathways that we could go down so we're definitely entering that instead of thinking about do i need it we're thinking about let's go and define why i need it for different reasons. So from our conversation, I think my biggest takeaways here would be that the first thing is that real-time detection is the game changer. Not realizing that you've lost a few million dollars after you've lost it is obviously, that's a problem. The second thing that was really important, you said that integration should not change your infrastructure and it should not drastically change your workflows, which I think is something to look out for when you're looking at different vendors. And if they require you to change all of these things, that's a red flag that we should definitely consider. And then I like your three bullets that you gave to us earlier saying that when you used the Swiss army knife and you said that when you look at cost, don't just look at what you'll be saving for entry-level voice AI. Look at what you won't be losing through fraud. look at things that allow you to get smaller signals because those are the signals that will actually prevent anything like fraud or any weird behaviors and then make sure that whatever you implement caters for unpredictable problems like you said today's problem is not tomorrow's problem is there before we wrap up is there anything else that you would want to leave our audience with i I think the biggest thing is that innately voice is the channel of trust. And we are going through a time where that assumption is being shook. And I think that there's this tension because I think we have evolved as humans to this place where we can't just let go of that. So we need to understand that there's going to be this connection, this innate trust with voice. and we really need to figure out how do we guard ourselves without losing that ability to connect. And I think that voice AI is a way that we can say, okay, we can reestablish trust in a way that allows us to continue to have meaningful connections. Absolutely. And trust to our enterprise leaders is not just an emotion. It is keeping their jobs and keeping everyone else's jobs in place. So that's completely true. So, Kane, thank you so much for sharing your insights with me today. I'm very excited to talk about this on a different episode, hopefully again. Yeah, absolutely. Thanks so much for having me. Wrapping up today's episode, I think there are three key takeaways from our conversation with Kane. First, before deploying voice intelligence, organizations must define specific measurable success metrics, whether that is reduction in fraud losses, improved escalation rates, or faster time to detection. because without clear benchmarks, leaders cannot know whether the technology is delivering value. Second, voice intelligence should integrate with existing infrastructure rather than replace it, and any vendor requiring significant workflow disruption is a signal worth taking seriously before committing to a deployment. And finally, smaller specialized AI models outperform large general-purpose systems in regulated environments because they offer clearer auditability, lower costs, and the flexibility to adapt individual components as fraud patterns evolve without requiring a full system retrain. Position your brand alongside the Fortune 500 leaders defining the enterprise AI roadmap. For the opportunity to showcase your solution to the executives currently funding and scaling global initiatives, partner with Emerge to reach decision makers holding the strategic mandate. Secure your partnership at go.emerge.com. That's go.emerge.com. For further executive level analysis and to join our network of leaders delivering workflow impact with AI, visit Emerge.com. On behalf of the team at Emerge, we'll see you on the next episode.