Darknet Diaries

165: Tanya

48 min
Nov 4, 20256 months ago
Listen to Episode
Summary

Episode 165 features Tanya Jinka, an application security expert who shares stories from her career in cybersecurity, including SQL injection vulnerabilities, data breaches, incident response challenges, and her mission to help developers write more secure code through education and practical training.

Insights
  • Security policies and critical documentation often fail due to poor discoverability and naming conventions, not lack of availability—accessibility is as important as existence
  • Help desk and IT support staff need security incident training to avoid destroying evidence and compromising investigations when they encounter suspicious activity
  • Organizational culture shift toward security requires demonstrating real business impact (cost of breaches) rather than relying on compliance mandates alone
  • Blind SQL injection attacks can extract data through boolean responses (yes/no) without direct data return, making them harder to detect in logs
  • Application security programs succeed when framed as collaborative support for developers rather than adversarial testing or enforcement
Trends
Security incident response requires cross-functional coordination and clear chain of custody procedures to preserve evidence for legal actionOrganizations struggle with incomplete application inventories, creating blind spots in security coverage and audit complianceHelp desk teams are first-line security responders but often lack training to recognize and properly escalate security incidentsDeveloper resistance to security testing decreases when business impact of breaches is quantified and communicated transparentlySecure coding and application security programs require ongoing education and positive reinforcement rather than punitive approachesNetwork bandwidth abuse (self-inflicted DDoS) can be mistaken for malware incidents without proper monitoring and investigationStatutory holidays and off-hours are attractive windows for attackers due to reduced staffing and monitoringCTF (Capture the Flag) competitions are effective recruitment and training tools for building diverse security teamsSQL injection remains a critical vulnerability despite decades of awareness, indicating persistent developer knowledge gapsOrganizational rumors about security incidents persist despite official corrections, requiring proactive communication strategies
Topics
SQL Injection VulnerabilitiesBlind SQL Injection AttacksApplication Security (AppSec) ProgramsSecure Coding PracticesData Breach Response and InvestigationSecurity Policy Documentation and DiscoverabilityIncident Response ManagementEvidence Preservation and Chain of CustodyHelp Desk Security TrainingPenetration Testing and Vulnerability AssessmentBurp Suite Security ToolOWASP ZAP Dynamic ScanningSecurity Incident TriageDeveloper Security CultureCapture the Flag (CTF) Competitions
Companies
SharePoint
Used for storing security policies and documentation; criticized for poor discoverability and navigation
OWASP
Organization behind ZAP, a dynamic application security scanning tool used for vulnerability detection
People
Tanya Jinka
Application security expert and author who shares career stories about SQL injection, data breaches, and building sec...
Jack Recyder
Host of Darknet Diaries podcast who interviews Tanya and shares his own security policy discoverability anecdote
Eric
Incident response manager who discovered evidence destruction issue with help desk technician and co-led security tra...
Quotes
"The problem wasn't my technicians. It was that the security policy was buried way too deep. It was named poorly. And nobody knew where it was."
Jack Recyder
"I hope software developers write more secure code. Boiled down to a word, that's called appsec, application security."
Tanya Jinka
"I know you want to help. I know you want to help. That's why you're so good at what you do. But if you see anything that you think looks criminal, you need to call us right away."
Tanya Jinka
"Tanya, I had no idea how serious this was. I'm sorry. This will never happen again on my watch. We are going to be number one."
Development Manager
"Application security is yelling at devs... It should be helping devs make more secure code and be agonized to them, ideally, most the time or all the time."
Tanya Jinka
Full Transcript
Hey, it's Jack, host of the show. For a while, I worked out a big company doing security engineering. Every year, someone would come in and do an audit on us, and they would ask us the same question. Do you have a security policy? Yes, of course we do. Is it available for all of your employees to find? Yep, it's right there on SharePoint. But this got me thinking. Yeah, sure, it was right there in SharePoint, but it was called something ridiculous. Like ISP underscore overview or something like that. And ISP stood for information security policy. And it made me wonder if this document was so important that we would be audited to check to see if we had it and make sure all our employees had access to it. Could any of them actually find it if they needed it? Like this policy said stuff like, what are our security objectives? Who are the people that we escalate things to? What's acceptable in our network and not? Who should be able to access what? As well as what we should do when there's an incidence, how often our security training should be, and what our security standards are. So one day when I was feeling feisty, I decided to do something to make a point. I asked everyone on shift in our network operation center, hey, you have 15 minutes to find the company's security policy. Winner gets a free item in the vending machine. Go. Everyone started looking. First they typed security policy in our department's portal. And that actually brought up security policies for some of our customers, which I thought was really cool that our customers were taking their security policies so seriously, that they wanted to make sure that their partners had the copies of it, but that wasn't our policy. Then people started looking through their emails. Nope, nothing in our email about security policy. Then they looked at shared drives. They couldn't find anything there. And eventually a few of them thought to look through share points. And of course, not a single one of them could find it because it had the worst name. And it was in the worst place. I don't know if you've ever used SharePoint, but it's a place to store documentation and files. And it's an awful mess to navigate and find stuff. None of their searches came close to finding it. And so I just said, all right, everyone, time's up, thanks for trying. And then I sent an email to our CISO, our chief information security officer, security policy test, Q1, 10 out of 10 of our knock technicians could not find our company's security policy after spending 15 minutes trying. And he responded, it sounds like your knock technicians have a hard time finding things. I waited another four months. We got a whole new batch of technicians. And I tried again. One guy actually found it. I was really impressed. I also retested all the people that I tested four months ago. One in five, remember where I told them it was. So I sent another email. Nine out of 10 of our new hires could not find our security policy. Four out of five of our senior technicians could not find it. He was like, why do you keep telling me this? Just show them where it is. I wanted him to understand. The problem wasn't my technicians. It was that the security policy was buried way too deep. It was named poorly. And nobody knew where it was. Nobody could find that if they tried which meant nobody knew what was in it. In my opinion, when there's a document that's so important, that auditors ask if you have it and if it's available for employees to find, then it should be way more front and center. Heck, I even suggested that we should print out a summary of it and tape it above the urinals. And things in the bathroom. So everyone sees it every time they go to the bathroom. That way the whole company would be familiar with our security policy and know exactly what to do when there's an incident and what's allowed and not allowed. But of course, our security leadership didn't see it that way. And never did change the name of it or the location. And we kept passing our audits somehow. Yet nobody in the company ever read it or knew where it was. The politics of office life and compliance. These are true stories from the dark side of the internet. I'm Jack Recyder. This is Darknet Diaries. It's a great thing to do. This episode is sponsored by DRAWDA. Let's face it, if you're leading GRC at your organization, chances are you're drowning in a sea of spreadsheets every day, balancing security, risk, and compliance in an ever-changing landscape of threats and regulatory frameworks can feel like running a never-ending marathon. Enter DRAWDA's agentic trust management platform designed for leaders like you. DRAWDA automates the tedious tasks, security questionnaires, responses, continuous evidence collection, and much more, saving you hundreds of hours each year. With DRAWDA, you can spend less time chasing documents and more time solving real security problems. With DRAWDA, you also get access to a powerful trust center, a live customizable product that supports you in expediting your never-ending security review requests in the deal process. It's perfect for sharing your security posture with stakeholders or potential customers cutting down on the back and forth questions and building trust at every interaction. Ready to modernize your GRC program and take back your time? Visit DRAWDA.com slash Darknet Diaries to learn more. That's DRAWDA spelled D-R-A-T-A. DRAWDA.com slash Darknet Diaries. This episode is sponsored by Delete Me. Delete Me makes it easy, quick, and safe to remove your personal data online at a time when surveillance and data breaches are common enough to make everyone vulnerable. Delete Me does all the hard work of wiping you and your family's personal information from data brokers websites. Since privacy is super important topic to me, a few years ago I signed up Delete Me immediately got busy scouring the internet from my name and gave my reports on what they found. Then they got busy deleting things. It's great to have someone on my team when it comes to my privacy. Plus, The New York Times Wirecutter has named Delete Me their top pick for data removal services. Take control of your data and keep your private life private by signing up for Delete Me. Now, a special discount for Darknet Diaries listeners. Get 20% off your Delete Me plan when you go to joindeleteme.com slash Darknet Diaries. And use promo code DD20 at checkout. The only way to get 20% off is to go to joindeleteme.com slash Darknet Diaries and enter code DD20 at checkout. That's joindeleteme.com slash Darknet Diaries code DD20. Today I have the pleasure of sitting down and hearing stories from Tanya Jinka. Thank you for having me. I've been going to a lot of conferences and time and time again, I see Tanya at almost all of them. But not only is she there, but she's almost always giving talks when she's there. She's on a mission and is very driven. I hope software developers write more secure code. Boiled down to a word, that's called appsec, application security. She's laser focused on how applications become insecure and how to make them secure. I was a software developer forever and then someone exploited one of my apps and showed me and it created this fascination. It was an SQL injection and it was on the login screen of one of my team's apps and eyes in charge of the team. And so if it's not secure, it's my fault. And I remember he was giving a demonstration to us and he showed me, he's like, this is one of your apps. I'm gonna get past this login screen without a password. And the only reason it's gonna take so long is because I'm talking and it's gonna be a minute. He demonstrated how he can easily get past her login screen and showed her how it's done and she was stunned. Oh my God. This was the moment that she saw the whole world differently. Just because there's a right way to use a website by putting in a username where it says user name and a password where it says password, doesn't mean people actually play by those rules and follow the website's logic. If you're clever enough to think outside the box, you can manipulate the website to do things that the developer didn't intend. And we ended up becoming very close friends and he became my first professional mentor for hacking. This was a career pivot. Instead of building things, she wanted to know how to break things. And all this is happening in Canada, by the way. Tonya was living and working in the capital city of Ottawa. So her new mentor was like, okay, so you wanna be a hacker? I've got some work for you. You can help me do some penetration tests. And she's like, okay, but I'm not exactly sure how. And so he told me on the Friday, okay, so go learn Burp Suite this weekend. There's videos on YouTube, just go watch them. It's not hard. So she starts watching them. Burp Suite is a tool used to monitor the packets that go between your computer and an application or a network or website. So you can redirect all your computer's traffic to it. And then Burp Suite will show you, hey, you went to this website and it responded with this code. And then your computer sent this information back. And then this website sent back a cookie, which has this data in it. It's kind of like getting under the hood of a car, but for network traffic. And Burp Suite is really cool. You can capture that data and replay it if you want. Like maybe you look in the cookie that a website sent to you. And it says that your user ID is 5,000. And so what if you change that user ID to 5,000 and one and then reconnect to the site and present this cookie, which has a different user ID, will it think that you are a different user? It's kind of like a way to do surgery on the packets that your computer is sending to an application or website. And it's possible to manipulate your packets enough to make the site do some very strange things. So she comes back in on Monday with the basics of Burp Suite understood. And he tells her, okay, great. Spend a few hours a day trying to hack this website and tell me what you find. And he says, I'm gonna be observing you silently while you're working. And I just need you to report anything that you find. And I was like, great. And so the first night, I just found really tiny things. The second night, I found really tiny things. And the third night, he kind of gave me a lecture and he's like, listen, Tonya, you've got to find something. You can't do a pentast and not find something big. And you should really think outside the box, take off your developer hat, put on your black hat. And so I just tried everything I could think of. And I found Service I Request forgery. Now I didn't understand Service I Request. How'd you find it? Basically, there was an email field. And I just started entering in code. And I entered in an email, but then after I just put in, everything I could think of, like all sorts of code, all sorts of stuff. And inadvertently, I start copying and deleting files on the web server. Yeah. And I ended up crashing the production web server. And it turned out I had polluted the database as well. They had to restore both from backup. And so I call my boss and I found something and I have crashed everything. And he's like, what's this is production? You can't crash production. And I'm like, well, I found an exploit. And you told me to prove that I had exploited it. So here's, I took the whole thing down. And he's just, he yelled at me. I was really angry. And then he's like, how did you do it? I'm like, these are all the commands I did. And he's like, well, this is garbage. This shouldn't do anything. And I'm like, well, I guess it did. Right? I was like, well, weren't you watching me the whole time? And he's like, no, that's just something I said to make you feel better. And I was like, well, like, what is the client going to say? And he's like, I'm like, I can talk to them. He's like, no, they don't know that you're testing. I'm like, but you said I'm a subcontractor, right? Like, so like the contract I signed with him that I was subcontracting, that they knew. They had no idea I was on their network. So he had given me the keys to production, to some random client. They had no ideas on there. I destroy everything. And he's like, great. Now I'm going to get yelled at tomorrow. And it's all your fault. I'm like, really, this is all, she was really pissed at me. I think they both learned a lesson that day. But Tanya was hooked more than before to copy and delete files on a web server, all by putting in some code through a form field. Wow, such power. But wow, such weakness. These apps she was seeing are surprisingly weak. And she was drawn to that. What are other tricks and techniques from making an app give you data that it's not supposed to do or do things that shouldn't let you do? She got more and more into security, wanting to get more hands on with everything related to it. And I kept annoying the security team constantly. I would report security incidents. I would fix all the security bugs. I kept asking if I could use security tools. I volunteered to work on their projects. And one day, they said that I could sit in on an incident and just watch and shut up. And literally, I did not have a seat at the table. They had only so many seats that would actually fit at the table. So I was actually against the wall at the back of the room, just being zipping in, being quiet, like they told me to. And then I remember them putting all this stuff on the screen and looking at it and being like, oh, that's SQL. Oh, that's pretty bad. And so I said to them, like, I am seeing code here. We need to look at this. Can we talk? And they're like, you can read that. And like, just because someone's trying to SQL and inject you doesn't mean they're successful, right? But I'm like someone's attacking us. And every organization's getting attacked all day all the time. And the fact that I could sit there and read code, they were like, oh, she's an asset. And then a few weeks later, they said, oh, we've opened a job on the security team. And I was like, oh, my gosh, I'm going to apply. They're like, obviously, it's for you, silly. And so they let me transfer on to the team. And I was just so excited to be a part of their team. With this new position, she proved herself again and again and rose up to ladder, eventually landing in a security leadership position at an organization, which was within the Canadian government. She was in charge of making sure that agency and all its abs were secure. And one day, she came to work and had an email waiting for her. I receive an email from Vice Magazine. And it says, dear Tanya, we know you work at this place and that you're the leader of this team. We would like a quote for you for our magazine about how your data is for sale on the dark web and how you feel about that your data is worth only $48. And here's a link, if you want to see more, and it's a link to paste bin. And on it, it says, here's a sample of the data from the name of my organization and to get more go here. And then I go there and they're auctioning my data for the Bitcoin equivalent of approximately $48. Which is not a lot of money. What a way to be notified that your agency has suffered a data breach by getting an email asking you for a quote on how you feel about your data being for sale. So I talk to my team, I'm like, ah, ah, and all of us are just flabbergasted. We're like, first of all, what is this data? Is this actually our data? And so we're looking through all of our apps. And this is when I realized my app inventory was not complete. We're missing lots and lots of apps I did not know about that I'm supposed to be securing. Step one of a data breach like this is to verify that it's your data. Find out which app or database it's from. And this will help you identify maybe which app is vulnerable. But it took them a while to figure out even where this data was in their network. Eventually, they narrow it down and figure out which app this must have come from. And so I go, I find that data is in there. The paste bin sample does look a little familiar. And I'm like, oh no. So I go and I talk to my boss and my boss was so pissed. He was like, have you ever had someone say your name in a way where it sounds like a swear word? There's like, time. Yeah. And I am not really good when people are upset with me. And so I was like, well, sir, our data is for sale on the dark web, that's true. But I think the bigger problem is that it's only $48. Don't you feel we're worth more than that? And he was not impressed. Somehow, my name was at like a higher pitch the next time he said it. And he's like, you are going to go fix this now. Yikes. There's a leak data from the Canadian government is getting leaked. And Tanya has had a security for this department that I got leaked out of. This is really bad. She pinned one of the application, though. And got the owners of that application together. And I sat down with the team and they're like, we have no idea what you're talking about. And I showed them and they said, yep, that's our data. And I said, OK. And one of my team members said, I think we should buy the data to make sure it's an exact match. And it's not like, because they're just showing two or three records on pace bin. They weren't showing all of it. And I was like, well, I don't want to give them money because I feel like that's encouraging them. And it feels pretty obvious that if they got three records, why couldn't they get more records? And so then we look at the data. And it's completely unclassified. It was actually data. It turns out that we have been trying to promote to the Canadian public for quite a while and had been being mostly ignored. Like some journalists would look at it for like a media piece or something. But generally, like no one was paying attention to a thing we were hoping that they would. So we're like, maybe this will help. My boss also did not find that funny. OK. This wasn't as bad as it seemed. A lot of data within government is in fact unclassified and publicly available. And it seemed like the hacker stole some publicly available data. And nothing sensitive was actually taken or sold. But what do you do here? If a hacker stole data that's publicly available, is it actually stealing? Is there any action even due here? Like, what's the big deal? Yes, this data was public in general, but they had the record ID number. And that's not public. So they someone clearly got a copy of our full data set as opposed to just what we wanted to show the public. And every single record except the ID identifier that we used to look it up was considered unclassified. So none of it was sensitive in nature. Let's see. Every line of the database has a unique ID, which are important or even sensitive information. However, it's not public information. It's only used for how the database sees the data. So whoever got this had full readable access to their database. So it was time to dudge through the logs to try to find out what happened. We had only database logs. We had no web app logs. The app itself did not log at all. It was really old. And so we looked through all the database logs. And very quickly, I figured out what the attacker was doing was one attacking us on every single statutory holiday. So we in the government would get paid a time and a half if we work overtime. But if you work a statutory holiday, you get two and a half times your regular pay. And it's policy that unless it's an emergency, you are never booked for on call or anything like that on those days. So we would never work those days. And so this person for a year, every single statutory holiday would start hacking us basically at midnight all the way until the next day. And so I started looking through the logs. We'll basically first I looked at the most recent logs. And then I was like, have I ever seen these commands before? She did recognize what was happening at least kind of. She recognized that the commands in the database were trying to do SQL injection. SQL is the type of database used. And injection is where you try to put your own database commands in through the web form field. So when you go to log in on a website, you put your username and password in. Well, the website will grab that information and then go check the SQL database to see if your username exists, which an SQL statement might look like select from table where user equals jack. If that's my username, right? And since jack is what the user typed in, then that's actually what gets queried in the database. But what an SQL injection does is it messes with that. Since you're going to take whatever username I type in and search the database for that, what if I type in the username jack, but then also write something else like jack or select from table all the passwords. So now if it's vulnerable to this, it'll take that input and go do this database command. Select from table where user equals jack or select from table all passwords. And if it's vulnerable, it might return all the passwords that you asked for. You see how adding extra commands into a form field can trick it to return extra stuff. The developers didn't want you to do to fix this. Developers of the apps need to sanitize their apps, not let users put in extra stuff like that. And really restrict what's allowed to be typed in those form fields. So Tonya recognized this was SQL injection by looking at the database logs, but it didn't quite make sense. It wasn't your classic SQL injection. But this one was doing true's impulses. And I was very confused. So they were looking for this person, and they're always true. And then it would say, and instead of or, which I was not used to, and it'd say, and, you know, the name of this table, the first letter is A. And I was like, what? And why do you need and? I'm very confused here. And so we, I started running them, and almost all of them were false. It would just return an error. There's no one named jack. And I was like, but there is a person named jack. And I'm like, but that's because they did the AND. You have to both of them be true. Like, this is so weird. So I kept going through and finally one was true. And I'm like, well, what the heck is this? And I look, and it's a letter from one of the names of the fields of that table. And I'm confused. I'm like, why would you look up an A? Like, it was like, is this A? Is this B? Is this C? And I was super confused. Well, they fixed that app so that it wasn't vulnerable to SQL injection anymore. But they were still perplexed on how those commands worked, how they got data with those commands. We spent two or three months looking at it. And no matter what we did, we couldn't figure out how they got the data. There were just errors or returning this one record. She never did figure it out. She ended up leaving that organization, still not understanding how that data got out with those commands. And so then I went to DEF CON. And I did a workshop called Blind SQL injection. And I was super excited to finally make it into a workshop because I don't know if you know Jack. But there are long lines. And there's a lot of competition to get those seats. And I made it. And so here I am at the back of the class. And the teacher is explaining, oh, well, what you're doing when you do Blind SQL injection is you are asking questions. And the questions you are asking is either the names of fields and the database, the names of tables, what's inside a field. So it's like, oh, this record exists. It's great. Is there a field called this? No, is there a field called that? Oh, there is. Is the first letter, because you can't say, return that record. It won't do it with Blind SQL injection. And so the only option you get back is yes or no. You can ask the database any question, but they're not going to give you data. They're just going to tell you yes or no. Exactly. And so if it's an error, it's a no. And if you receive the record that you've searched for every time, it's true. And so I went to this workshop and it's like this giant light bulb went up for me. And I was like, oh, my gosh. And so I call my old boss. And I'm like, I know what happened. And he's like, have you been poking around since you left? I'm like, no, I went and I took a workshop and I learned. And I know exactly what happened. And so I went back and we had a meeting in our special secret room. And that wasn't very secret. Anyway. And we had a meeting. And it's funny because I walked them through the logic of this is how you ask the database questions. And this is how you can know for sure that it's true. So I explained this. And then all of them except the really, really big boss. The really, really big boss was like, I still don't get it. But everyone else is nodding. So that's fine. So they did X-Full Trade our data. And that is what happened. And OK, so now we know. We're going to take a short ad break here. But stay with us because Tanya is going to tell us more stories about the fires that she's extinguished. This episode of Dark Net Diaries is brought to you by Flashpoint. 2025 has proven to be a pivotal year for security leaders. It's not just cyber threats anymore, physical risks, and geopolitical tensions are colliding, creating a web of challenges no one can afford to ignore. That's where Flashpoint comes in. As one of the largest private providers of threat intelligence, Flashpoint delivers what security teams need most. Clarity. By combining cutting edge technology with the expertise of world-class analyst teams, their Ignite platform gives organizations instant access to critical data, expertly analyzed insights, and real-time alerts, all in one seamless platform. From Fortune 500 companies to government agencies, Flashpoint is a name trusted to keep people, assets, and operations secure. To access some of the industry's best threat data and intelligence, visit flashpoint.io today. That's flashpoint.io. Tanya had a lot of roles in different companies and organizations over time. And at one point, she was leader of incident responders. You know, if there's a severe security problem in the network, it would be her and her team that would manage the problem. She would identify the problem, engage with the right people, and get working on it. And to leadership, what's happening? And then stay on the incident in order to make sure it gets the resources it needs to get resolved. And so I was the lead of the incident responders. So we had a guy that did malware analysis, all of those things. And so I was the abstract expert as not surprising, right? And so I would always do the software incidents. I came in twerk late one day because I had a dentist appointment. And I had told my boss, I had told my team where I was. It was in my calendar, anyone could see. And I come in at maybe 10 a.m. And basically there were two of us that managed incidents. Me and this amazing person named Eric. And I come in and all my team sitting there, including the Eric that is the incident manager. And I'm like, hey guys, what's up? And they all look really tense. And they're like, there's a really big incident. And everyone's in the really big boardroom. And I'm like, but Eric sitting there, and I'm standing here. So who's managing the incidents? And they're like some guy named Dan from Help Desk. And I've changed Dan's name because that is what you do. And I was like, what? And they're like, yeah, they wouldn't let us in the room. And I was like, what is happening? They're like, we need you to go in there. They won't listen to us. So I go in. And I open the door. And they're like, Tonya, where have you been? I'm like, at the dentist, no cavities. And no one thought that was funny. And they're like, we needed you. And you weren't there. Everyone stared at me. And I'm looking. And there is the director of every department, a bunch of managers and all of the executives from our organization in this room. So this is an extremely expensive meeting. Everyone looks really stressed and upset. And there's, so this was a while ago. So there was that big, huge thing in the middle of the table with the phone, with the giant button. But it's in it. And it sounds terrible. And yeah, it's one of those. And there's this guy on the phone named Dan from Helpdesk. And they're like, we're having this huge incident. And you weren't here and we needed you. And it's but Dan's helping us. So we don't need you. And you can go. And I'm like, I'm not going anywhere. Like I'm the head of incident response. I'm the incident manager. There's on duty now. And I'm doing the thing. I'm like, I've got this Dan. And he's like, oh, no, I have it. I'm handling it. She's like, who the heck is this Dan guy? Dan was from Helpdesk, which is often the front line for office workers when they have problems, right? If your computer stops working or they internet it's how, or you're locked out of your computer, or your password doesn't work, who are you going to call? Helpdesk. And that's where Dan was working. And he was answering a lot of phone calls that day. He just kept getting call after call from that office. People were saying nothing is working. Managers run a panic. They can't do their work. People were getting so upset in that office. And I found out later we had people go home because they'd had at least one panic attack. Just several people were just too nervous and upset that they actually went home for the day because they just felt very uncomfortable and unsafe. And just call after call was coming into the Helpdesk. And Dan was answering these calls. And he was doing his best to solve the issues. I'm like, OK, so what is happening? So I'm standing there in HQ or headquarters office. We have a satellite office that's maybe 20 kilometers away. And I am informed that our satellite office is infected with malware. And I said, oh, someone has malware. No, we'll go mop it up. We'll be right there. And they're like, no, no, the building has malware. And like, the building's dumb. It can't have malware. And I laugh. And then someone says, don't call them dumb. They're nice. No, no, no, the people aren't dumb. The building's dumb. And they're like, don't call them dumb. OK, the building's not smart. And that didn't go well either. I'm like, so a smart refrigerator is internet connected. It's not internet connected. It's cement. Cement does not get malware. And they're like, Dan knows and you don't. You weren't even here. You were busy at the dentist. I got so much flack about the dentist. You would not believe. But anyway, so everyone's very upset. I try to calm them down. I'm like, listen, my team will look into this. And Dan's like, we should evacuate. They're in danger. He's like ramping them up. So they are panicking. I'm like, Dan, that's not true. Everything's fine. Let my team look at this. And finally, I get everyone, I wouldn't say settled. I would say that they were less panicky. I'm like, everyone, go back to your desk. I am going to update you in half an hour. I am going to find out what is happening. Everything's going to be OK. And they're like, someone needs to go to the dentist instead of helping us. But literally, people were so upset with me. They're furious. So I just miss everyone. I hang up on Dan. Dan's not helping. And he said over and over again, the building has malware. We should evacuate. And I was like, no one's evacuating. And so I go back to my desk and I'm like, someone flip on Wireshark. He's claiming the entire building has malware. We all know that's not true. They all respond. But the building's dumb. I'm like, I know. I know, guys. We all know. Dan has whipped everyone into a frenzy. We need to do something about this now. So we flip it on. And so there are some stereotypes about Canadians. And some of them are true. Like they take our passport if we're rude. We all eat poutine. There's many, many stereotypes. And one of the stereotypes is that we love the Winter Olympics. We love watching hockey. We love watching the figure skating as an entire nation like we tune in. We really like it. And so when we turned on Wireshark, we immediately saw every single person in the entire building was going to the exact same site. And the figure skating for the Olympics was on in Canada, Whiskating. So there is no malware. The reason why nothing was working is if everyone is live streaming the Olympics, that takes up a ton of bandwidth. So the work that those office workers were supposed to be doing, they couldn't do it because the network was basically clogged up, bogged down. They essentially did a DDoS attack on themselves. And the funny thing was they had a policy in place that should prevent things like this from happening. We have a policy in the government, or we did at the time, where when the Olympics happened, we knew Canadians are going to Canadian. And so we would make a boardroom in one building. And that was where the Olympics are showing. And so if you need to go see your guy, win his thing, you go and you watch the skating and the twirling and whatever it is you're going to do. And no one's allowed to stream it because if every single person is streaming, there's no internet. So we blocked that and make many Canadians cry. And we found out later that some executive had decided, oh, you're going to take a vacation day if you want to watch the Olympics. Like you're here to work blah, blah, blah. And had gone against policy thinking they were super smart. And this is what had happened, right? And so a column meeting on the next hour and I'm already sending emails, explaining to everyone, there is no malware, there was never any malware, everything's fine. So I call everyone into the room and like, hi, everyone. Everything's fine. Everything's cleaned up, there is no problem. There was no malware. They're like, but when are we going to clean up the malware? I'm like, there never was any. Everyone was just watching the Olympics, the internet slowed down. Everything is fine. It was actually always safe. We do not need to panic. I need you to all go calm your staff, especially the satellite building staff, tell everyone everything's fine. They were always fine. We just were too busy streaming and not busy enough working. And everyone seemed not super satisfied with that answer, but enough, right? And so everyone left. But going forward, people talked about how that building had had malware for six months. Like I couldn't squash the rumor. It didn't matter how many times I corrected people. They're like, yeah, she doesn't believe it. She doesn't know. I'm like, I'm the Incident Manager. So after it was all fixed and resolved, it was time to pay a visit to the help desk to help them identify and handle incidents better. So help desk wants to help, right? Like people that are really good at help desk, they love literally helping and solving problems. And so they are the first line of everything, right? Like you call help desk. You, first of all, you've tried, you fiddle around yourself, you try to fix it. If not, you go to them. I go to them, right? If I can't fix it myself, which it happens. And so this person received this call and they're like, I know what I'll do. I will solve this problem for them. Because that person, because I know, because I was working at that work, had never had any training about what a security incident looks like. And so what my team did to solve this problem going forward is we had help desk in and we gave them a training on what security incidents look like and we told them, we will never, ever, ever get angry if you call us and it's a false alarm. I'd rather 20 false alarms than one where you didn't call and we made a mess. So her and one of our incident managers named Eric gave some training to them. And Eric had a doozy of a story himself to share with the help desk team. So at Eric's last job, he was an incident handler. If there was a security incident, it would go across his desk. And one day someone from the IT help desk discovered a problem. They were given a computer to fix something on and when they were looking through the computer for problems, the help desk technician discovered sexually explicit images of children. And he, understandably, was extraordinarily upset. Yeah, I mean, of course, seeing images like that, you can't unsee it. It feels like you did something wrong just by taking a look. Well, this IT help desk technician was like, well, that's wrong. Employees shouldn't have this on their computer. And he deleted the images and then he was still upset and he formatted the drive. Which actually makes sense. When people who work in IT help desk see problems, it's usually on them to fix it. Virus on computer, clean it off. Apps installed that are against company policy, delete them. Apps missing, which should be there, install them. Software out of date, update. Help desk people are action oriented. They take control and fix things all day, every day they're fixing things. So for him to delete these photos seemed like the right thing for him to do. So he calls incident response. He's like, man, I was just fixing problem on some employees' computer and I found sexually explicit images of children. And this feels like something I should report to you. And Eric, the incident response manager is like, okay, wow, thanks for telling me, how bad is it? Real bad. Okay, well let's be careful here. Can you show me what you found? Essentially what happened is the entire chain of custody the evidence was ruined. Because to help desk technician deleted all the evidence and didn't take any screenshots, I mean, how could you take screenshots? And then he reformatted the hard drive. There was zero proof that what he saw was actually there. So there was nothing for the incident manager to evaluate. But they did report it to HR. They were able to fire that person for violating the acceptable use policy of the computer. But HR was like, hold on. This is actually more than an acceptable use violation. This is illegal. We should report them to the police. And so they did. But then the police are like, okay, show us the evidence. And they had nothing to provide. There were traces of backups and archives that they could have dug into, but it didn't matter because the chain of custody was broken. So they had nothing admissible to give. So they're unable to prosecute that person? They had a what a blunder by help desk there, huh? The poor help desk guy, he feels incredible guilt. And the person from help desk ended up in therapy for a long time. Why? Well, probably for two reasons. So one is he felt incredible guilt because he did not know better. So he did what help desk does, which is usually a race, reformat, a reimage. And so he did what his training told him to do, right? But meanwhile, he saw things he can't unsee. And he also unintentionally let a very bad criminal go free. And so when I give training on the topic, and I talk to help desk, I'm like, I know you want to help. I know you want to help. That's why you're so good at what you do. But if you see anything that you think looks criminal, you need to call us right away. If you see anything where you're like, this just makes absolutely no sense. I need you to call us right away. Like if all your normal steps have fixed something, don't work, please call us. And we will come in because we have different tools than you have. So we started this annual training that me and Eric would give, where it was just like, these are the things that we need you to know. And the training would just be like 20 minutes. And it was just very basic. If you see this, call us, we will never, ever be angry. Now, Tonya has been to a lot of conferences. It's a great way to learn and meet amazing people. But one really cool thing you'll often see at conferences are CTFs, which stands for Capture the Flag. It's a game where you can form teams and then try to hack into something. Like there is a computer that's intentionally vulnerable. And if you can hack into it, you'll see a flag. And if you can get that flag, you'll get points. And the team with the most points wins the CTF challenge. I did do a few CTFs. I went to a bunch the first year, year and a half, and I was trying to become a pentester because I heard their great way to learn. And I did learn lots of things. And I also learned that I was always the only female everywhere I went, everywhere I'm the only woman. And I was a little tired of that. So I put a note on LinkedIn and said, hey, do any women want to form a CTF team with me? Because I don't want to be the only woman everywhere I go. Where was this going to be? It was going to be in Ottawa. And I ended up having so many women say, yes, we had to form two teams, which was really exciting. I was pretty surprised. And all of them said the same thing. Like I was curious to go, but I felt like I didn't know enough. And I'm always the only woman in it. And it's weird. And so a bunch of us were party dresses, which was really fun. And so I was showing them, OK, so here's this log in screen. We're supposed to try to get past the log in screen. And I'm like, I know how to do this. I'm sure there's going to be some sort of SQL injection opportunity. And so I was walking them through it the way that my mentor had walked me through it. And I showed it to them. And then we got in. Tonyo was able to use SQL injection to bypass log in screen. Basically, when you type in the username and password, the website sends the data to the database. And if they are a match, the database returns true. If they aren't, it returns false. Well, she put in the username field, something that will always return as true. Like, is there a user named Tonyo? Or does one equal one? And because there's an OR statement there, and one equals one is true, the database returns true. No matter what the username is. So since the database returned true, she logged in without providing a valid password. Her teammates were amazed at how she did it, and asked her to explain. Yep. And then two of us got up and did happy dances. And a third one got up and she's like, hi, I have to go. And we're like, where are you going? And she's like, I have to go to work right now, because I am not sure that we are safe from this. And I need to go test every app I've ever built and make sure that it is OK. And I have to go right now. And she literally went to work. And apparently, she was there quietly, because she came to the CTF, Quake Blurry, the next morning. And I was like, oh, how did it go? And she's like, we're fine now. And I'm like, now. And she said she had fixed a whole bunch of things. And she's like, what's the next thing I'm going to learn to fix? Let's do this. So in the new CTF, she learned she was vulnerable and ran out of there. I think she suspected. I don't know if she knew for sure, but she's like, I am shocked. And she just ran out of there. So professionally, Tanya has two passions. Application Development, which is coding and cyber security hacking. And so over time, she simply found her favorite place to be was at the intersection of these two things. She's given talks and written frameworks on how app developers can write secure apps, which is known as secure coding or application security. OK. So application security is yelling at devs. Why do you laugh? Why do you laugh? It should be helping devs. It should be helping devs make more secure code and be agnized to them, ideally, most the time or all the time, in my opinion. And so I was in charge of pen testing and doing, like, running and launching their first app-seq program. And so there was five developer teams. I was asking to be able to pen test their apps before they went to prod. And I was hoping that they would scan their apps with Zap for me first. Zap? Yes. So Zap is a dynamic scanning tool that used to be part of O-Loss. And it's the most used dynamic scanner on the planet. And basically, I wanted the developers to scan the app first and I'd made a grid. So I'm like, if you find this, fix it. If you find that, just ignore it. But the manager of that development team did not want his developers to do any of this. And one of the teams, their manager, told me, leave my devs alone. We don't have time for your crap. I was pretty new to AppSec. It was only my second job in AppSec. And he felt I was inexperienced. And that in his words, I was a pain in his ass. And I was like, I'm here to help. And he's like, then go away. That would help. And I was like, listen, I need to take a look at your apps for securities. Like, they're fine. Just trust me. And I was like, well, I'd like to talk. He's like, I don't have time. And each time I kept trying to approach him, he was more aggressive. And so the last time I'd talk to him, he'd literally said, go fuck yourself. Get the fuck out of my face. And was pointing in my face and pointing away. And then you just started yelling at me. And so I left. Rude. Rude. But this is why I don't want to be a manager. Managers take on too much stress, directives from higher ups, deadlines with not enough resources to get it done. And their team always having problems too. And they can't always be transparent about things either. Like, how much their budget is or plans for upcoming layoffs if their manager has a bad day. And that rubs off on them. And that means that manager's team has a bad day too. Or someone like Tanya gets yelled at for no reason. My boss was like, I know what we're going to do. We're going to hold a meeting. And we're going to tell them about a whole bunch of security incidents. And we're going to deputize them and tell them not to tell me when. So don't worry. I'm like, I'm very worried. And it's going to be fine. And then they'll listen. And I was like, this is a terrible idea. And so he invited them in. And he explained what it's like when a computer gets malware. And he's like, and then this guy on the team, he does the malware analysis. And he does this. And you lose all your local files that you should have had these sites. So this is why we don't stick USB keys in our computers. And they're like, OK. And then our worst incident recently, there was this app. And there was an SQL injection in it. And they managed to expel trade a whole bunch of our sensitive data. We had to report ourselves to the privacy commissioner. We ended up having to, because they attacked the SQL server itself, we ended up having to send that server away for analysis. He's like, we had to do this and that. And we ended up spending all these weeks of overtime on it. And he's like, it ended up costing over half a million dollars. And they're like, oh my gosh. And he's like, yeah, we could hire five engineers for that. And they're like, oh my gosh. Wow, what a giant screw up. And he's like, that was your app. That was an app that Tanya asked in writing. M, she came up and asked you personally if she could test it. And you said, no, she has been bugging you for six months. And you have not let her test a single one of your apps. Tanya can't do this job by herself. She needs you. She needs your help so bad. She keeps asking for it. And you told her to f off, dude, that's rude. We need you guys. We can't do it without you. Please, please, please, please help us. Let us test stuff. Let us tell you when things are wrong. Work with us, please. And the manager was like, oh my god, I'm so sorry. I had no idea. And he's like, dude, we spend so much on appsack. Like we have her full time. That costs money. But he's like, there's the tools she has to buy. There's the time it takes. There's when there's an incident happens. It's a mess. Like, we can't do this without you. You guys are so much more important than you realize as this piece of the puzzle. And he's like, I need you to let her test. And I need you to fix things if she says they're serious. Please. And the guy said yes. And then everyone chatted a lot. And then when everyone walked out, the manager that had been so unfriendly with me, he came up to me and he put his hand on my shoulder. And he's like, Tonya, I had no idea how serious this was. I'm sorry. This will never happen again on my watch. We are going to be number one. You tell us everything we're going to fix. Everything our apps are going to be bulletproof. This is over. And he did it. Like he would fix all the things. He had them open up their old apps that weren't even on my list. And he had them scanning it with zap and fixing things. And like his team, like the next luncheon learned I had, they were all sitting there right at the front eating the bagels. Because I bribe people with carbs. And like all of them were there. Like the whole team right at the front. We're ready, Tonya. And I was just like, oh my gosh, this is so amazing. And like I thought by hiding, it sounds dumb and retro-respect. But I was like, if we show them we've made mistakes, they're not going to trust us anymore. They're going to think we're stupid and we're bad at our jobs. We can't let them know we're having lots of incidents all the time. They'll think we're failures. But in fact, that made sympathy and empathy. And then it was like a completely different workplace then. Thank you, Tiptonia Janko. For coming on the show and sharing these stories with us, she's written two books. Alice and Bob learned application security. And Alice and Bob learned secure coding. She also has a newsletter. And we'd love it if you joined. You can find the newsletter at newsletter.shehaxpurple.ca. It's totally free, but it's crammed full of great, helpful information on how to make your apps more secure. And you know what your loved ones would love most. A Dark Net Diased T-shirt. And if they don't want something like that, then you tell them to get new one. And by the way, these shirts don't all say Dark Net Diaries on them. Most of them are just really cool designs that I came up with. You have to check it out. Go to shop.darknetdyries.com. The show is created by me, the spaghetti coder, Jack Recyder. Our editor is the copypasta coder, Tristan Ledger, mixing done by proximity sound in our intro music is by the mysterious break master cylinder. One day I hope to change the world. But I don't want my maxus to the source code. This is Dark Net Diaries.