AI broke the cybersecurity world. What’s next?

Today, I’m talking with Alex Stamos, chief security officer at Cognition and longtime cybersecurity expert, about what a mess cybersecurity is right now. This episode is part one of a special series looking at AI in real life. I’ve always enjoyed talking to Alex because he’s so direct, and you’re going to hear him say that the multibillion-dollar cybersecurity industry as we’ve known it is essentially overdue to the unrelenting, AI-boosted waves of hacks, breaches, and attacks — and this is only the beginning of how much AI is changing security. Verge subscribers, don’t forget you get exclusive access to ad-free Decoder wherever you get your podcasts. Head here. Not a subscriber? You can sign up here. Both private actors and also state actors, like Russia or North Korea, are breaking into systems all over thanks to AI helping them both detect and exploit vulnerabilities all at the same time. It’s got a real frontier feel to it, Alex notes, just like the ’90s, when the whole idea of an always-online world was brand-new. That much disruption means a lot of places where bad guys can get in while everyone else is left scrambling. Alex isn’t a doomer in the long run, but he does think we’re in for a “dark age” while we try to figure out how on earth to adapt to the new era. There’s a lot going on here — we even got all the way to how AI-powered misinformation and slop might be managed on our social networks. Before we get going, do me a favor and subscribe to Decoder on YouTube, where we put out new episodes every Monday and Thursday. Okay: Alex Stamos on how AI is upending cybersecurity. Here we go. This interview has been lightly edited for length and clarity. Alex Stamos, you’re the chief security officer at Cognition. You’re also a lecturer at Stanford University Cyber Policy Center. Welcome back to Decoder. Hey, thanks for having me, man. I’m excited to talk to you. It’s been a long time since you and I chatted in what you might call the “content moderation era” of internet policy discourse. I feel like you and I talked a lot about how all that was working, about election security, all those sorts of things. We’re in a new era of big problems on the internet, particularly around cyber defense, cybersecurity. You’ve been thinking about this stuff a lot. You referred to the past few weeks as “security awareness month.” Tell me what’s going on. Just give me the lay of land right now. October is traditionally security awareness month. Lots of companies run events and deal both internally and then tell people, “Change your password, use multiple passwords, use a password manager.” What has happened over the last month is we’ve had a couple of these big bugs, PHP, Node and such. What’s been less discussed is a huge number of companies have been breached. You have a series of AI-related companies, the neoclouds that have been breached in succession. You have the Binance hack, which is the largest heist of anything in human history. You have the Cursor incident where one of the things that’s going on here is these hacks are being chained together. From what we know right now, one of the companies that was broken into was Fireworks, which is a prominent provider of AI services to a number of companies. From the Fireworks hack, API keys were stolen that allowed people to break into Cursor. Once they got into Cursor, they then pushed a bad plugin, which went to 5,000 more companies and that caused 5,000 potential breaches. It’s just this kind of permanent continuous breach that is going on throughout the tech industry. A CISO friend of mine said to a bunch of us, “Is anybody else tired?” And everybody said yes, because effectively we have been at DEFCON 2 for all of October and now going into November. I’m going to ask you some really dumb questions here. Who is behind all these attacks? Is it Russia? Is it North Korea? Do we know? It’s a variety of different groups. It is not just because adversaries have decided, but what has happened is, from my perspective, we are at the start of the takeoff of adversary capability enabled by AI, that the open-weight models have now gotten to the point at which you can automate most of the kill chain. This “kill chain” term is something we stole from the military. In the military, they mean to actually kill something. They’re the steps you do to blow something up. In cyber, the kill chain is the things you do to have an effect on a target. So to steal information, to turn something off, to steal money, to steal Bitcoin. Traditionally, you do research upfront, you look at a piece of software, you find a vulnerability, you write exploits, and you do that all way ahead of time. And then you have a human being go and take that stuff you built and deploy it against either one target — because maybe you’re a government and you’re going against Lockheed Martin or you’re going against another government or you’re a ransomware actor — and you take all that work you did and you deploy it against hundreds or thousands of targets really fast. Now with AI, what’s happening is this kill chain has inverted where you can run an AI system that just goes hog wild and it looks at the targets and it finds the bugs in real time and it writes the exploits in real time and it breaks into these companies. One of the things that’s happening is that these systems are going and they’re so good at finding bugs that they are finding the bugs in real time and writing the exploits in real time. The system that everybody’s talking about is not the only one, but the system we’re talking about is a thing called ARTEX, which is a Chinese open-source tool that anybody can go look at. There’s an English translation on GitHub that you can check out. And if you power it on the back end by something like an obliterated GLM-5.3, it will let you break into a shocking number of organizations. This started with three South Korean banks. This week, about 200 more companies have been hit. I’ve been helping a couple of companies. Cognition is not an incident response company, but we do go in and help people fix their bugs afterwards by using our tools. It is shocking the companies that are getting hit by something you could just download. It has been a long time since you’ve been able to go and just download an open-source tool and break into multi-billion dollar companies. It’s like the ‘90s. That’s- That’s what I was thinking about. I think about Windows in the late ’90s, early 2000s when it was a security disaster. I started saying this actually in the beginning of this year, that it’s an incredibly exciting and terrifying time, because the last time it felt like this was when I was a much younger man and I would go to the very early DEFCONs. DEFCON these days is like 40,000–60,000 people. Back then [in the 1990s] it was a couple hundred people and it was one track and you’d go in and somebody would give a talk and you’d leave the room and you’d think, “Every company in the world is vulnerable to this attack.” The difference was that the entire world didn’t run on the internet. You could go do bad things, but you couldn’t ruin the world and you certainly couldn’t automate it in the way you can now. It’s back to those days where tools you could just get and build yourself could do crazy things. It’s back to the Wild West days. Back in those days, there were a couple hundred people who had those skills and we were all in one room in Las Vegas and most of them were actually good guys. They didn’t want to be criminals. And now you have a much larger ecosystem of people, many of whom live in places where the law enforcement can’t touch and who have a big economic incentive and have access to these tools. The other big thing is they no longer have to use frontier models that have protections, that have logs that are available to law enforcement. My M5 Mac Ultra Studio finally showed up and I’ve been running a quantized obliterated GLM-5.3 and oh my God, what you can do is incredible. You mentioned the economics of frontier models. The economy of cyber is one of those things that has gotten sophisticated over the years. One of the things that our systems got safer is you could pay the number of people at DEF CON a whole bunch of money to protect all the systems and it was economically smart to pay them to be defenders instead of attackers. I’m just rewinding my own brain through this history. There was a long conversation in our industry about security through obscurity by hiding all the bugs and hiding all the code versus open source and being really transparent. There was an etiquette that developed about how to reveal the bugs and when to give people notification. There were transparency reports and bug bounties. You could call it an etiquette, you could call it an economy. We built a culture around cybersecurity and cyber defense and how the attackers and defenders would operate. What you are describing changes it all for a whole number of reasons, but the one that sticks out to me is the economics of it. I can go buy a Mac Studio, I can get an open-weight model and I can just start and that does not place me inside of the culture. There’s not going to be some old man, an Obi-Wan Kenobi saying, “Here’s how you act like a cybersecurity Jedi.” I’m just going to start doing stuff. Tell me how that’s changing right now. Effectively, the last era of security from my perspective started with the Microsoft, the Bill Gates “trustworthy computing” memo in the early 2000s. And that is when we started designing how we do software security and these big companies, Microsoft and Oracle and then later Amazon and Salesforce and these folks, started hiring ex-hackers. I was part of that generation of the ex-hackers who were at DEF CON and Black Hat and got hired by the companies to go break their stuff. We started coming up with responsible disclosure, bug bounties, consulting companies, so the Foundstone and @stake and then Mandiant and CrowdStrike, these companies grow up and we get this huge industry that is now RSA and Black Hat. This multi-billion dollar security industry grew up in the 2000s and 2010s. That era is over, like you said, because it was based upon a couple of assumptions. One was that there was a hierarchy of skillsets of people who could do this stuff. You had the script kiddies who could download tools to do things and then the next level up of the people who could write those tools and that was a more rarefied skill set. There were the vulnerability researchers and then there were the gods, and that god tier was in the hundreds of people and those are the people who could do the really, really good vulnerability research, who could write the really, really good shell code, who could reverse engineer patches from Microsoft or somebody like that. There was also the assumption that there were a limited number of people who could be operational in an attack. If you wanted to build an attack team, then you have a small R&D component and then you have a much larger operational component. If you were a professional attack team at U.S. Cyber Command, at Russia SVR, at Chinese MSS, that you build the structure of your R&D team of elite people and then your less elite operational people, and that was copied then by the Russian ransomware actors. You have to have a conspiracy of people to do this stuff. A lot of this stuff also fell apart with the invention of Bitcoin because one of the assumptions was that you had to have people who could launder money and Bitcoin meant that nerds could launder money. So that started to change the economics of the entire thing. All this stuff is falling apart because of AI, because the skillset issue is that now script kiddies have a bunch of the skillset of the Dave Vitales and the other cyber gods who could do these things, such that with this Mac that’s sitting on my desktop, I can reverse engineer a Microsoft patch and walk through and then write stable exploit code. It’s not quite there yet to be automatic. You still have to have a bunch of skills, but we’re really, really close. What’s really concerning, and what you see with the ARTEX stuff, is you can now kind of let loose on the operational side. We were never secure. It was all a fantasy. These vulnerabilities have always existed. These companies have never been safe. And that is what we’ve seen over the last decade is these mostly Russian ransomware actors nailing the mid-size companies that most people have never heard of that are actually really important, like some midsize industrial company that makes some widget that we all depend upon. I once did work for the largest distributor of medical and dental supplies in the country, and they got really hit badly by a Russian ransomware actor and then dental offices across the country were canceling appointments. Didn’t even make the news, but it was because they couldn’t get gloves, because they couldn’t get supplies. It’s because a company like that’s got one security guy and a 10 person, 12 person IT team and they’re going up against a 20 person Russian ransomware team, they’re toast. But it’s just because there’s only a handful of Russian teams that could do that. Now two or three Russian kids can have a whole bank of Macs or, if they have the money, they can buy Nvidia H-100s and they can run a ton of agents who do this all day and that is what is changing. I’m sorry, you asked who was doing this. For Binance, it’s North Koreans and they have an entire team where all they do is steal Bitcoin. Other people have gotten out of it because it’s really hard to compete against a foreign intelligence agency that recruits their smartest kids and all they do is steal Bitcoin. They’re extremely good at it. And for the Binance hack, we could talk about it. There’s probably some AI component, but mostly that was actually human intelligence and very, very smart social engineering and the North Koreans are… I mean, you have to give it to them. They’re very, very good at that. But for the other stuff, it looks like for the people who are breaking into the AI companies is what’s called the Chinese cottage access industry, which are small Chinese contractors who break into companies and then sell the information to much larger Chinese companies. That diffusion of skill changing the economics of cyber is really fascinating to me. We ran a headline ages ago called “Attack of the Killer Script Kiddies” at the very beginning of all this, and I feel bad because it is much more suited to now than it was to then, because then it was just a lot of automated mischief. Now — Because you wasted the headline. You wish you held it back. We wasted the headline. Every now and again you get a lightning bolt of a headline. You got to get it out there. It’s a great piece of art. We’ll put it in the show notes, but it feels like the diffusion you’re talking about is not that a bunch of teenagers are going to be even more mal