Signup for the free Tangle Newsletter
Processing your application
Please check your inbox and click the link to confirm your subscription.
There was an error sending the email
The tech fragility is here.
Headquarters of the National Security Agency in Fort Meade, Maryland | National Security Agency

Back in April, Anthropic announced a new AI model called “Mythos” and said it was too dangerous to be released to the public. 

The company’s general message was: We built a product that’s so good at finding security vulnerabilities that we can’t release it publicly right now. Instead, we’ll launch a project (named Glasswing) to give American businesses a head start to shore up their security vulnerabilities before the public gets access to these tools. Anthropic said it expected other models to catch up in the coming months, at which point we’d see a slew of new tech vulnerabilities and cyber intrusions pop up across the world.

I saw this story through a simple lens: Anthropic was like a company selling a security system while telling you it had just identified all these fresh ways a burglar could break into your home. It was fear-mongering. Before, I’d been skeptical of these companies’ claims about their products; now, I was outright scornful. It felt like a marketing ploy from a bunch of people who were getting very good at talking about how powerful and advanced their product was.

I also wrote this:

If, over the next few months, everyone from Amazon to Chase to Meta starts scrambling to prepare for a new security threat, that will confirm to me that this threat is real. If these guys all play with the model and nothing really changes, I’ll assume that the threat was overhyped.

Well, it’s now been a few months. I pride myself on being an open-minded person willing to admit when I’m wrong and change my mind when presented with new evidence. Today, I have to admit: The threat is beginning to look very, very real. So it’s time for me to eat some crow and update my assessment. 

Based on our knowledge right now, the wave of outages, tech issues, and general all-encompassing tech fragility across various sectors is not the result of rogue AI bots. It isn’t cybercriminals, and it’s not Iran or China, either. It appears to be exactly what Anthropic said would happen. A bunch of large corporations, institutions, and tech companies are using rapidly developing AI tools to identify their own security issues. Then they are trying to update, patch, and fix those issues. And in many cases, they are breaking things along the way.

That’s really it. That’s the story: AI-assisted corporations and institutions are in a race against AI-assisted threats, and everyone is moving so fast that our already fragile systems are beginning to crack. 

Get more of the story...
We broke down the outages with just the facts, and then we picked out the best arguments from conservative, progressive, and independent writers in today's free newsletter so you can make up your own mind. Read them here. 

It isn’t your imagination that this is happening more frequently, either. Yes, part of it is that we are just noticing these failures — and this tech fragility — more. But we are also actually experiencing a higher frequency of outages. Rohit Krishnan, a Wharton fellow and author of Building God: Demystifying AI for Decision Makers, put together a helpful chart mapping rates of tech outages from the last 10 years. It shows we are not at all-time highs, but we are in a period of much higher system failures than normal. According to Krishnan, we haven’t seen this volume of disruptions across sectors since 2013. 

All of this has me seriously reconsidering my rather blasé attitude about the rapid development of AI. Among other things, in recent writings, I’ve said that on a scale of 1 to 10, with 1 being the very early stages of AI capabilities and 10 being AI as a fully realized technology, I think we are closer to a 7 than a 3. In other words, these models weren’t going to get much better than they are now. 

That thesis is looking less likely every day. The models and products are continuing to improve rapidly, and we are not getting many signals that they are slowing down. Mythos was just one marker on the highway, but Anthropic's warnings about what models like it would do appear to be accurate. 

I’ve also been skeptical about how dangerous it would be to allow these products to hit the open market. But a collection of events has changed my opinion there, too: I’ve had several conversations with friends, readers, and experts in the space who convinced me I should be more worried than I am. I listened to Bill Gates’s recent interview with Ezra Klein, which genuinely scared me, not least because Gates himself seemed so thoroughly afraid. The Google Threat Intelligence Group then warned this week that the “era of autonomous attacks has arrived,” and we are now up against a “hacker that doesn’t eat, sleep, or get tired, and never stops searching for a way in.” And through all of this, I’ve been watching news story after news story about all the ways major industries seem unprepared for this moment. 

Think about it this way: The FAA called a national ground stop, the NSA had an internal communication system crash, and online banking service issues popped up all in just the last week. Each was (purportedly) caused by attempts to update or patch vulnerable systems. Which has me thinking: If the people trying to be helpful can so easily break the things they are working to improve, what does that say about the odds that actual AI-assisted cyber criminals (or, God forbid, rogue AI agents) will have much trouble bringing all these institutions to their knees?

Given all this, I feel much less confident about our potential to prevent a banking collapse, rogue AI from escaping containment, or government systems from being compromised. All of a sudden, a multi-week grid collapse or the potential for the U.S. military to be kneecapped in a “cyber incident” doesn’t seem so far-fetched. It’s honestly starting to feel a little Pollyannaish to think it won’t happen.

There’s no “but” here. I’m frustrated and a little uneasy to say that I don’t have an optimistic landing. I take solace in people like John Hultquist, the chief analyst at Google Threat Intelligence Group, who said this week, “Things are going to go offline periodically as the world manages a generation of tech debt. It will be a nuisance, but it will be okay.” I hope it’s as simple as that, but I can’t say I share his optimism anymore.

The people leading the AI revolution seem hellbent on pushing forward no matter the cost, despite their constant warnings that they might wipe out all of humanity. The government leaders responsible for regulating away the dangers of these kinds of technologies are busy obsessing over what we call them, or are totally in cahoots with the AI leaders themselves, or are so technically illiterate that any attempt to be helpful is nearly useless. And the rest of us common folk are just along for the ride.

So here’s to hoping the techies, nerds, and their billionaire bosses know what they’re doing. Because it’s starting to feel like the game is afoot. And I can’t say I’m keen to find out who wins.

Staff dissent — Senior Editor Will Kaback: Assuming that most of the recent disruptions are a byproduct of companies and government agencies trying to patch vulnerabilities identified by new AI models, I have a different reaction than Isaac.

He writes, “If the people trying to be helpful can so easily break the things they are working to improve, what does that say about the odds that actual AI-assisted cyber criminals (or, God forbid, rogue AI agents) will have much trouble bringing all these institutions to their knees?” I would be far more alarmed if this work wasn’t being done — that is, it’s relatively reassuring that new vulnerabilities are being found and urgent work is being done to address them. That things are breaking along the way isn’t ideal, obviously, but I think it’s a reasonable price to pay to improve critical systems. I don’t see it as evidence that we’re on the precipice of mass cyberattacks.

I appreciate Isaac’s message about keeping an open mind on new developments, and I want to do the same. If the disruptions and outages become more significant — a major bank going down for multiple days, issues with the electrical grid, another ground stop that persists for more than a few hours — I’ll start to see the trade-offs differently. Just today, the European Commission said it was targeted by an “AI-generated cybercampaign,” and that’s cause for real concern. Similarly, if AI-enabled cyberattacks become more frequent and impactful despite the work being done to prevent them, I think it will mean Isaac’s theory is, distressingly, coming true.

For now, though, I’m viewing the underlying work behind the recent system outages more optimistically.

Moved by the take? Share it!

Member comments

More from Tangle News related to this article

Recently Popular on Tangle News