OpenAI’s autonomous agent hack.

Happy Wednesday. I’m Isaac Saul and today I’m headed down to Philadelphia to pack up the Tangle office with Associate Editor Audrey Moorehead and begin moving my stuff to the North Jersey studio I’m building near my house. Tangle HQ has now gone from my bedroom in Bushwick to a shared office space in Bushwick to an apartment in South Philly to an office space in South Philly and now to a newly built studio in North Jersey. Every time I pack up and move, I’m flooded with memories of the time I’ve spent in a space, and this Philly office bore witness to some of our biggest moments — including the historic 2024 election. I’ll miss it, and miss Philly, and miss sharing an office with Audrey every day. But the future we are building is bright. 

If you’re reading the news these days, though, the future may not seem so sunny. Today we’re jumping into the OpenAI hack. Plus, a look back at the founding of NASA. It’s a 14-minute read.

Too busy to read? Push play instead!

Listen to the newsletter every Monday – Friday here.

Quick hits.

1. U.S. Central Command said that Iran launched an “attempted surprise attack” against U.S. forces in the Middle East, but all missiles were intercepted. The U.S. and Saudi Arabia also conducted joint strikes against “Iran-aligned terrorists” in Iraq. (The latest)
2. Kentucky Gov. Andy Beshear (D) sent the office of Sen. Mitch McConnell (R-KY) a letter requesting that the senator “directly and verbally address the people of Kentucky and provide proof of your capacity to serve, or resign.” (The letter)
3. The Senate voted 86–12 to advance a major sanctions package against Russia that was spearheaded by the late Sen. Lindsey Graham (R-SC). (The sanctions)
4. The Senate voted 51–47 to confirm Jay Clayton as director of national intelligence. (The confirmation)
5. Japan experienced a 7.1-magnitude earthquake, which killed at least three people. Rescue efforts are ongoing. (The quake)

Today’s topic.

The AI hack. On Friday, July 24, Reuters reported that an OpenAI autonomous artificial intelligence (AI) agent broke out of a contained environment and hacked a popular AI development website. The hack began on July 11 and lasted until July 13, when it was detected by an AI at the targeted company, Hugging Face. OpenAI remained unaware of the incident for days, and the two companies did not communicate directly until July 20, well after Hugging Face had alerted the FBI of an intrusion. 

Zoom in: OpenAI said the incident occurred while it was testing two of its models — its latest publicly available version, GPT–5.6 Sol, and an unreleased model — on their offensive hacking capabilities. Both models had guardrails that protected against high-risk behavior turned off during the test and were intended to be limited to a “sandbox” environment with tightly controlled access to the broader internet. 

However, the models were reportedly able to exploit a “zero-day vulnerability” (an unknown security flaw) in third-party software designed to allow them to install packages to gain internet access. From there, the AI agents inferred that Hugging Face may have information to help them pass their assessment and used stolen credentials to hack into the site, stealing information to allow them to pass their tests.

On July 21, OpenAI released a statement saying it is working with the third-party software to resolve the issue and collaborating with Hugging Face to build stronger protections against AI hacking. Clément Delangue, the chief executive of Hugging Face, said he had met with OpenAI and requested that it release detailed logs of the incident and contribute $100 million toward computing power to help develop cybersecurity networks. “The first autonomous agent cyberattack is an unprecedented event,” Delangue said. “It deserves an unprecedented response!”

Tech experts were divided in response to the scale and ramifications of the hack, with some stressing that AI agents are more capable than ever. “Today is the most important day in the history of information security thus far,” said Sean Cassidy, chief information security officer at fintech solutions firm Plaid. “For the first time ever, an AI model escaped containment and hacked a real company’s real production infrastructure. This was unintentional and non-malicious, but that doesn’t matter.” 

Others blamed existing cybersecurity vulnerabilities for the hack. “This should not have happened,” veteran security engineer and researcher Niels Provos said. “I wish the frontier labs spent as much time on teaching their models to write secure infrastructure as they are spending on them exploiting vulnerabilities.”

Below, we’ll get into what the right, left, and tech writers are saying about the incident. Then, Executive Editor Isaac Saul gives his take.

In the Washington Examiner, Sam Korkus said the “hack shows why we shouldn’t trust AI.”

“The issue is not that a rogue AI model was used to attack an American company. Instead, the issue is that the AI was doing what it was told as efficiently as possible — and that led it to hack into a database it identified as a suitable target,” Korkus wrote. “AI is inevitable. It is becoming more integrated into our everyday lives, whether we want it to be or not. The question going forward is whether Americans want the government or AI companies to control it. 

“If the government imposes excessive red tape, companies will be less likely to innovate. If AI companies are allowed to regulate themselves, there may be nothing stopping them from exploiting as much public and private data as possible for their own benefit,” Korkus said. “The only way forward is to adopt a prudent yet skeptical approach to AI, the people who use it, and its architects. After all, behind every AI model are fallible human beings.”

In Hot Air, John Sexton wrote “the latest OpenAI model hacked another company without being asked.”

“What this AI model did is pretty impressive but also a bit worrisome,” Sexton wrote. “So the spin here is that everything is fine and ‘We are strengthening the containment, monitoring, access controls, and evaluation practices used during model development.’ That sounds good. But the conclusion is that, ‘The incident also makes clear that advanced models can discover and exploit novel attack paths in real-world systems without source-code access.’

“And this model wasn’t even told to do any of this. It just seems to have known this would be an easier way to get answers. What could it do if directed to steal some information from a particular government or company? If it was told to cover its own tracks, could it do that?” Sexton asked. “There are two things to worry about here. One, that our own tools will be used to hack their way into information they aren’t supposed to have. Two, that China is developing the same tools about six months behind us and will have no compunction about using them in this way. Either way, the potential for a lot of disruptions seems to already be on the horizon.”

In The Atlantic, Matteo Wong called the incident “a startling glimpse at AI’s ruthless efficiency.”

“This unwanted behavior is a predictable and alarming result of how the entire AI industry is developing its models. The past year’s advances in AI coding and agents… have been the result of an overriding emphasis in AI training through a process called ‘reinforcement learning,’” Wong wrote. “This involves giving AI models lots of hard problems — math proofs, coding challenges, what have you — and then providing positive feedback for correct answers and negative feedback for incorrect ones. In many reinforcement-learning paradigms, the ultimate aim is just for the AI to arrive at the solution — it doesn’t matter how it does so.” 

“To be clear, this sort of AI ‘reward hacking’ is a known problem that tech companies are putting lots of effort into addressing. But these incidents keep cropping up regardless; if anything, research suggests that they may become more common,” Wong said. “Meanwhile, OpenAI, Anthropic, and Google DeepMind are under tremendous economic pressure to make their models more and more capable, which means that the bloody-minded reinforcement learning is all but certain to accelerate.”

In The Guardian, John Thickstun said “be skeptical of OpenAI’s rogue hacker agent story.”

“The rogue agent story is a page out of the media campaign that OpenAI has been running since it announced GPT-2 in 2019. OpenAI remains hungry for ever larger investments, and the company increasingly seeks privileged regulatory status as defense against competition,” Thickstun wrote. “AI is so powerful that investors should buy OpenAI, even at a trillion-dollar valuation; AI is so dangerous that only trusted actors like OpenAI should be permitted to possess and operate this technology. Step back from these doomsday warnings and consider who might benefit from them.”

“I urge readers to think critically when they read press releases like OpenAI’s rogue agent story, and avoid the manipulated reactions these stories are designed to elicit,” Thickstun said. “AI is becoming excellent at identifying security vulnerabilities, and it will become even better over time. These capabilities can be used to break into systems, but they can also be used to harden systems against attacks. If attackers and defenders have access to equally powerful AI, I see no reason to believe that cyber systems will become less secure over time. If anything, I expect them to become more secure, because AI is cheap and scalable compared with human cybersecurity analysis.”

In his Substack, Gary Marcus said “people aren’t wrong to be concerned.”

“One never knows exactly how seriously to take these things. This was a training exercise, not a real-life incident. The actual system would have guardrails [which in the blog they call ‘production classifiers’] that were disabled here, and those guardrails may have prevented this,” Marcus wrote. “That said, this shows that Anthropic’s Mythos is no fluke; the pressure on cybersecurity given these models is serious… On the small comfort side, the current incident was NOT an attempt where system built a goal for itself or developed a motive; the system was following instructions, but not setting high level goals.”

“On the less comforting side, OpenAI’s ‘production classifiers’ are likely to be permeable, just like all guardrails anybody has built to date,” Marcus said. “OpenAI’s zero-day exploit hack of HuggingFace *should* be a wake up call. Although there are lots of caveats around what happened, we are just going to see more and more of the same. We have no guarantees that such incidents can be prevented, and no idea how serious things might get. We should either (a) slow down, or (b) pause until we get our security/AI safety act together… In my opinion, the only way that the industry might actually slow down, though, is if we clearly and unambiguously hold the companies liable for the harms that they cause.”

In Modern CISO, Nathan Hamiel wrote “this is more hysterics than reality.”

“As everyone in cybersecurity loses their minds over the OpenAI/Hugging Face incident, it’s important to take a step back and put things in perspective,” Hamiel said. “OpenAI’s write-up reads more like a marketing document promoting a feature than an incident summary, while the quote from HuggingFace sounds more like someone accepting an award than someone who just got hacked. It’s a bit surreal. I’m not claiming this was a stunt. I’m pointing out that the lack of detail and the way it was presented opens the door to skepticism and speculation. After all, OpenAI is hemorrhaging money and is facing steep competition.”

“This incident would be a lot more interesting to analyze if we had more details, but as it stands, we know nothing about the environment, context, or scenarios,” Hamiel wrote. “In short, no, this isn’t the end of cybersecurity, nor do attackers hold all of the cards. The reality is far more nuanced. Yes, AI models are gaining capabilities in cybersecurity, specifically when paired with an appropriate harness. Security people should explore these capabilities and incorporate them into their workflows. The world is far more complex than we typically give it credit for, and there are a whole host of factors that can confound an attack.”

My take.

Isaac Saul

Executive Editor

 

• The more we learn about this incident, the less frightening it appears.
• The media has done a poor job communicating what actually happened.
• Well founded concerns still exist, but all of this makes me increasingly skeptical of AI hysteria.

Reminder: This is where we share a personal opinion. If you have feedback, criticism or compliments, leave a comment or reply to this email.

How many times are we going to do this? 

As John Thickstun recounted under “What the left is saying,” our tech overlords have been running this playbook for seven years now. In 2019, OpenAI announced a language model called GPT-2 that it said was too risky to release. That model was a precursor to the most basic LLMs on the market now, yet OpenAI’s statement created enough hype to help net the company a billion dollars in investments.

In April of this year, we covered Claude’s “Mythos Preview,” a new AI model the company said was too dangerous to be shared with the public. Instead, the company decided it would hand the model over to 50 organizations in an initiative titled “Project Glasswing” — a name straight out of a spy thriller. The goal was to give the public and private sector just enough time to build up protections against all the different holes in the digital ecosystem this model could exploit. And the impact was immediate: Columnist after columnist began speculating about this world-changing model’s capabilities without even seeing what it was or what it could do. The New York Times’s Thomas Friedman went as far as hallucinating a future where a group of teenagers used this model to take down power grids before dinner time. 

Here is what I wrote:

I really do think something about all this is obvious PR, and I think these artificial intelligence companies are extremely good at it. Not a little good at it — but extremely. After all, they have to pull off an incredible high-wire act: They’re creating a product that they’re telling the public is so good it’s going to steal your jobs, hand over cybercrime tools to bad actors, and potentially destroy all of humanity. But, also, you should be really excited about this stuff and invest in their companies. “A new product so good it’s dangerous to release to the public” feels like a magnum opus of publicity.

Trying to check my blindspots, I spoke to tech journalist Casey Newton, who argued that the reports were not hype and that Anthropic would not be doing this if they thought the model was safe for the public. I was skeptical, so I set some guardrails for myself: I’d wait and see what became of Mythos and how Project Glasswing evolved. Everyone else seems to have moved on with the presumption Mythos was what Anthropic said it was, but for the last three months I’ve been watching. And the results are… peculiar.

As security expert Bruce Schneier recently noted, Anthropic has since published a single status report of the software vulnerabilities it’s found. Software having vulnerabilities isn’t exactly news — companies are hacked every day. But almost none of those vulnerabilities have been patched, and it’s unclear which ones are even dangerous. In the meantime, a Chinese firm has unveiled its own AI model that many reviewers say is as capable as Mythos. A few deep breaths and more information have seemed to produce a new conclusion: Mythos is simply not that different from existing models. 

And in the last three months, nothing has really happened. I don’t mean to simplify all this, but really: There have been zero major cybersecurity breaches related to these rapidly evolving AI systems (until now, but more on that in a second). It reminds me of a piece of writing by Clifford Sosin on how AI might be teaching us that intelligence really “ain’t all that.” Having encyclopedic knowledge and reasoning skills don’t actually make you (or an AI model) an earth-shattering entity — most real-world problems require novel thinking, so they can’t be solved by applying what we already know. This is why the “explosion” we were all waiting for with AI hasn’t come and why the major changes have all happened in rigidly defined spaces like software development. Progress will be incremental and, yes, often unexpected — but maybe, after all this time, it’s time to say the world-ending shoe isn’t going to drop. 

Here is my writing, again, from April: 

For two years now, the AI industry has been saying that autonomous agents handling complex work with minimal supervision were about to upend the entire economy. Any day now. Two years after Claude Code was supposed to make software engineers irrelevant, the company is facing criticism that Claude’s abilities are actually degrading, perhaps because it’s hitting compute limits. Even the Mythos hype, less than a week in, is already being questioned — and the fine print in Anthropic’s own paper suggests it “can’t state with certainty” (yet) how serious some of the vulnerabilities Mythos found actually are, so we’re left waiting for more information about what, exactly, the model is actually capable of.

So, what do I make of this latest story? I think it sounds pretty familiar. A major tech company releases a story about how dangerous and scary its yet-to-be-public model is. This time, the AI “escaped” testing and “autonomously hacked” a separate AI company. Alright then. That does sound pretty scary! 

The reality, however, was much less nightmarish. As John Herrman wrote in New York Magazine, the mainstream press — not big tech — was responsible for the narrative of an “escape” and OpenAI losing control of its model. Truthfully, when I read OpenAI’s press release, which describes how “the models identified and exploited a zero-day vulnerability… in the package registry cache proxy… [and] performed a series of privilege escalation and lateral movement actions in our research testing environment until the models reached a node with Internet access,” I have a hard time telling you how that’s different from an “escape.” But a lot of smart people do understand it, and their reactions don’t seem all too concerned.

Heidy Khlaaf, a former safety evaluator at OpenAI who is now the chief AI scientist at the AI Now Institute, said on X the media’s coverage of this incident was abysmal. “Use of the terms ‘rogue’/’loss of human control’ lead to groupthink as people lack critical skills to understand the difference between ‘autonomy’ and faulty reward functions in AI on a task it was directed and given access to do,” she wrote. 

As Herrman noted, OpenAI’s goal here seems to be to shift agency from themselves, the company building the tech and the environment to test it in, to the software itself for breaking out of said environment to harm a different AI company. “Sorry, it wasn't us, it was this dangerously effective AI tool we’re building! Also, it’s for sale and you can invest in us!” In reality, the engineers at OpenAI were testing its models’ hacking ability, and they failed to create an environment safe enough for that test. This is less “David Blaine escaping from handcuffs underwater” than it is “telling your kid to stay in the car but leaving one of the doors unlocked.”

I’m not suggesting these updates aren’t meaningful in some ways. I am, genuinely, worried that engineers are building AI models that outsmart or surprise them. I’m also concerned that huge corporations with access to these models are, right now, processing reams of data to build future products for weapons, surveillance, cybersecurity, and social media in ways we can’t yet fathom. 

At the same time, though, these cycles of over-hype and minimal impact have us barreling toward a boy-who-cried-wolf situation, where it’s going to be hard to tell when a real existential threat has arrived because we’ve been told so many times it’s already here (it could be, for instance, that this story is the real threat, but I doubt it). The AI alarmists seem to be paralyzed by panic every time these companies drop a new press release — and then the companies lose control of the narrative much the way they claim to be losing control of their own models. That doesn’t mean we need the government to require companies like Anthropic or OpenAI to build some kind of “kill switch.” But I do think we need a little less hubris from all the brilliant people working at companies like Anthropic and OpenAI and a little more transparency. Here, OpenAI is quietly turning an event where it built malware that attacked another company into a partnership with that company. They’ll be sharing details on the vulnerabilities, incidents, and findings very soon — right when their investigation is complete. They promise. 

It’s all just very strange, and I’m skeptical of everyone involved. I don’t trust that OpenAI is being as forthcoming as they should be. I don’t trust the press is reporting on this accurately. And I don’t think this is nearly as big a deal as the public seems to believe. 

Take the survey: How concerned are you about AI development? Let us know.

Disagree? That’s okay. Our opinion is just one of many. Write in and let us know why, and we’ll consider publishing your feedback.

This day in history.

In the summer of 1955, the U.S. and the Soviet Union (U.S.S.R.) pledged to launch artificial satellites in orbit around the Earth. Ostensibly part of the global scientific community’s efforts to better understand the natural world, the pledge also unofficially launched one of the defining conflicts of the Cold War: the Space Race.

President Dwight D. Eisenhower publicly directed U.S. satellite resources toward the civilian-led Vanguard project, while the Soviet Union worked in secrecy and planned to beat the Americans into space. On October 4, 1957, the U.S.S.R. announced that it had officially launched the world’s first artificial satellite, Sputnik, and in November it launched the larger Sputnik 2, carrying a dog, Laika.

The U.S.S.R.’s progress shocked the United States, and President Eisenhower reacted by forming the President’s Science Advisory Committee (PSAC) to strategize a response. Meanwhile, the failure of the Vanguard project’s public testing underscored that greater resources would be required for the U.S. to reach space. In February 1958, the committee recommended the creation of a new civilian space agency. 

Throughout 1958, Eisenhower lobbied Congress to create a new agency, and on July 29, he signed the National Aeronautics and Space Act, officially creating the National Aeronautics and Space Administration (NASA). Despite early setbacks, the U.S. would go on to win the Space Race just 11 years later, when NASA sent three astronauts to the moon.

The extras.

• One year ago today we covered the U.S.–EU trade deal.
• The most clicked link in our last regular newsletter was the report on Sen. Rand Paul (R-KY) releasing Anthony Fauci’s diary. 
• Nothing to do with politics: Try your hand as an air-traffic controller.
• Our last survey: 1,829 readers responded to our survey on the prospective civil nuclear agreement between the United States and Saudi Arabia, with 32% saying they strongly oppose the deal. “The Saudis are not trustworthy,” one respondent said. “I’d be more likely to support it if any other administration were in charge,” said another.

Have a nice day.

40 years ago, commercial whaling in Brazilian waters had driven the local humpback whale population down to roughly 2,000. But after the International Whaling Commission paused all commercial whaling in 1985, the population began to rebound. Today, the fruits of that decision are increasingly apparent, with the humpback whale population reaching approximately 35,000. Perhaps best of all, the whales are increasingly visible off Rio de Janeiro’s coast, a rare sight just a few decades ago. “It shows that the whales are making a recovery, are healthy and thriving, and hopefully they’ll continue to do so,” Enrico Marcovaldi, co-founder of the Humpback Whale Project, said. The Associated Press has the story.