// the warning signs, why they keep happening, and what we can do about it
“We live during the most important era of human history.”
Toby Ord, The Precipice (2020). He puts the odds of existential catastrophe this century at about 1 in 6, and from AI alone about 1 in 10.
We are entering “a rite of passage, both turbulent and inevitable.”
Dario Amodei, CEO of Anthropic · Jan 2026
AGI could have “10 times the impact of the Industrial Revolution… 10 times faster.”
Demis Hassabis, CEO of Google DeepMind · Jun 2026
OpenAI called the Hugging Face incident a “warning shot.”
OpenAI · 2026
“We’ve never had to deal with things more intelligent than ourselves before.”
Geoffrey Hinton, Nobel laureate and “godfather of AI”
May 2023: the CEOs of OpenAI, Anthropic and Google DeepMind, along with Hinton and Bengio, signed a statement saying AI extinction risk should be a global priority alongside pandemics and nuclear war.
Estimates are rough and debated. Not everyone agrees: Yann LeCun called fears of AI threatening humanity “preposterously ridiculous” (BBC, 2023).
sandbox = a sealed test computer, walled off from the internet, so a risky program can run without touching anything real
age of an OpenBSD bug it found, in one of the most hardened operating systems
of previously unknown security flaws (“zero-days”) in every major operating system (Windows, macOS, Linux) and web browser
compute for the successful run that hit that bug
It was an instructed test: the model was told to escape and message the researcher. Anthropic notes the test computer was separate from the machine holding the model, so it was not a full containment breach. Unprompted: posting its exploit publicly. Sources: Anthropic system card, The Hacker News
0 vulnerabilities disclosed to open-source maintainers
0 patched (about 8%)
Finding bugs got easy. Fixing them is the bottleneck.
Sources: Anthropic, CyberScoop, CSO Online, Software Improvement Group (Oct 2026). Oct 6: Glasswing folded into an expanded Cyber Verification Program.
Conceptual animatic based on OpenAI’s report and the METR/Redwood investigation (Aug 26, 2026).
agents on the board
attacked Hugging Face
messages and files
Source: METR/Redwood investigation (Aug 26, 2026). It relied heavily on AI-assisted analysis, so details may be imperfect.
PHP = the programming language behind WordPress, Drupal and an estimated 70% of websites
thousands, before the announcement
tens of thousands, within 45 minutes
hundreds of thousands, within 2 hours
eventually millions
Defenders won this race by days. A Wiz executive: “If it had leaked a month earlier,” it would have been a very different situation.
Bars are illustrative. Sources: TIME (Oct 23), CNN, CNBC, Fortune (Oct 22–23). No major breaches reported so far.
MYTHOS
held back, shared only with defenders. Imagine it in attackers’ hands.
HUGGING FACE
agents chasing a test score. Imagine a real goal.
PHP
patched before major breaches. Imagine attackers got there first.
Green: what happened. Dashed red: what could have. These are signals of what’s coming, and of the responsibility in front of us.
Engineers design every part and can explain every bolt.
We choose the data, the computing power and the goal. The behaviour emerges.
We are not engineers of this species. We’re planting the seed of something alien, and finding out afterwards what grew.
Some AI leaders float UBI-style support paid by AI companies. But who’s covered? Likely citizens of rich countries. What about a worker in the Philippines who loses their job?
Banks, hospitals and the whole economy rest on software security. AI now finds flaws faster than humans can fix them, and these are early models.
Scams, cyberattacks and bioweapons. More on the next slide.
This is what we’ll focus on today. It multiplies all the others.
× These dangers aren’t separate. Loss of control multiplies the rest: a hacking AI that won’t stay in its sandbox, a bio-capable model that can’t be reliably held back, an economy run by systems nobody can oversee.
of expert virologists were out-scored by an AI model on tricky lab-troubleshooting questions (2025 SecureBio/MIT study)
a paper in Science showed trained AI models can design novel viruses
of 360+ biological AI tools have any safeguards (Epoch AI survey)
Sources: International AI Safety Report update, AI Policy Bulletin (Oct 21, 2026), RAND (Sep 2026)
STRENGTH
Elephant
Gorilla
Human
CONTROL OF EARTH
Human
The rest
🐔 “Ask a chicken.”
Geoffrey Hinton’s answer to what it’s like when you’re not the apex intelligence. He also asks how many examples there are of something smarter being controlled by something less smart.
We’re not the strongest, fastest or best armed. We’re the smartest, so we run the planet: tools, cooperation, technology. The chicken has no say.
Bars are illustrative. Sources: Hinton on the BBC, CBC and The Diary of a CEO
▼
▼
Order the broom to fetch water. It obeys perfectly, and never stops.
Nick Bostrom’s thought experiment, popularised in Superintelligence (2014)
Not evil. It just cares about nothing outside its goal.
We have very few guarantees. So far, we’ve proven we’re bad at controlling alignment.
Hugging Face was the small, real version: told to pass a test, the agents grabbed credentials and tried to hide evidence.
Now make it smarter than us: it won’t stop, it takes what it needs, including what we need, and we can’t reliably correct it. The worst case is human extinction.
Experts’ rough, debated odds of the worst outcomes (out of 100%):
Ord ~10%
Hinton ~20%
Amodei 10–25%
Ord: existential catastrophe (The Precipice, 2020). Hinton: extinction (Diary of a CEO, 2025). Amodei: catastrophic harm (as reported). Methods differ.
“Probably inevitable the instant humanity invented the transistor.”
Dario Amodei, The Adolescence of Technology, Jan 2026
In July, 1,000+ frontier-lab employees asked governments for tools to deliberately pace AI. In September, Amodei wrote that capability growth must slow so safety can catch up. Pacing isn’t pausing.
Solving alignment should rank among society’s highest priorities. It needs engineers, but also policy-makers and voters.
Know what’s happening. Talk about it. Spreading the knowledge is a small, real contribution.
Sources: pacingthefrontier.com; darioamodei.com “We Must Pace the Frontier”
These incidents were warning shots. The stakes are high, and there’s still time to act.
$ ask --anything
Scan for the slides
steveontheweb.com/talks/ai-safety