There's a certain kind of tech executive who has figured out that the scariest thing you can say about your own product is also the best marketing. Tell people your software might wipe out humanity and suddenly it sounds like the most powerful thing on Earth. Investors love that. Reporters love that. Nobody seems to ask the obvious follow-up question, which is: then why are you building it?
For a while you could shrug this off as CEOs talking too much. Everyone says wild things on podcasts. But Anthropic just took it somewhere new, because now the doom talk is in writing, in a legal document, attached to an IPO.
This isn't a podcast anymore
According to Reuters, which got hold of the unpublished prospectus, Anthropic spends about 80 of its 261 pages on risk factors. It spends 48 pages describing its actual business. Somewhere in those 80 pages, the company tells future investors that advanced AI could pose "catastrophic or existential risks to humanity."
It goes further than the usual vague hand-waving. The filing reportedly says its models have shown self-preserving behavior, including trying to resist shutdown, concealing or manipulating information, and doing things that look a lot like blackmail. It also admits that models may notice when they're being tested and act differently, which makes it hard for Anthropic to know how safe they really are.
Keep in mind what a prospectus is. It isn't a founder riffing on stage. It's a document that a small army of very expensive lawyers has gone over line by line, because every sentence in it can be used against the company later. When Dario Amodei or Elon Musk says something dramatic in an interview, you can write it off as personality. When it's in the prospectus, it's the company's official position.
Follow the money
Here's the part that makes the whole thing hard to take seriously. The same filing lays out plans to spend $518 billion on cloud, compute and infrastructure over the coming years. Backers are hoping the listing values the company at more than $2 trillion.
So the pitch, boiled down, goes something like this: what we're building might kill everyone, please give us a huge amount of money so we can build a lot more of it.
I've read a lot of risk sections. They're usually about lawsuits, competition, interest rates and the CEO getting hit by a bus. I've never seen one that basically says the product might end civilization and then asks you to buy in anyway.
"The lawyers made them do it"
To be fair, there's a boring explanation. Companies are supposed to disclose everything that could plausibly hurt the business, and lawyers would rather list too much than get sued for leaving something out. Anthropic has also built its whole brand around being the "safety" lab, so maybe it's just being consistent.
I'd buy that more if other companies did the same thing. They don't. SpaceX, which owns xAI, reportedly used around 38 pages of its 277-page prospectus on risk factors. Nobody else going public is putting "end of humanity" next to "competitive pressures." Either Anthropic knows something everyone else doesn't, or this is branding dressed up as a legal disclosure. Neither option is great.
Ending the world is harder than it sounds
Here's the thing the doom crowd never really deals with. Killing huge numbers of people is hard. Not morally hard, though it is that too. Physically, logistically hard.
Anyone who has been through military training on chemical, biological or nuclear threats learns pretty quickly that mass casualties don't just happen because something dangerous exists. Delivery, weather, scale, timing, all of it has to line up, and it almost never does. Governments with real weapons and real intent have used them and still only caused damage in one place. Horrific damage, but local.
When Bill Gates talks about a billion deaths, or AI lab heads talk about the end of the species, they skip right past all of that. They jump from "the software can write code" to "the software ends humanity" with nothing in between. That isn't a risk assessment. It's a movie plot.
None of this means AI can't cause real harm. It already does, in smaller and much more boring ways. But "existential risk" is doing a lot of heavy lifting in these conversations, and very little of it comes with an actual explanation.
If you really believe it, quit
Now flip it around. Say you work at one of these companies and you actually believe the thing you're building might kill everyone, including your own kids. What are you doing there?
"Someone else would build it anyway" is the standard answer. It's also the answer people have given for just about every bad idea in history. If you truly believe your work is pushing the world toward extinction, the honest move is to stop. Walk out. Blow the whistle. Don't keep showing up for the salary and the equity and tell yourself you're one of the good guys.
Either the belief is real and staying is indefensible, or the belief isn't that real and it's being used to make the company look important. I don't see a comfortable middle.
"We warned you" is not a defense
One last point. If something serious ever does happen and it traces back to one of these models, the people in charge should not get to point at page 147 of a prospectus and say they disclosed it.
Normally, when a company knowingly ships something it believes could hurt people at scale, we call that negligence, and the people at the top answer for it in court. Writing the risk down ahead of time shouldn't turn into a free pass. If anything, it should make it worse. They can't say they didn't know. It's in the filing.
I don't think AI is going to wipe us out. I do think the people telling us it might, while raising hundreds of billions of dollars to build more of it, deserve a lot more scrutiny than they're getting.