TLDR: the models are getting good. the future by default is bad. AI safety is bottlenecked on good people. There are many ways for you to become good and to still have an impact if you can’t be. I beg you to seriously try. I believe in you.
Looked around recently? The models are getting very, very good. The agents that chain zero days today will create the mirror life pandemic of tomorrow.
Capabilities keep growing: log-log plots keep bending upwards and random startups are selling tokens from models that will generate instructions on how to make sarin nerve gas to make a quick buck. The technical and policy situation is dire, and everything points to an EXTREMELY WEIRD future.1
If you know all this, but still spend your time pursuing good grades, a promotion, a PhD or clocking in 3 hours at your remote B2B SaaS job Claude does for you or just generally doing anything other than helping things go well, why?
You may have an image that there’s Competent Adults everywhere. This is not true. I don’t know why this is not true. It just is. Ask any org, they are all talent constrained and lacking in people who take things seriously.
There will be money to pay you. Polymarket thinks there’s a 95% chance Anthropic IPOs by the end of the year. That’s 2 billion extra dollars landing into the lap of grantmakers by December.
There are people to train you. People will literally pay you to try to get good. There’s a whole AI safety early carreer industrial complex to help you! There are so many fellowships. Please try, make an effort, I believe in you. People will literally pay you to try. Even if you don’t qualify, there’s SO MANY OTHER THINGS TO BE DONE.
We need policy people. We need economists to figure out what anything looks like after ASI. We need someone to convince your parents. We need people to talk to people that talk to people that talk to Xi Jinping. We need someone to call your senator. We need someone to hand out flyers. We need book tours. We need people that are good at making fucking tiktoks.
The american people do not know what a huggingface is. And any sensible projection of the future says they won’t find out in time
The status quo is you anxiously spectating a 100 meter sprint where no one knows what happens when someone crosses that finish line. Inaction will not save you from the singularity’s wrath.
The ball is being dropped everywhere, I don’t even know where to begin. Here’s like twenty things that we needed done yesterday that you can help with:
Understanding how language models structure literally anything internally2
Making sure no country uses AI to create literal permanent sci-fi 1984
Calling your politician about AI safety
Trying to make us not die in a pandemic
Is there anyone in bio making sure that when the biolabs get an order to synthesize some novel protein that they check first if it will kill all of us.
Monitoring what the hell agents on the internet are even doing.
Working on compute verification
Counting the number of datacenters that exist, seriously!
Creating ways for labs to ensure safety/science researchers can get access to models without them being able to break safeguards
Making sure no weights get stolen
Calling your representative to say “hey there’s a guy selling 5$/1M context window access to a sarin-nerve-gras-instruction-generator and if you don’t talk about how this should be illegal I will not vote for you”3
Drafting policy proposals
READING TRANSCRIPTS
Writing/creating anything that reaches normal people to inform them about the problem
Writing/creating anything that reaches weird people to inform them about the problem
Starting an AI safety organization at your local university4
Setting up whistleblower lines for lab employees5
Tracking Chinese model capabilities, what can a nation state level actor do? What can a terrorist organization do?6
Figuring out if the things are sentient/conscious
Drafting proposals for international red lines.
Figuring out if models are secretly scheming against us.
Tons of random philosophy.
Sure, there are orgs for most of these, but guess what? They. Need. Good. People. And they can’t find them. All the above could have 10x people working on them.
You can become such a person. There will be money. There will be demand. There is an industry, nay, a HUMAN RACE that NEEDS you. We will need MILLIONS of people in the field.7
I invite you to do the following. Grab a notebook and a pen, go sit somewhere and write down all the reasons for you to get and not get into AI safety. What are some concrete actions you could take to make the world better. Actually do it. Close your laptop.
If you think you’re just bad and hopeless, how sure are you? Because I took six years to finish my bachelor’s degree and was only ever good at League Of Legends. A little ambition goes a long way. I believe in you.
I know it’s hard. I know it’s scary. But inaction puts everything at risk too .
Things are about to get serious; We will not get a second chance.
Hand out flyers in front of schools. Please. Anything.8
We need you.
The “line on graph” people keep predicting capabilities better than anything a quant could muster. Extraordinary evidence is required to claim the line won’t stop growing. This evidence is absent. Exponential capabilities is the default trajectory.
Humanity has no fucking idea what it’s doing. Some kid is just hosting a jailbroken GLM 5.3 on an API where he sells it to you for 5 bucks/million context window for whatever you want to do. This model can autonomously hack companies, by the way.
It’s cartoonish. All of this is currently legal. The current trajectory points towads this being legal for tomorrows models. Models that could help an idiot make a pandemic that kills literally everyone. Models that could hack banks. Politicians don’t understand the technology. Voters aren’t paying attention
linear representation hypothesis is clearly incomplete
seriously, WTF!!!
this one is mostly just easy, I don’t know how effective it is
be careful with this one
Be extremely careful with this one
If timelines are so short this statement is true, I seriously worry
Please no acts of violence. They are immoral, illegal and don’t work. This is not a copout answer because I’m self censoring. I am serious: if you kill people you will literally just make everything worse.
I have mixed feelings about this.
1. EA has a history of making strong exhortations of being talent constrained, but when it comes time to make job offers, it's competitive and not everyone who wants a job can have one. Organizations are incentivized to encourage everyone to apply because they benefit from a larger pool of applicants -- and they're rationally trying to combat imposter syndrome, and shift the supply curve for their labor input. But a pivot is still a bad individual outcome for the people who don't end up with better jobs. Targeted outreach to specific high-impact personal connections is likely more effective than mass-messaging. Also, intrinsic motivation is important so people don't feel led on or bitter if things don't work out.
2. The Mythical Man-Month still applies in 2026.
3. With Fable capabilities, the optimal org size is smaller, and people are less of a scaling constraint.
edit: please see my reply to Vals below for more nuance. I totally support encouraging people to go into AI safety.
Hmm I've been doing ai safety research since mid 2024 and im still not paid for it, nor accepted to MATS, so I don't think its that easy. I've only gotten compute grants and travel reimbursements, even if mentors at some programs have used my research. And I don't think my resume is bad, I've gotten final interviews at 2 AI capability-ish companies. (I wouldn't accept those jobs nowadays). Asking for grants is complicated too because they are about specific projects, and working many months on the same idea is mostly pointless, as these get quickly into dead ends.