Controlling AI
Is it already too late—maybe–maybe not —– But don’t dismiss the warnings as hype.
What’s important is:
“AI could become extremely dangerous”
There are basically three different claims that often get mashed together:
- AI will cause serious disruption —- Very plausible.
- AI could cause a civilization-scale catastrophe —- Plausible enough that responsible people should prepare for it.
- AI will inevitably wipe out humanity. We simply don’t know that?
The part to take seriously
The frightening thing isn’t that AI suddenly “becomes evil.”
It’s that we could eventually create systems that are:
More capable than humans & increasingly autonomous & connected to real-world infrastructure & pursuing objectives we don’t completely understand.
If that combination arrives before we have reliable ways of controlling and aligning such systems, then “we’ll just turn it off” may not be an adequate safety plan.
We’re having this argument before humanity has necessarily crossed the point of no return—–or has it?
We can still:
- slow or pause particular capabilities;
- require independent safety testing before deployment;
- restrict autonomous access to critical infrastructure;
- build international agreements;
- make companies legally accountable for catastrophic negligence;
- invest enormously in alignment and interpretability research;
- keep humans—not autonomous AI systems—in control of nuclear, biological, military and other civilization-critical decisions.
In other words, the warnings themselves are evidence that the problem is being recognized — maybe not evidence that the outcome has already been determined.
There’s actually a strange and important moment happening right now: competing AI executives are publicly saying that the industry may need to slow down, while critics argue that this is years overdue.
So if you’re asking, “Should I be terrified that humanity is doomed?”
If you’re asking, “Should humanity treat this as one of the most consequential technological risks we’ve ever faced?”
Absolutely.
It’s——
We don’t know whether extinction is likely. We know the consequences would be unimaginably large. Therefore, we should act as though preventing it matters enormously.
It’s a situation where being early is exactly what you want, but————–maybe it’s already too late