I have a short Substack post about AI regulation which is itself a teaser for my much longer article in Areo Magazine about AI risk & policy.
When Chat GPT 3.5 was released, I was tribally. But within weeks my emotions switched tribes, even though my actual rational opinions have been more or less consistent. Basically
-
We have almost no real-world understanding of AI-alignment, we need open, visibile experimentation to get it.
-
There really is risk, AI-development needs legal limits,
-
Those limits should be more about rule-of-law than administrative power.
-
The goal is to create and delimit rights to work on AI safely.
But do read the actual articles to unpack that.
That's what I want, but what I'm afraid we'll get (with the backing of the AI-risk community) is a worst-of-both-worlds. Large unaccountable entities (i.e. governments and approved corporations) will develop very powerful Orwellian AIs, while squelching the open development that could (a) help us actually understand how to do AI safety, and (b) use AI in anti-Orwellian tools like personal bullshit detectors.
I understand the argument that crushing open development is Good Actually because every experiment could be the one that goes FOOM. But this Yuddist foomer-doomerism is based on an implausible model of what intelligence is. As I say in Areo (after my editor made it more polite):
Their view of intelligence as monomaniacal goal-seeking leads the rationalists to frame AI alignment as a research problem that could only be solved by figuring out how to programme the right goals into super-smart machines. But in truth, the only way to align the values of super-smart machines to human interests is to tinker with and improve stupider machines.
Any smart problem-solver must choose a course of action from a vast array of possible choices. To make that choice, the intelligence must be guided by pre-intellectual value judgements about which actions are even worth considering. A true-blue paperclip maximiser would be too fascinated by paperclips to win a war against the humans who were unplugging its power cord.
But even if you did believe in foom-doom, centralising development will not help. You are just re-inventing the Wuhan Institute for Virology.
Jump in the discussion.
No email address required.
Notes -
I think you're making two separate arguments here and not distinguishing enough between them.
You think that letting governments build high-powered AI while shutting down others' access to it is a bad idea.
You don't like Eliezer Yudkowsky and those who follow him.
The thing is, these are entirely-separate points. Eliezer Yudkowsky does not want to let governments build high-powered AI. Indeed, his proposal of threatening and if necessary declaring war against governments that try is the direct and inevitable result of taking that idea extremely seriously - if you really want to stop a country from doing something it really wants to do, that usually means war. And he's fine with interpretability and alignment research - he doesn't think it'll work on neural nets, but in and of itself it's not going to destroy the world so if people want to do it (on small, safe models), more power to them.
So it's kind of weird that you set up Yudkowsky as your bugbear, but then mostly argue against something completely different from the "Yuddist" position.
As an aside, I think you're wrong to say that pursuing Yudkowskian Jihad will necessarily end in WWIII with China. The PRC has actually started requiring interpretability as a precondition of large AI deployment, which is a real safeguard against the machine-rebellion problem. For all that the CPC is tyrannical, they still don't actually want to kill all humans; they cannot rule humanity if they, and humanity, are dead. I won't pretend; there will probably be some countries that will attempt to cheat any international agreement not to pursue NNAGI, who will have to be knocked over. But I think there's a decent chance of achieving Jihad without a nuclear war being required.
In practice he does. How are you even going to tell a country like the US is developing one in secret?
Have Chinese/Russian inspectors looking at GPU production and use, as well as feedstocks for such in case of clandestine factory.
(The reverse would also be used.)
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link