site banner
toadworrier  10mo ago (amphobian.info)   735 thread views

Rules before Governance (AI edition)

amphobian.info

I have a short Substack post about AI regulation which is itself a teaser for my much longer article in Areo Magazine about AI risk & policy.

When Chat GPT 3.5 was released, I was tribally. But within weeks my emotions switched tribes, even though my actual rational opinions have been more or less consistent. Basically

  • We have almost no real-world understanding of AI-alignment, we need open, visibile experimentation to get it.

  • There really is risk, AI-development needs legal limits,

  • Those limits should be more about rule-of-law than administrative power.

  • The goal is to create and delimit rights to work on AI safely.

But do read the actual articles to unpack that.

That's what I want, but what I'm afraid we'll get (with the backing of the AI-risk community) is a worst-of-both-worlds. Large unaccountable entities (i.e. governments and approved corporations) will develop very powerful Orwellian AIs, while squelching the open development that could (a) help us actually understand how to do AI safety, and (b) use AI in anti-Orwellian tools like personal bullshit detectors.

I understand the argument that crushing open development is Good Actually because every experiment could be the one that goes FOOM. But this Yuddist foomer-doomerism is based on an implausible model of what intelligence is. As I say in Areo (after my editor made it more polite):

Their view of intelligence as monomaniacal goal-seeking leads the rationalists to frame AI alignment as a research problem that could only be solved by figuring out how to programme the right goals into super-smart machines. But in truth, the only way to align the values of super-smart machines to human interests is to tinker with and improve stupider machines.

Any smart problem-solver must choose a course of action from a vast array of possible choices. To make that choice, the intelligence must be guided by pre-intellectual value judgements about which actions are even worth considering. A true-blue paperclip maximiser would be too fascinated by paperclips to win a war against the humans who were unplugging its power cord.

But even if you did believe in foom-doom, centralising development will not help. You are just re-inventing the Wuhan Institute for Virology.

1
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

I think you're making two separate arguments here and not distinguishing enough between them.

  1. You think that letting governments build high-powered AI while shutting down others' access to it is a bad idea.

  2. You don't like Eliezer Yudkowsky and those who follow him.

The thing is, these are entirely-separate points. Eliezer Yudkowsky does not want to let governments build high-powered AI. Indeed, his proposal of threatening and if necessary declaring war against governments that try is the direct and inevitable result of taking that idea extremely seriously - if you really want to stop a country from doing something it really wants to do, that usually means war. And he's fine with interpretability and alignment research - he doesn't think it'll work on neural nets, but in and of itself it's not going to destroy the world so if people want to do it (on small, safe models), more power to them.

So it's kind of weird that you set up Yudkowsky as your bugbear, but then mostly argue against something completely different from the "Yuddist" position.

As an aside, I think you're wrong to say that pursuing Yudkowskian Jihad will necessarily end in WWIII with China. The PRC has actually started requiring interpretability as a precondition of large AI deployment, which is a real safeguard against the machine-rebellion problem. For all that the CPC is tyrannical, they still don't actually want to kill all humans; they cannot rule humanity if they, and humanity, are dead. I won't pretend; there will probably be some countries that will attempt to cheat any international agreement not to pursue NNAGI, who will have to be knocked over. But I think there's a decent chance of achieving Jihad without a nuclear war being required.

[This is quick, a partial response, I'll have to read your comment more carefully to give it a full and fair thought. Thanks!]

  1. You think that letting governments build high-powered AI while shutting down others' access to it is a bad idea.
  1. You don't like Eliezer Yudkowsky and those who follow him.

The thing is, these are entirely-separate points. Eliezer Yudkowsky does not want to let governments build high-powered AI.

My views on Eliezer are much more complicated than (2) that, but for the sake of this argument, I'll accept that as a simplifying approximation. Roughly I like Yud but disdain Yuddism.

The thing is, these are entirely-separate points. Eliezer Yudkowsky does not want to let governments build high-powered AI. Indeed, his proposal of threatening and if necessary declaring war against governments that try is the direct and inevitable result of taking that idea extremely seriously

Yes, to his credit (and I do give him credit for that in the article), Eliezer bites that bullet. But in this he becomes his own reductio

If you fear such experimentation in the west, then you should fear the use of AI by autocracies even more. Eliezer Yudkowksy is honest enough to acknowledge that—but it commits him to the absurdity of advocating a shooting war against a nuclear power as a hedge against Doomsday.

Eliezer's global ban is not going to happen. Even a ban in the west is not going to happen. The Biden administration is gearing up to set up an AI bureaucracy to regulate it on the advice of Sam Altman. And I see the LessWrongers cheering and declaring victory at each new headline.

[This is quick, a partial response, I'll have to read your comment more carefully to give it a full and fair thought. Thanks!]

I'll wait for the full response before replying; I don't want to go off half-cocked, and it'd also be annoying to have two parallel conversations.

Ok, I've reread your post, and I think I stand by what I said above.

So it's kind of weird that you set up Yudkowsky as your bugbear, but then mostly argue against something completely different from the "Yuddist" position.

I don't think that's a fair reading of either my substack post or my Areo piece. Yes, I do criticise his position, but my main target is centralisation. The reason I bring in Yuddist foomer-doomerism into this is (a) this position is actually quite prominent, Eliezer got published in Time for chrissake, and (2) the "smart" argument I hear for why centralisation is Good Actually is based on foomer-doomer assumptions. Eliezer himself is more consistent: he knows that his assumptions lead him to a ban even on government activity. His acolytes seem to think "well the worst of both worlds at least gets us part of the world we want, so let's go for it".

The PRC has actually started requiring interpretability as a precondition of large AI deployment, ...

This is just naive.

But anyway, even if you believe the people who brought us the Wuhan Institute for Virology have got it all covered, then you still have to worry about all the other countries in the world.

For all that the CPC is tyrannical, they still don't actually want to kill all humans; they cannot rule humanity if they, and humanity, are dead.

Communists don't all always kill millions of people on purpose! For example Great Chinese Famine, was more incompetence than malice. Here's what I say in Areo

The tyrannical governments of the past were dependent on human beings to administer the machinery of repression, but an AI-powered tyranny has other means at its disposal. Totalitarian states have never been reluctant to depose their own leaders and an AI-powered ruling party could afford to dispense with every last cadre. The machines might be the true leaders. There’s no telling what such machines may choose to do with us humans. They might simply kill us all, since we are superfluous and a little unpredictable.

And I see the LessWrongers cheering and declaring victory at each new headline.

His acolytes seem to think "well the worst of both worlds at least gets us part of the world we want, so let's go for it".

I think this is more that a lot of the LWers had (incorrect) priors that the world would never listen until it was too late, so even insufficient levels of public and government reaction are a positive update (i.e. the amount of public reaction they are seeing, while not currently enough to Win, is more than they were expecting, so their estimated likelihood of it going up enough to Win has been raised). I'm open to being disproved, there, if you have a bunch of examples of LWers specifically thinking that national governments racing for NNAGI could possibly end well.

But anyway, even if you believe the people who brought us the Wuhan Institute for Virology have got it all covered, then you still have to worry about all the other countries in the world.

Sure! Like I said, I think that instituting a full global ban on large NNAI will probably require war at some point. But most countries do not have large nuclear stockpiles, so this doesn't necessarily mean a nuclear WWIII with X * 10^8 dead. I think the latter would probably still be worth it, but as a factual matter I think it's fairly likely that the PRC specifically would fall in line - while in theory a party machine can be made of AI, the current CPC consists solely of humans who do not want to kill all humans.

The thing is, these are entirely-separate points. Eliezer Yudkowsky does not want to let governments build high-powered AI.

In practice he does. How are you even going to tell a country like the US is developing one in secret?

Have Chinese/Russian inspectors looking at GPU production and use, as well as feedstocks for such in case of clandestine factory.

(The reverse would also be used.)

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats