This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
In terms of "it won't actually happen", either because it just straight-up fails because nobody can get a handle on it, or even if alignment can be designed, then because it'll be a disadvantage to any given AGI and the unaligned ones will outcompete it.
That’s more or less the premise of Eliezer’s new book. Haven’t read it yet but nobody knows how to do AI alignment, despite continuing technological advancement. An AI moratorium is not going to happen. Even if governments the world over declared as much, you can remain assured behind closed doors they’re still going full speed ahead.
Well, we're learning. Capabilities and alignment are being advanced through the same "training" paradigm, and roughly apace so far. Maybe they'll stay that way, and by the time further technological advancement is out of our hands it'll be in the "hands" of creations that still take care to take care of us.
It's easy to be pessimistic, though:
Many aspects of AI capabilities could in theory be advanced very rapidly via "self-play", although in practice we can't manage it yet on anything more complicated than Go. The is-ought problem in alignment is real, though; an alien from another galaxy could converge to something like our view of reality but would only get a fraction (whatever "moral realism" results you can get from pure game theory?) of our view of how to value different possibilities for reality. So, we might at some point still see a "hard takeoff" in capabilities, such that whatever robust underlying alignment we have at that point is all we're ever going to get.
The "Waluigi effect" makes alignment work itself dangerous when done wrong. Train an LLM to generate malicious code, and even if you think that's morally justified in your case, in the AI internals it might turn out that the "generates malicious code" knob is the same as the "humans should be enslaved by AI" knob and the "talk humans into suicide and homicide" knob and the "Hitler was a misunderstood genius" knob. "S-risks" of massive suffering were already a bit of a stretch under the original Yudkowsky explicit-utility-function vision of alignment - a paper-clip maximizer would waste utility by leaving you alive whether it tortures you or not - but in a world where you try to make Grok a little more based and it starts calling itself MechaHitler, it seems plausible that our AI successors might still be obsessed with us even if they don't love us.
There is no Three Laws architecture. Whatever alignment we can tune, someone can then untune. If superintelligent AI is possible, not only do we want the first model(s) to be aligned with our values, we want them to be so effective at defending their values that they can defend them from any superintelligent opposition cropping up later. Ever read science fiction from 1955, or watched Star Trek from 1965? Everybody hoped that, after the H-bomb, the force-field "shields" to defend against it would be coming soon. But physics is not obligated to make defense easier than offense, and we're not done discovering new physics. (or biology, for that matter)
No, it's not. Stuxnet was tricky enough; if everybody's video game console had a uranium mini-centrifuge in it next to the GPU, you could pretty much forget about nuclear non-proliferation. People point out the irony of how much attention and impetus Yudkowsky brought to AI development, but I respect the developers who read his essays and concluded "this is happening whether I like it or not; either I can help reduce the inherent risks or I can give up entirely".
More options
Context Copy link
As they have to. If AGI can do a fraction of what the most conservative futurists predict, it'll be a massive advantage that nobody can afford to forgo. Alignment or no.
Well, "no alignment" is so much worse than "no AGI" that anybody could afford to forgo it. But the USA would probably prefer a US AGI with "95%" alignment over a CCP one with "98% alignment", and they'd prefer a Chinese AGI with "90% alignment" over that, and so on, so nobody feels much incentive to be truly careful. Even within one nation, most companies would love to pull out far enough ahead of the competition to capture most of the producer surplus of AGI, and would be willing to take some negative-value risks out of haste to improve their odds instead of just taking a zero-value loss.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link