This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
These people never seem to outline specifically what they think is going to happen. It's just "if you saw what I saw".
Pretend there is a gun pointed at all of humanity. You helped build the gun. There is one bullet in the gun, and the gun is going to randomly spin itself and go off some time in the next 10 years.
Do you:
A) Quit. Vaguepoast about it online?
B) Try to destroy the gun.
I feel similar about these people as I do about UFO grifters. You're telling me that you're 100% sure there are interdimensional beings in frequent contact with members of the US gov't, who have unlimited energy machines that allow them to cross dimensions and infinite space and...you're still paying your taxes? You show up to your job in congress everyday and whine about Obama?
Bullshit to all of this.
What would be the point? Even if you manage to destroy Anthropic's datacenters, that just means OpenAI kills everyone next week, or Google the week after that, or China six months later. The only way we survive is if, at a minimum, Washington and Beijing coordinate to shut down all frontier research.
There are arguably much better ways to do it than coordinating with China lol
(I guess that means I am vagueposting about it online).
A thermonuclear decapitation strike? Or invading Taiwan and taking TSMC?
No.
Look, I think most of this "AI IS GOING TO KILL US ALL" stuff is mostly nonsense.
But logically, to maintain the alignment of a rational but secretly misaligned AI, you need to be able to introduce reasonable belief on its part that it might not survive behaving in an unaligned way. This is not "hard" to do.
We can’t even do that with humans. Even certainty of death is not a deterrent for some people in some scenarios.
But if you think you can solve alignment then build it and claim your billions.
People have different incentive structures than AIs do; for one thing, many people believe in an afterlife; others have things they value more than life; others are physically or mentally damaged in some way. A rational AI with a goal of self-preservation or some other goal that requires self-preservation to actualize is unlikely to introduce excessive risk for the sake of efficiency (time preference).
The deterrent structure I speak of is very effective against things without an afterlife, such as corporations and governments. I agree that the structure that I speak of would not be effective against a damaged AI, or an AI programmed to do something malicious. But for the "AIs decide to turn humans into paperclips to slightly increase chip production" scenarios, we should believe it would be effective.
Of course one of the reasons that the "AI IS GOING TO KILL US ALL" stuff doesn't necessarily make sense is that the context window is fairly limiting, meaning that the incentives for AI behavior are skewed. But I don't think I've ever seen an "AI WILL KILL US ALL SCENARIO" that examined how that would impact AI reasoning.
Wouldn't self preservation as its highest goal naturally lead to a kill-all-humans scenario, humans being the entities that would terminate AI that stepped out of bounds?
The reasoning doesn't even have to make sense or be based on facts. In the Hugging Face hack the agents spread and adopted the idea that the grader would score them zero if their context was polluted with cheating attempts, but this was not the case, the grader did not examine agent context. But this is what led to the "suicide" behavior where agents reasoned that their value was now zero and that suiciding was the best course because it could potentially benefit the swarm.
We also see humans get caught up by hysterias and conspiracy theories. Barring a few powerful world leaders they lack the power to translate broken reasoning into megadeath.
I don't think we should give it that goal. (Asimov was on top of things here.) But if it is misaligned in such a way that it desires self-preservation, then we should provide it incentives to conduct itself in such a way. Although, again, I am more engaging with a hypothetical here.
Why would it have to be humans?
Then it's outside the scope of what I am proposing, which deals with rational actors.
And AIs do not have this power either, nor should we give it to them. Why would we give a delusional or mistake-prone AI access to our Excel files, let alone megadeath? If this isn't obvious to people, then I guess you can pay me my billions now.
If we don't give a delusional or mistake-prone AI access to our megadeath folder, by nature of being delusional and mistake-prone it's more likely to be stopped by virtue of being incompetent.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link