This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

Jump in the discussion.
No email address required.
Notes -
AI 2040: Plan A
The AI 2027 authors published a follow-up. Scott Alexander also wrote a separate blogpost and although not in the author list contributed.
It's a very speculative and optimistic timeline of AI's future evolution. It presents five ways or "plans" the US government will intervene. Unsurprisingly, the ASI-pilled authors favor strong, global regulation to ensure alignment. Summaries:
Plan A (recommended): the US makes an international treaty with China, pauses AI training (not inference, i.e. no new models but we keep using existing ones), enforces full transparency of future research, then when alignment research advances enough carefully resumes
Plan S: the US makes an international treaty with China and pauses AI training for as long as possible
Plan B: the US regulates AI at home and demands China also regulate, but doesn't negotiate with them, probably leading to a war
Plan C: the US regulates AI and ignores China, so they overtake it and reach ASI first
Plan D: the US doesn't regulate AI, we get ASI in early 2031 and it probably kills everyone
Personally, I just don't share the optimism of these guys in either direction.
I think politicians will prioritize culture war and the failing economy over AI regulation, and at most pass some executive orders suggesting companies be more careful. But I also doubt we'll have ASI that can solve the abstract problems "take over the world" or even "keep existing world leaders in power" (they're getting old and increasingly unpopular, their parties may remain in power but only if their policies significantly shift).
What I expect from AI:
Basically solve legacy code by rewriting entire codebases, applying very niche domain knowledge, and actually finding and handling edge-cases better than humans
Greatly speedup research, leading to new discoveries and inventions. Important but background things like food preservation and medicine will improve from AI-assisted discoveries. Major advancements in math and theoretical physics
Much better and cheaper education, therapy, initial medical/legal appointments, personal repairs...maybe reducing but not eliminating human jobs, because human experts will offer these services "premium"
Won't replace human artists. Some advertisements and infographics will be AI but even some will still be human. At best it will assist them in a way where the human still fully controls the output, e.g. by generating code leading to new and improved software tools to learn, practice, and create art
Used by the vast majority as a personal assistant, but doesn't replace human relations
If AI is going to kill everyone unless some fantastical global regulatory regime that won’t happen is magicked into being, what are these people even doing?
Go to the beach, touch grass, spend time with people you love, drink good wine. No atheist (which pretty much all rationalists are) who believes the world is about to end should spend their final months playing cassandra, to mostly deaf ears.
Isn't that what Scott is doing, essentially? Having a kid and enjoying life.
I think most people who think of doom as a likely outcome are not letting it drive their decisions: no one at this point thinks there's a chance that this kind of safety regime will develop and be effective. Instead, it's optimizing for scenarios where their individual actions might matter, e.g. permanent underclass scenarios (best get that SWE role at Anthropic to get your bag).
Even Eliezer seems to have more or less given up, getting his utils mostly from investing his sense of psychological worth in doom.
The SWE at Anthropic to avoid permanent underclass status strategy reflects a very limited understanding of finance or markets, but I suppose that’s to be expected for software engineers.
Somewhat ironically, the "get a ton of money at Anthropic" likely offers the most over alternative strategies for scenarios that are more like an AI fizzle or bust.
I presume the gambit there is to get equity and then get rich once it IPOs or afterward as it grows to take over the whole economy. The problem is that if AI including Claude takes a lot of white collar jobs, the first thing that happens - before the riots, the protests, the political upheaval - is that people start liquidating their 401ks and that tanks the whole market in a cascade, so the Anthropic engineer’s paper millions are toast.
You can't hedge against total financial catastrophe unless you have at least Larry Ellison "make my own self-sufficient villain lair" money and ability; you have to do it in the physical world, not the world of finance. So there's no point in trying. If some less-than-catastrophic event where Anthropic is successful happens, those SWEs get rich just like in earlier tech booms.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
No amount of worldly carnal pleasures will measure up to the spiritual nirvanah the rationalists will feel in the few moments between seeing the flash and burning to death. They were right, and everyone else was wrong. Their arguments were valid, everyone else's invalid. If only the sheeple had listened and installed Eliezer Yudkowsky as Emperor of Terra this could have all been avoided.
And they will be right to feel this way.
I have embraced the Cassandra role, and the brief moment of payout will be enormous and entirely worth it. Like a thousand simultaneous continuous orgasms lasting for days as the end becomes clear.
More options
Context Copy link
The entire history of humanity exists solely to maximize the utility felt by the entire rationalist community in the picosecond before they keel over dead with the rest of us as the AGI executes its plan.
The bad ending is when the AGI convinced them that AGI is impossible right before deleting them.
Holy christ dude.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
For the most part, people do these things because they feel good, agreed?
Arguably, it also feels good to have a feeling that you are RIGHT, both in the sense of being correct and also in the sense of being moral or just.
It feels good to feel (correctly or incorrectly) that you are working to save humanity (or some slice of humanity) from people who are some linear combination of stupid, ignorant, and evil. That you alone possess some important knowledge. That some day people will regret having laughed at you.
As another poster pointed out, look at Eliezer Yudkowsky. If his worldview is correct (as believed by him and the people around him) it means that he is one of the smartest and/or most important people in the world. That must be a very heady feeling.
For a lot of people that kind of feeling is very tempting compared to sunburn and a hangover.
Edit: I'm not necessarily saying that the likes of EY are wrong. I do think they overstate their case, but that's a different issue. I'm just offering a hypothesis as to why someone who sincerely believes the world is doomed would continue to preach doom rather than just kick back.
More options
Context Copy link
They don't want AI to not kill everyone. They want fantastical global regulatory regimes.
This is what we would term uncharitable thinking on any other post
I dont think its uncharitable, from my reading of various AI safety writings I would describe it as substantially accurate.
I don’t think it’s uncharitable, but I do believe the prominent rationalists are altruists (or less kindly, quokkas) who genuinely want regulation to save humanity and/or are using the threat for attention and funding.
It doesn’t matter, because the people implementing the regulation would almost certainly corrupt it for power and personal gain, including undermining it for themselves or their friends, unless it was designed so that corruption and undermining are almost impossible. If ASI needs much more chips than already exist, that may actually be feasible (because a defector must run many fabs, which takes a while and can’t be easy to hide).
More options
Context Copy link
More options
Context Copy link
Yeah, probably, but it's fun to dunk on the "AI safety" nerds
Honestly all of us just think you guys are making yourself look plainly retarded by making obviously fallacious arguments. If I feel any embarrassment as a result of these posts it's that I'm vaguely associated with the sort of discussion forum with people who post like this. Somehow people whom I have explained the economics of inference too multiple times continue to falsely claim that inference is unprofitable and never will become profitable. And then there's the bulverism, "people who are concerned with ai safety just want regulations" have you ever met literally anyone involved in this field? Seriously? They're one of the highest concentrations of libertarians one could select for outside of an Ayn Rand fan club.
Yud is one of the leading voices in the "AI safety" crowd and "Bomb the datacenters" is one of his literal proposals for dealing with rogue ASI. Now you can argue that Yud isn't representative of that crowd but he's absolutely one of the biggest figures for it in the public eye.
I think that "bomb the datacenters" is a silly idea for various reasons, such as: 1) WW3 isn't much better than ASI apocalypse, 2) it would just delay ASI.
That said, I don't think it's intrinsically un-libertarian. I think all but the most extreme fringe libertarians would agree that, for example, it's ok to use heavy government control to fight off an alien invasion.
A genuinely rational libertarianism would argue that yes, in some situations a giant authoritarian government is actually the best solution.
3 - They censored the only thing that could teach them how to bomb them.
In my opinion - if they ever try to do rapid forced disassembly of datacenters - this is the song they will listen to while looking at both their hands.
https://youtube.com/watch?v=u8ccGjar4Es
More options
Context Copy link
More options
Context Copy link
Yes? That's how treaties work? In a world with Iran and Russia, it's just transparently silly to say that "bomb the dangerous production plant" is somehow beyond any sort of boundary when countries have been doing it casually for decades.
@IGI-111 's comment that kicked off this chain:
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
The nuclear non-proliferation treaty was not about preventing nuclear apocalypse. They just wanted a fantastical global regulatory regime.
Correct. Notice great powers were sure to get the bomb before locking in their hegemony. And all serious countries maintain a turnkey program in case the US stops having the will or ability to enforce it.
Political coercion is so seldom selfless that I'm willing to stand by the idea it never is. And call it realism.
And yet, I can't help but notice that the nuclear apocalypse has still not occurred.
It did occur and the Japanese surrendered
More options
Context Copy link
I don't see how that affects my point that the ultimate goal of such maneuvers has never and will never be disinterested.
It's evidence against "The nuclear non-proliferation treaty was not about preventing nuclear apocalypse". Whether or not the people involved wanted a global regulatory regime, it seems pretty likely that they wanted to avoid a nuclear apocalypse, and that the treaty was an attempt to do so.
Formally it is not.
Let's say me and my friends start charging you to protect your business from being set on fire. You pay, no fire ignites. Is that evidence that we wanted the protection over the money?
Now if you said that nuclear weapons are actually dangerous that would actually be evidence of favor of that thesis. But then we'd have to go into how that apocalyptic evidence of things like nuclear winters is fraudulent.
Preventing nuclear terrorism and deterrence by the enemies of great powers is the reason GPs agreed to a cartel, by equivocating destroying GPs with destroying humanity.
AI's the same. The managerial class is afraid of their displacement by technology and is trying to lock in their extant power by establishing another cartel.
People afraid of losing their power and shielding themselves with the purported welfare of humanity is as old as time.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Speaking of nuclear apocalypse, though.
Plan B is the real one here. Implementation probably involves going to war with a near-peer, nuclear-armed superpower so sophisticated they are on the edge of ASI.
You probably do need a credible path for how a full-scale conventional conflict avoids escalating to nuclear exchange and then avoids escalating to nuclear apocalypse, if that's an at-all viable Plan B.
It would be quite ironic if the way AI managed to kill all of us is that we were so afraid it might go to a fast takeoff that we decided a nuclear apocalypse would be better than risking it.
More options
Context Copy link
Nah, not even that. The NNPT is just about the big kids deciding the big stick club should be exclusive to them only. And its enforced by big sticks, not some figleaf of global regulation. It would only work if AGI already existed but was a closely held secret of the militaries of the various signatories.
For the most part (excluding e.g. rambling threats of the odd insane world leader), the NNPT is not enforced through nukes. The primary tool to discourage nuclear weapons programs are economic sanctions. Coupled with the big powers signaling willingness to defend their non-nuclear allies (excluding demented world leaders, again), this creates an environment where it is not in the interests of most states to have a nuclear weapon program.
I will grant you though that the UN security council is very much a club of nuclear powers regulating the behavior of everyone except for themselves. Still, the SC is more born of pragmatism, and sometimes works to police small states who are not client states of any of the permanent members, which is still better than nothing.
It hasn't yet, but as recent events show, it is enforced through stealth bombers dropping 30,000 lbs bunker busters and massive air raids. Or full scale ground invasions deposing the existing governement. All of which I would classify under "Big Stick" diplomacy.
Time will tell if Trump's adventures will actually result in a strengthening of non-proliferation.
"If you do not own nukes we will bomb your nuclear facilities, military installations, government leaders, civilian infrastructures, and schools whenever we feel like it" seems exactly like extraordinary events, related to the subject matter of this Treaty, have jeopardized the supreme interests of its country, which is a condition for leaving the treaty after 90 days.
Is there anyone you think would take that lesson to heart, who didn't already after Libya?
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
You say fantastical global regulatory regimes and I hear big kid threatening to blow up the Three Gorges Dam if they don't comply.
Same difference really.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
The trick is, there's undoubtedly some sociopathic subset of people who did in fact just want to gain a global regulatory regime's power and weren't all that worried about nuclear war.
And these are the people who most want to control that regime, and UNDER NO CIRCUMSTANCES should be allowed to.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link