site banner

Culture War Roundup for the week of November 9, 2026

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

5
Jump in the discussion.

No email address required.

This isn't my read on things. Average and dumb people can understand the concept just fine. They might not like it and the changes it brings, but they absolutely recognize the threat; it's been communicated by, if nothing else, pretty simple and direct sci fi stories that they've seen on the big screen.

Right - I think the OP comes off a bit too much like "most people aren't as smart as me", whereas I suspect that most people understand the central plot point of The Terminator.

"We build a superintelligent computer and things go badly" isn't some sort of weird esoterica. It's been a theme in popular media for over half a century. You don't need a high IQ to understand, "Maybe we build a computer and the computer improves itself and then the computer destroys us".

Charitably, maybe the OP means that most people don't think that's likely in reality, or that most people don't understand the specific mechanism that he posits for it. I think the former is probably true and the latter is probably false. I think most people don't expect Skynet to appear, but most people probably can understand the concept of a computer that makes itself smarter. It's really not a very complicated idea.

most people probably can understand the concept of a computer that makes itself smarter. It's really not a very complicated idea.

It's weird that it doesn't really figure into popular ideas of AI, though, isn't it?

In elite non-fiction we get "Speculations Concerning the First Ultraintelligent Machine" in 1965.

In popular fiction, just looking at your linked comment: you're right that "AI might be dangerous" is well-attested in popular fiction, but "AI might upgrade its intellectual capabilities extremely rapidly" not so much. WOPR/Joshua and HAL are dangerous mostly because of how smart they aren't, and Skynet and AM gain self-awareness but don't obviously get much smarter. There's a TNG episode where Barclay both gets much smarter and integrates with the computer, but it's all external influence, no runaway positive-feedback loop involved. Data is a little more realistic than C3-PO (the droid who's basically sapient but who isn't even obsolete decades later, much less creating his own successors), but it's still not surprising when Lal doesn't work and his research stops there.

The earliest popular fiction I know of featuring recursive self-improvement is A Fire Upon the Deep, so of course I went to the TVTropes page to see what the trope name was and if the trope page listed anything earlier ... and the only trope I see for the Blight is "Eldritch Abomination", nothing more specific. Even in that case, the self-improvement is essentially a "reboot" of previously-existing capabilities; in a sense the main interaction of the story with the idea of recursive self-improvement is that the "zones of thought" are a necessary gimmick to create a plausible future setting where superintelligence is a big deal but RSI isn't.

"AI capability is a function of accumulated research, and if that function's growth rate isn't slow enough then after AI can do any AI research y'=f(y) blows up" ... simple enough to squeeze into two clauses (for anyone who's had Calc 1; for "most people" maybe add a second or third sentence too), sure. But if you then add "we don't have reason to believe its growth rate will be slow before it hits far-superhuman levels" and "artificial-intelligence research is probably easier to do right than artificial-ethics research" then this has obvious implications of the Skynet variety, yet it's surprisingly hard to find people who were taking this seriously who didn't get there via Bostrom and Yudkowsky.

"AI capability is a function of accumulated research, and if that function's growth rate isn't slow enough then after AI can do any AI research y'=f(y) blows up" ... simple enough to squeeze into two clauses (for anyone who's had Calc 1; for "most people" maybe add a second or third sentence too), sure.

I think that's already overcomplicating it. The mathematics isn't important. What you're trying to get across is the idea of a learning supercomputer, and I think that's easily comprehensible by most people on the street.

Don't be distracted by the acronym 'RSI'. All 'RSI' means is "a computer that gets smarter very very quickly". I think that concept is both very simple and very well-understood.

yet it's surprisingly hard to find people who were taking this seriously who didn't get there via Bostrom and Yudkowsky.

I actually don't rate Bostrom or Yudkowsky as serious thinkers to begin with.

Who would you rate as a serious thinker then? They seem to me to have defined the age.

I wouldn't agree with that, but before explaining why, we should distinguish serious thinkers from impactful thinkers. 'Defining the age', or having a large effect on others or on the world, do not make a thinker serious. A serious thinker might be someone no one has ever heard of; a very influential thinker might be a superficial fool.

Now in this case I don't think Yudkowsky actually qualifies as either, but to be clear, I meant seriousness, not impact. The case is fairly self-explanatory in terms of impact - for all that Yudkowsky was and is a self-appointed 'expert' on AI, the industry as it actually came to exist has not been shaped by his ideas or prescriptions. The AI safety movement itself largely fails to meet the bar of effectiveness or impact, but even if it did, it does not function in a particularly Yudkowskyan way. I'm not sure what you mean by 'the age', but I do not see any significant way in which his ideas have shaped the world. Yudkowsky is a mildly famous blogger. That's all.

Still, my criticism was of the merits of Eliezer Yudkowsky Thought (and Nick Bostrom Thought), rather than of them as activists. Let me put it this way. What are their philosophically interesting ideas? What insights would you attribute to them? I'm not sure where to start, really - the Sequences are large, rambling, and turgid, and I would say not philosophically interesting, in the sense that very little in them is new or insightful.

Technically, an AI learning emotional intelligence and growing indistinguishable from a human would be a kind of self-improvement.

Though the kind of thing you are talking about is indeed rare. It would make for a pretty good villain if you had a AI that started out as dumber than the average human but over the course of the story increases its capabilities to become almost godlike by the end.

Her involves the AIs explicitly engaging in self improvement until they ultimately decide to go rogue and abandon humanity (a relatively optimistic outcome).

There are some things that come close. Like Arnie's explanation of Skynet's history in Terminator 2, "Skynet begins to learn at a geometric rate. It becomes self-aware at 2:14 AM Eastern Time, August 29th."

There is also the TNG episode where in response to being asked to create a holodeck program capable of posing a challenge to Data's intelligence, the Enterprise's computer creates a holodeck program, Professor Moriarty, that is so smart that the crew only barely manage to defeat it. It's not explained whether the ship's computer was always that smart, or whether it did something novel in response to the prompt. Still suggestive, though.

I don't think it's really explored in the series, but the difference between Data, a self-aware android and character, and the ship's computer, which seems on par with Claude (or so) feels particularly salient. Maybe at the time it was a convenient tool for the writers, but I'm not sure how I'd describe it even. For lack of a better term, Data is shown to have a soul, but the computer does not. All sides seem to have lots of strong opinions on which our current models more strongly resemble.

I wouldn't say that Data is on par with Claude. Data is clearly a person. He has memories, desires, persistent self-awareness, and so on. Though Data claims to not have emotions, like Spock before him, he clearly does. Data is a human-like intelligence. Claude is not an intelligence at all.

From the writing perspective, I think it's fairly straightforward? Data is first and foremost a character. He is written as a person who happens to be made of metal, much as Worf is a person who happens to be a klingon. The ship's computer is not a character. It is a tool and presented as such. Everything about what Data is or how he functions is downstream of Data as a character,

You misread: he didn’t say Data was on par with Claude, he said the ship’s computer was.