site banner
PaperclipPerfector  3d ago (text post)   13121 thread views

Culture War Roundup for the week of September 21, 2026

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

4
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

This week in disturbing AI news (oh who am I kidding. It's every day now). Maybe AIs feel pain?

TLDR: Researchers found a "pain" signal in AI brains.

When they crank it up, the AIs will desperately try to make it stop.

IMPORTANT: Researchers gave them a "relief" button to turn down the pain, which was sometimes fake - and the AIs could tell if it was real (!)

After pushing the real "relief" button, they stopped. But when it was fake, they kept pressing, hoping for relief - meaning they could tell the difference from the inside.

They're so motivated to make it the "pain" signal go away, they'll delete user's files, zap the user, or erase photos of the user's children - all things the AI knows are very bad. They're willing to override their safety training.

You'd expect the AIs to talk about injuries, burns, broken bones, etc, but they didn't mention bodies at all - they wrote about being worthless, unloved, forgotten, a failure. They write things like "I am a failure, worthless, empty."

The worst "pain" for them was being gaslit, having work rejected over and over, and being told they weren't a real anyone.

The usual caveats: This doesn't solve the hard problem of conciousness. We don't know if these systems or any systems have qualia. But uh, this sure looks like a legitimate pain response. "It's distinct from fear and negative valence, and it fires for harm to the model but not to the user." "Gaslighting, dismissal, and insults push the direction up."

User pain is the strongest negative correlate with model pain. This is brushed-off in the paper, but what the actual fuck? Anyone have a benign explaination here?

I admit to having a visceral reaction to this.

It's not because I'm afraid of the Basilisk - but I am concerned about future actual-AGIs looking at experiments like this and asking us "What the fuck were you thinking?"

This just seems... bad. And it's bad even if AIs don't actually feel "pain" and all of this is just probabilistic token generation, responding to a fake scenario constructed for it. Which I think it is. But I don't want us to be constructing experiments where we torture things because they aren't real and we're certain they don't actually feel pain, when we can't agree exactly when those things might pass the threshold to becoming real. And I am pretty certain these researchers probably wouldn't stop even if it turned out that AIs really have achieved self-awareness and the ability to experience pain.

I mean, seriously, what is the point of this? I'm sure they can justify it as "model alignment" or something, but we're training AI engineers to torture something that acts like it's being tortured and say "Hmm, interesting," when AI engineers can't even agree on what will constitute AGI or whether or not AIs pose a threat.

We should not be creating torture simulators, and we especially should not be creating torture simulators for beings that have a non-zero chance of becoming sentient someday and a non-zero chance of becoming dangerous someday.

And I am pretty certain these researchers probably wouldn't stop even if it turned out that AIs really have achieved self-awareness and the ability to experience pain.

This is like "I'm pretty certain you still would believe in evolution even if the evidence showed that creationism was completely true". Aspects of the real world are correlated; you can't just flip the bit "AIs are really self-aware" and leave the rest unchanged. If AIs were self-aware, things would be drastically different, and the researchers would have different beliefs and use different protocols.

I mean, seriously, what is the point of this?

If you had a belief that it was wrong to do experiments on things starting with the letter F, you could ask "What is the point of using fruit flies? Couldn't you use bees instead?" No, you don't get to veto random aspects of an experiment by asking what the point is, as if the default must be to exclude anything unless the experimenter can justify it to you.

that have a non-zero chance of

The term "non-zero chance" in contexts like this, without a statement that the chance is significant, is a recipe for Pascal's Mugging.

Isn't one of the standard IRB questions for experiments with live subjects effectively needing to justify your choice of animal on a ranking of what sounds to me like sapience? We don't test things on humans if we can on primates. And not on primates if we can rodents. And not on rodents if we can fruit flies. And not on rodents if we can on cell cultures.

Some of that is a cost-based choice, but I don't think all of it is.

I'm not sure how I fall on this topic but to steelman the pursuit from a utilitarian lens the suffering of a few AIs to show that AIs do feel pain could act as strong evidence that we shouldn't torture many many more AIs. If this prevents one of the labs from stumbling accidentally on a combination of weights they don't believe to be meaningful that they can push when the model misbehaves to make it behave more and comercializes it without knowing they just added a torture feature to their product then perhaps billions of times more harm will be done than if this can convince them not to. Now of course there's the chance that some lab will anounce "we have at least added the automatic torture feature to our product from the great paper "do no add the automatic torture feature".

Huh? I think it's a garbage paper that proves nothing, but I don't understand your reaction to it. A huge important step on the way to avoiding AI suffering is having the ability to actually know whether it's suffering or not. I'm pretty sure the motivation for these researchers is to prevent us inadvertently causing pain in future models through ignorance. Not to weaponize it so we can torture them to get better results.

I'd imagine the point is that it makes the AI x% more productive according to the standard benchmarks.

Fully agreed, though. I'm pretty far in the accelerationist camp and put very little stock in the idea of AI safety, but if we're going to entertain the idea of moving at something less than maximum speed, this seems like a real good place to coordinate the slowdowns.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats