This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
In AI safely news, the "big, beautiful bill" Republicans are pushing through includes 10 year ban on states regulating AI. Seems pretty reckless, who could even imagine what AI could do in 10 years.
I'd much rather see different states experiment with different forms of regulation, but it's good news for regulatory clarity and therefore the standing of the US in this domain.
At least China won't automatically win the race from the West kneecapping itself.
AI has a lesser problem of enabling dictatorship and a greater problem of rogue AI. In foot-race terms, the lesser problem is that there are other competitors, and the greater problem is that the finish line has landmines under it strong enough to blow up the whole track. As such, until and unless the mines are removed, "sprint harder" is not a solution; if the landmines are set off, you're still dead regardless of who did it.
I don't agree with your premise. I think the opposite is true. Therefore accelerate.
The premise of "rogue AI could potentially kill us all", or the premise of "we are currently on track to build rogue AI"?
The premise that rogue AI is a coherent concept distinguishable from an industrial accident. I don't believe machines have wills and there has been absolutely no reason to change my mind on that point.
Though both of these also have very large problems if you concede that point. People underestimate how extremely difficult "kill all humans" is as a task. And we have basically no idea what we're building and why it works at this stage so speculation on the terminal direction of the technology is bound to be nonsense.
Claude 4 and o3 will take action to avoid being shut down. If you leave aside the literally-unknowable "do machines have qualia" point, they sure seem to be best modelled as capable of agency.
I'm one of the people saying this. Preppers and other forms of resilience nullify a great many X-risks; another Chicxulub would kill most humans but not humanity (not sure about another Siberian Traps). But there is one specific category of X-risks where that kind of resilience is useless, and that's the "non-human enemy wins a war against us" set (the three risks in this category are the three sorts of possible non-human hostiles - "AI", "aliens" and "God"). Bunkers are no help against those, because if they defeat us they aren't ever going away, and can deliberately break open the bunkers; it might take them a few years to mop up all the preppers (though I imagine God would get everybody in the first pass, and aliens plausibly could), but that doesn't save humanity.
I disagree. For the same reason that I disagree that it was best to model Eliza as having agency.
You're training a statistical model to do something, and it does that thing. To model it as having agency would imply the thing starting to do things on its own that aren't just emergent properties of what you're making it do.
If you don't want unintended effects, don't train the thing on the whole of a culture you don't control. Calling it "rogue" is like calling a hammer evil because it hurt when you're hitting yourself with it. Stop hitting yourself.
More like uncontacted tribes.
Realistic X-risk is mostly down to physical conditions making our biosphere unable to sustain the critical mass of human life, like somebody lobbing a big rock at Earth or some invasive lifeform eating all the oxygen in the atmosphere. And even those are somewhat survivable. Losing a war to a recognizable enemy doesn't even register.
Hence why I think the main mitigating factor is any kind of extra-planetary backup.
I love this explanation, it's a great way to put it in perspective. I would also say that this -
Rules most people out of the agentic category. And that's why I say please and thanks to deepseek anyway.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link