This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
In AI safely news, the "big, beautiful bill" Republicans are pushing through includes 10 year ban on states regulating AI. Seems pretty reckless, who could even imagine what AI could do in 10 years.
The issue with state regulation of AI is obviously that all relevant AI in the US is produced in California, maybe a little bit in Texas. This would have never flied with a Republican administration, but even that aside it's clearly discriminatory against everyone else who will experience the effects of Californian regulation.
More options
Context Copy link
Makes perfect sense to me, AI is a national-level issue. Really it's global, a server farm in Ohio can take jobs off Uzbeks and Bolivians, not to mention Floridians. Makes sense to regulate nationally.
Plus, would you really want California regulating a critical sector of the economy?
More options
Context Copy link
isn't a 10 year ban better than a bill which just bans states from regulating AI. at least the 10 years creates is a sunset clause on the regulation and would require congress to pass new legislation if they think continuing the ban is a good idea. though, maybe generally we should be pushing congress to include short sunset clauses in all legislation it passes because the future could be very different in X years.
More options
Context Copy link
We ask that top-level comments have more meat on the bone.
Since we’ve asked you specifically about this in the last couple months, one day ban.
More options
Context Copy link
I'd much rather see different states experiment with different forms of regulation, but it's good news for regulatory clarity and therefore the standing of the US in this domain.
At least China won't automatically win the race from the West kneecapping itself.
What AI problems are intrastate, rather than interstate?
Anything that has to do with law enforcement, healthcare, and any of the nuts and bolts of your life that don't happen on the internet?
I'd much rather have both a place that does and doesn't allow the pile of algebra to decide if you get the loan or not. I'm not certain which gives the best outcome.
I was asking from a place of pessimism about interstate companies faithfully executing independent processes for customers in different states (if the human-made algorithm is less profitable than the AI black-box, will the company scrupulously avoid benchmarking the former against the latter or pulling out of the market?), but fair point.
More options
Context Copy link
It's pretty obvious that using a pile of linear algebra gives the best outcome for loans, and literally every bank in the word has been doing this for >30 years. The more old fashioned banks have just been using a person to enter the data into the linear algebra.
To be more specific, consider some regulation that says that you actually have to be able to explain and defend the algorithm in a court, and aren't allowed black boxes.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
AI has a lesser problem of enabling dictatorship and a greater problem of rogue AI. In foot-race terms, the lesser problem is that there are other competitors, and the greater problem is that the finish line has landmines under it strong enough to blow up the whole track. As such, until and unless the mines are removed, "sprint harder" is not a solution; if the landmines are set off, you're still dead regardless of who did it.
What if you believe that when you reach the finish line, there's a 5% chance that the track will blow up, but if the other guys reach the finish line first, there's a 10% chance the track will blow up. Also you believe the other guys don't take the risks seriously so they won't stop running. Is "sprint harder" a valid option?
I mean, the preferred solution to "the other guys don't take the risks seriously so they won't stop running" is generally "whip out a pistol and shoot them", although the numbers you've given are on the edges of that solution's range of optimality.
I will note that in reality, the CPC appears fairly cognisant of the risks, probably would enforce stricter controls than "Openly Evil AI" and "lol we're Meta" (Google and Anthropic are less clear), and might be amenable to an agreed slowdown (there are other nations that won't be and will need to be knocked over, but it's much easier to invade a UAE or a Cayman Islands than it is the PRC).
Also, my P(Doom|no slowdown) is like 0.95-0.97, although there will likely be a fair number of warning shots first (i.e. the "no slowdown" condition implies ignoring those warning shots); to align a neural net you need to be able to solve "what does this code do when run" (because you're checking whether a neural net has properties you want in order to procedurally mess with it, rather than explicitly writing it, and hence to train "doesn't kill me when run" you need to be able to identify "kills me when run" in a way other than "run it and see whether it kills me"), and that's the halting problem (proven unsolvable in the general case, and neural nets don't look to me like enough of a special case).
I think the main point of contention is this. As evidenced by discussions up and down this thread, for most people, the number is closer to 0.05 than 0.95, so there's no political will to do the other things you suggest. A real-world example: Anthropic. When the OpenAI engineers quit the company because it wouldn't slow down for safety, they didn't shoot the remaining employees, instead they created a competitor to sprint faster with the belief that if they reach AGI first, it'll be better aligned for humanity.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I don't agree with your premise. I think the opposite is true. Therefore accelerate.
The premise of "rogue AI could potentially kill us all", or the premise of "we are currently on track to build rogue AI"?
The premise that rogue AI is a coherent concept distinguishable from an industrial accident. I don't believe machines have wills and there has been absolutely no reason to change my mind on that point.
Though both of these also have very large problems if you concede that point. People underestimate how extremely difficult "kill all humans" is as a task. And we have basically no idea what we're building and why it works at this stage so speculation on the terminal direction of the technology is bound to be nonsense.
Claude 4 and o3 will take action to avoid being shut down. If you leave aside the literally-unknowable "do machines have qualia" point, they sure seem to be best modelled as capable of agency.
I'm one of the people saying this. Preppers and other forms of resilience nullify a great many X-risks; another Chicxulub would kill most humans but not humanity (not sure about another Siberian Traps). But there is one specific category of X-risks where that kind of resilience is useless, and that's the "non-human enemy wins a war against us" set (the three risks in this category are the three sorts of possible non-human hostiles - "AI", "aliens" and "God"). Bunkers are no help against those, because if they defeat us they aren't ever going away, and can deliberately break open the bunkers; it might take them a few years to mop up all the preppers (though I imagine God would get everybody in the first pass, and aliens plausibly could), but that doesn't save humanity.
I disagree. For the same reason that I disagree that it was best to model Eliza as having agency.
You're training a statistical model to do something, and it does that thing. To model it as having agency would imply the thing starting to do things on its own that aren't just emergent properties of what you're making it do.
If you don't want unintended effects, don't train the thing on the whole of a culture you don't control. Calling it "rogue" is like calling a hammer evil because it hurt when you're hitting yourself with it. Stop hitting yourself.
More like uncontacted tribes.
Realistic X-risk is mostly down to physical conditions making our biosphere unable to sustain the critical mass of human life, like somebody lobbing a big rock at Earth or some invasive lifeform eating all the oxygen in the atmosphere. And even those are somewhat survivable. Losing a war to a recognizable enemy doesn't even register.
Hence why I think the main mitigating factor is any kind of extra-planetary backup.
I love this explanation, it's a great way to put it in perspective. I would also say that this -
Rules most people out of the agentic category. And that's why I say please and thanks to deepseek anyway.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Could you elaborate on the intra-state impacts of AI you'd expect states to want to regulate? I don't think any of the AI companies even tell you where their data centers are based, so "this all happened in Oregon" seems unlikely to even be true. Isn't most regulation of the Internet as a whole at the federal level? Even nationwide collection of state sales tax online didn't happen broadly until 2017.
More options
Context Copy link
More options
Context Copy link