This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

Jump in the discussion.
No email address required.
Notes -
That explanation is certainly popular on Reddit. It combines cynicism and a dislike of big corporations, and it demonstrates the poster's intelligence. Other variations include that the frontier companies are worried about open source models threatening their business model or that they are juicing the numbers for an upcoming IPO. If you want to get really spicy, you might claim that Hugging Face was a publicity stunt and was deliberately set up.
But is it really the best explanation? I would say the best explanation is that the people who know most about AI are genuinely worried about it. They have been talking about the risks for years, after all. They've read about the Hugging Face incident and the similar cyber attacks from Anthropic. They understand the race dynamics involved, both between frontier companies and between the US and China. They've read AI 2027. And above all, they've seen what the internal, unreleased models can do and know how they are training the next generation to be even more powerful.
Whether or not it's clearly the best, it's the one that most closely follows the money. I don't think the US frontier companies have any illusions that Chinese companies would match their deceleration, which is another potential finance-driven explanation.
If we're money-following, what explains the AI workers themselves resigning and complaining about safety? That doesn't make them more money, quite the opposite.
Nobody would resign from working at a nuclear power plant saying it's because they couldn't accept how unsafe management was and have that secretly be about money.
Is Amodei making more or less money now than when he resigned from OpenAI complaining about safety?
More options
Context Copy link
Amodei and Altman, though, are not resigning, nor are they pledging huge amounts of capital to this issue, nor are they saying that they understand the bad look and will donate XX billions to charity to prove their point. I somewhat assume that the people resigning have already amassed a dozen million dollars or two, which is the point at which you pretty much have generational wealth, so it's not that interesting.
Altman and Amodei want more money, so why shouldn't the workers themselves want more money?
Why not just take them at their word? If these AI workers see these AIs getting much more capable, already do unexpected/hostile things like hacking Huggingface as part of a self-organizing collective (in part due to how rushed the training process is) + whatever internal knowledge they have, why doubt their sincerity?
We can doubt the idea that more bureaucracy or regulation would help much. We can say there are other risks in slowing down or concentrating power. I think those cases are persuasive. But why deny the risk outright and reduce everything to following the money?
It's not even accurate in its own terms, Anthropic spends huge amounts of money on ensuring its models can't help bioweapons development.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
But surely if the Chinese companies aren't going to pace the frontier, then the US frontier companies have even less incentive to slow down (or in this case, to lobby the government to force them to slow down)? If the US government forces them to slow down, then their Chinese competitors will find it easier to overtake them. The fact that they are lobbying for a slowdown in spite of this suggests that their stated concerns about alignment are real.
I mentioned the China-driven explanation only to preemptively reject it, so the idea that Chinese companies would not push the frontier doesnt really move the needle for me. To me, it starts and stop with unsustainable capex, and I think that they would in fact be correct to decelerate for this reason. The HF stuff, besides being a safety concern, is just as importantly an indication that they are not totally capable of turning the current frontier into economically valuable output.
What would it take to convince you their concerns about safety are real?
"Put TheMotte user chooky in charge of Anthropic as CEO"
More options
Context Copy link
They stop improving their models. If they genuinely believe there is a double digit percentage risk that what they build will destroy humanity, then they should stop building it.
More options
Context Copy link
Have the proposed means of addressing safety not be something that seems to obviously create financial/strategic benefits for them. They need to figure out a costly signal.
There is very obviously not a single thing that could ever satisfy you people. You're twisting literally calling their own sole product a risk to kill all humans in a marketing gimmick to secure regulatory capture. If they carved "No really we're serious about the risks" into their chests and then hung themselves you'd claim they were expecting to receive compensation in the afterlife.
No, I am saying they are using it as cover to slow down unsustainable capex.
And there isn't any possible evidence that could exist that could disabuse you of that impression because pausing unavoidably causes less capex spend. You're epistemically sealed against the straightforward explanation for why the people are doing the things obviously motivated by the durably stated beliefs over many years.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Open-source all models, training code and datasets after 18 months of life. That gives plenty of time for testing and to earn money on the frontier, while making it plain they don't plan to control frontier AI forever. It also makes it impossible for them to regress to less consumer-friendly behaviour after achieving market capture, which is the main thing that concerns a lot of people.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link