site banner
PaperclipPerfector  2d ago (text post)   7105 thread views

Culture War Roundup for the week of July 27, 2026

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

2
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

AI safety & open models

Yesterday, Kimi K3's weights were published on HuggingFace. We now have an open model comparable to GPT 5.5 and Opus 4.8, which were SOTA only a couple months ago.

An example of what it can do on its own: this website that imitates macOS Desktop (background). Click around, every app has lots of features, and again, this was implemented in one shot. Do you not think that's impressive?

You can, at least in theory, run this on your own equipment. Millions of dollars of equipment, sure, but far more attainable than running GPT or Opus. A medium-sized corporation or small government can.

Which presents problems: its impressive capabilities can be used for evil, without guardrails or surveillance unlike GPT or Opus. Like Fable, Kimi has already found several vulnerabilites. For examples of real evil, see how other LLMs are being used for terrorism by Boko Haram (and almost certainly other groups).

Regulation concerns

Allegedly some US officials are considering restricting US companies from using Chinese open models. Despite this claim being repeated across many outlets, I didn't actually find any evidence. However, I did find plenty of tweets criticizing the models' development and themselves, like this tweet by Treasury Secretary Scott Bessent:

We support open-source AI and the innovation it unlocks. But open source is not open season on American IP. When PRC firms conduct covert, industrial-scale distillation attacks that cross the line into IP theft, sanctions and Entity List designations will be on the table.

IP theft from Anthropic, who themselves are disregarding IP? Really?

What seems more indicative of a potential future restriction, the US government is already rushing unclear regulations for US models Fable/Mythos and GPT 5.6. If an open-weight model reaches somewhere around their capability, intuitively it would also be restricted, and Kimi is close.

Tech companies...support open weights models?

You have people like Dean Bell arguing for regulation, but many companies including Andressen Horowitz, Dell, IBM, Meta, Microsoft, and front and center Nvidia came out in support of open models, in this letter.

Key paragraphs (emphasis mine):

To be sure, open weights carry real and distinct risks. Once released, the weights are beyond the original developer's control, and modified versions are difficult to trace or reverse. But the right response to this risk is not to prohibit open weights. In a world where cybersecurity attackers use advanced AI, defenders need access to models with comparable capabilities so they can detect, simulate, and respond to emerging threats. Open models broaden defensive capability, increase transparency, and allow vulnerabilities to be discovered and remediated across many teams

In fact, openness may be one of the most important paths to AI safety and security. Relying solely on closed models is not inherently safe: they can be breached, misused, or fail in ways that outsiders cannot detect. And concentrating advanced AI capabilities behind a small number of closed models compounds that risk. It results in a small number of single points of failure, weakens competition, and leaves critical technology in the hands of a few providers. Open weight models, on the other hand, allow a broad community of researchers and developers to examine their behavior, identify vulnerabilities, develop safeguards, and improve them over time. Just as open-source software demonstrated that transparency can be more secure than obscurity, AI safety may depend on giving more people the ability to test and strengthen the models on which society relies. It allows for rigorous benchmarking and evaluation, red teaming, and protections tied to real and demonstrated harms rather than assuming that closed systems are safer by default.

Also

In shaping this ecosystem, policymakers should be careful not to conflate legitimate model-development techniques with misappropriation. Distillation, or the practice of using one model's outputs to help train or improve another, is a widely used technique for model improvement, evaluation, and validation. It reflects a long tradition of learning from, building upon, and improving existing technologies, a tradition that has helped drive innovation since the rise of the open-source software movement. By contrast, unlawful efforts to extract value from closed models raise legitimate concerns. Those concerns should be addressed through targeted legal and commercial frameworks rather than sweeping restrictions on techniques that play an important role in AI innovation.

Unlawful extraction from companies that have themselves unlawfully extracted? Again, really?

Regardless, I think overall it's a good sign.

Anthropic's position

Key quotes (emphasis theirs)

Anthropic has never advocated for a ban on open-weights models.

I do support the following three measures, which I and Anthropic have consistently advocated for:

  1. We should not sell powerful chips or chipmaking equipment to China...
  2. We should crack down on industrial-scale distillation operations...
  3. All sufficiently capable models, open and closed, should go through mandatory safety testing...

This brings me to the open letter. I agree with much of it...But I don't agree with the letter's assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers...For example, I worry that biology will have a strong attacker-defender asymmetry...Questions like this should be empirically answered by rigorous pre-release testing, not assumed in advance.

The important part is that Anthropic claims they don't want to ban open models, but want "mandatory safety testing" applied to all models. They use the threat of an AI-assisted superbug, which admittedly could cripple civilization, but so far is merely plausible. But that could effectively ban open models if it's implemented such that only closed models pass, like how "nobody can sleep under a bridge" applies equally to rich and poor but only affects the latter.

But it's also part of Plan A, proposed by the AI rationalists, who are supposed to be experts on this topic (what else have they been doing the past 10+ years?). Plan A actually argues against open models entirely, although it specifies that access to the models should be open, and all development and regulation discussions should be public. But how can we publicly develop models without making them open?

I'm curious what the Plan A authors think about Kimi K3, the open letter, and Anthropic's response; I haven't seen anything on lesswrong.com yet, although admittedly I only skimmed the front-page and recent.

My thoughts

For now, I support open weights models.

If someone comes up with a way to regulate AI development that doesn't eventually consolidate power into corrupt hands, sure. But who can be trusted? Even if future atrocities are caused by open models, they may be lighter than the atrocities committed in an alternative timeline, by a tyrant who gained power with the help of regulation, or lack of open models that prevented them.

As for "unlawful training": I still maintain the position that IP should gradually be completely abolished. AI training has already been ignoring IP, so I believe that should continue.

That includes China training on American models. I doubt China will surpass American companies if they're training on American models, especially since America has more hardware. And incentive? Come on, the American companies have enough incentive even if they had to distribute their models freely, from the dream of ASI.

I understand why large LLM developers are against distillation. It is essentially patent infringement, basically a way to copy their weights. It destroys their business model when someone can just wait until they finish the expensive training process and copy their results. But seeing as these corporations did not care about the intellectual property concerns regarding the data they used for training, I have a hard time seeing that they really have a patent to infringe upon. They neither produced nor owned the data they used for training. It follows then that they also don't really have a claim to their weights.

I understand why large LLM developers are against distillation. It is essentially patent infringement

Honestly, given how much FOSS code they ate to train these models, they should be compelled by law to release everything about them.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats