site banner
PaperclipPerfector  9d ago (text post)   22256 thread views

Culture War Roundup for the week of August 17, 2026

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

2
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

Fetishizing algorithmic design is, I think, a sign of mediocre understanding of ML, being enthralled by cleverness. Data engineering carves more interesting structure into weighs.

I misremembered the exact quote even if I captured the core of overall discussion on Feb 26th 2025. It still being a transformer-decoder arch is not what is really under contention. Your overall stance was that data + compute is all that really mattered and that worrying about architecture, or seeking to make architectural improvements demonstrated a fundamental poor understanding of ML. The problem with taking such a provocatively maximalist position is that now you must defend it in the future when architectural improvements to the transformer arch are made and which you call innovative. Does MoonshotAI have a mediocre understanding of ML? Or instead were you wrong?

You act like the arc of history is settled when you make comments like that. And like every pronouncer of "History has ended, all discoveries that will be made have been made", the march of progress leaves you blacked in the soot of arrogance. It's frankly an anti-science position. Algorithm/Architecture design is as much a core part of ML as is data and compute. The transformer, as it was invented, is unlikely to be the endpoint of ML arch research, just has the CNN, or LSTM were not the endpoint of ML research a decade prior. The transformer of today is different from the transformer of 2018, and I would not bet against the transformer of 2036 being different than that of today. I would not bet against the core elements of the transformer are metastasized into another architecture in 2046. I don't have a crystal ball, I don't know, but I do know planting a flag and saying "The Transformer has solved all architecture problems, no improvement of consequence will ever be made", and then calling anyone who disagrees with you an idiot, is liable, as it has now, to required you to defend increasingly convoluted arguments.

First, you did say that real big boy algorithmic research means leaving this entire basin, and dismissed the kind of innovation I praise in K3 as tinkering with the assembly level (that's not all DeepSeek did, of course, but that was your understanding; and spiritually you were right, it's all about pumping compute more efficiently through a Transformer). I didn't remember that, but it does reinforce my point about Kimi. You said:

Regardless of whether transformers are a dead-end or not, the current approach isn't doing new science or algo design. Its throwing more and more compute at the problem and then doing the Deepseek approach of finetuning the assembly level gpu instructions to exploit the compute even better so you can throw more compute at it. I doubt, Hinton, Goodfellow, LeCunn, Schimdhubber et al. have any desire to do that. Maybe if xAI did something revolutionary like leave the LLM space or introduce a non-MoE-Transformer model for AGI, then talent of that caliber might want to work there. Currently they exist so Elon can piss all over Altman.

Then I clarified my claim:

– I meant concretely that this is why leading companies now prioritize creation of training signal sources, that is: datasets themselves (filtered web corpora, enriched and paraphrased data, purely synthetic data, even entirely non-lingual data with properties that induce interesting behaviors), curricula of datasets, model merging and distillation methods, training environments and reward shaping – over basic architecture research, in terms of non-compute spend and researcher hours; under the (rational, I believe) assumption that this has higher ROI for the ultimate goal of reaching "AGI", and that its fruit will be readily applicable to whatever future algorithmic progress may yield

Now let's look at what the Chinese are actually doing. Their strongest model right now is arguably GLM 5.3. What is GLM 5.3? A basic DSMoE reusing DeepSeek's discarded architecture experiment ["DeepSeek Sparse Attention-prototype"] from October 2025, with one small twist. How did it become so strong? They're very blunt about this:

Scaling post-training is all we did for GLM-5.3. With GLM-5.2 we built the stack: IndexShare for efficient long-context processing, SAO for RL on long-horizon tasks, and slime for large-scale asynchronous training — all running on the long-horizon task environments we have been accumulating. Over the past month we kept scaling on this stack: more environments, more diverse tasks, and more compute spent training on them.
Today we are releasing GLM-5.3. It uses the same base model as GLM-5.2 — every gain comes from post-training.

They explain some aspects of how they did it, might illuminate why scoffing at "mere data engineering" is misguided.

What's the second/equally strongest Chinese model? Kimi K3. It's much more innovative in architecture, but their attention still depends on MLA (invented by DeepSeek in early 2024). No MLA model is this strong. How did it come so far? See image, which illustrates nicely a part of what I was going on about with my breakdown. (Edit: seems like we don't have images. pages 14-16 in the tech report).

Meanwhile, DeepSeek itself went on to redesign attention the third time (fourth if we count the apparently unsuccessful NSA project), to wring even more capacity out of their limited compute, and now for all their sophisticated V4 architecture they are, as @dailydogma tells me with a sneer, "a second rate company in China". What's their most impressive recent result? Flash-0731: «We’ve massively upgraded its Agent capabilities—benchmark scores are now far surpassing the V4-Pro-Preview… DeepSeek-V4-Flash-0731 keeps the exact same model architecture and size as the preview version.» (Recently updated Vision-Exp is the same model with a vision encoder). Just more post-training, just better training signal. If they reach the domestic frontier again, it'll be because they do this more and better. Architecturally, they are ahead, and it's just not doing enough for them.

Grok itself is very competitive now, largely because Elon has bought Cursor, which had a lot of valuable data and expertise on post-training. We don't know its architecture, but from rumors and what I can infer (high cache hit costs, for starters), it's very banal, probably behind all these Chinese models. Inkling from Thinking Machines is clearly banal («The MoE design largely follows DeepSeek-V3»), though it makes some small departures which are basically judgement calls. And these are researchers from frontier American labs.
I could go on (eg this small model from a third tier lab does surprisingly well on ARC-AGI 2, and it's just DSA + SWA again, and uses a bit different RL algo and data). The bottom line is, architecture really does not decide peak model intelligence, and innovations here are overwhelmingly about economics of inference, and the high-leverage research is all happening on the training signal side.

But China is China. I believe my point was much more true for large American companies we were discussing, who are not so compute-constrained. They'll build very strong models with conservative algorithms, and then use those to disassemble all published tricks, make new ones and overtake the crafty Chinese on efficiency too. That's the plan, at least (I don't know how close they are to doing this; GPT 5.6-Luna suggests they are not very far). For them, investing more effort into algo research over data is plainly an opportunity cost.

Maybe you still think that True Geniuses like Hinton or Goodfellow would be disappointed by this. If that is so, I say they were geniuses in a small and uncompetitive pond, and the current crop of talent knows better. It certainly knows better than LeCunn, who by the way got ousted out of Meta by the Pinoy slavedriver Wang we've discussed back then. (You said: «Maybe you can compete with ScaleAI, they do data engineering. Definitely the top AI research company.») Anyway, I'm not walking back shit, my point stands.

The transformer, as it was invented, is unlikely to be the endpoint of ML arch research, just has the CNN, or LSTM were not the endpoint of ML research a decade prior. The transformer of today is different from the transformer of 2018, and I would not bet against the transformer of 2036 being different than that of today.

If we're still doing Transformer of any kind in 2036, that'll be pretty wild. It'll suggest that even superhuman AI with like a yottaflops for parallel experiments can't find a better primitive than a bunch of Googlers found in 2017 by going through literature and thinking at it. I wouldn't be so optimistic as to predict that. I am very secure in claiming that the priority on data remains rational and empirically backed from the perspective of reaching AGI faster, and it's more rational the more resources a company has; and that people who try to wriggle out of this reality with clever architectures will flounder (case in point: SakanaAI).
You suggested sarcastically that I found an AGI company. Honestly, I don't think that my vision, as outlined here and before, has enough alpha to get anywhere with a yet another company; I believe that AGI is now a resource-intensive heavy industry field, kind of like fracking. But in fact, many people who thought they know better did just that! Have you heard anything from Keen lately? What's your favorite non-transformer lab? What do you think of LeCun's Advanced Machine Intelligence, would you bet they ship anything competitive by 2028? How about you start one?

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats