This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I don't think GLM is really that high. In my experience it may be more comparable to, like, Xiaomi V2-Flash or Minimax M2.1. Chinese ecosystem is uneven, and GLM team has massive clout thanks to their Tsinghua ties. I believe they're a bit overhyped.
It probably will have the advantage, but a) unclear what this advantage gives you practically, and b) the divergence from compounding this advantage keeps getting postponed. Roughly a year ago, Dario Amodei wrote:
Well, American companies already have millions of chips. We're nearing 2026. Multiple models trained on those superclusters already got released, RL cost is now in high millions, probably tens if not hundreds of millions for Grok 4 and GPTs, and likely Claudes. Result: Opus is not really far smarter than V3.2, an enhanced version of a year-old model Dario writes about, with total post-training costs around $1M. On some hard math tasks, V3.2 Speciale is not just like 20x cheaper per task but straight up superior to American frontier at the time of release. The gap has, if anything, shrank. Wasn't «gold at IMO» considered a solid AGI target and a smoke alarm of incoming recursive self-improvement not so long ago? V3.2-Speciale gets that gold for pennies, but now we've moved goalposts to Django programming, playing Pokemon and managing a vending machine. Those are mode open-ended tasks but I really don't believe they are indexing general intelligence better.
Maybe we'll see the divergence finally materializing in 2026-2027. But I think we won't, because apparently the biggest bottleneck is still engineering talent, and Americans are currently unable to convert their compute advantage into a technological moat. They know the use cases and how to optimize for user needs, they don't really know how to burn $1B of GPU-hours to get a fundamentally stronger model. There's a lot of uncertainty about how to scale further. By the time they figure it out, China has millions of chips too.
There is an interesting possibility that we are exactly at this juncture, with maturation of data generation and synthetic RL environment pipelines on both sides. If so, we'll see US models get a commanding lead for the next several months, and then it would be ablated again by mid-late 2026.
V3.2 was a qualitative shift, a sign that the Chinese RL stack is now mature and probably more efficient, and nobody paid much attention to it. Miles is former Head of Policy Research and Senior Advisor for AGI Readiness at OpenAI, and he pays attention, but it flew under the radar.
Another reason I'm skeptical about compounding benefits of divergence is that it seems we're figuring out how to aggregate weak-ish (and cheap) model responses to get equal final performance. This has interesting implications for training. Consider that on SWE-rebench, V3.2 does as well as «frontier models» in pass@5 regime, and the cost here is without caching; they have caching at home so it's more like $0.1 per run and not $0.5. We see how even vastly weaker models can be harnessed for frontier results if you can provide enough inference. China prioritizes domestic inference chips for 2026. Fun fact, you don't need real HBM, you can make do with LPDDR hybrids.
But all of that is probably secondary to social fundamentals, the volume and kind of questions that are economical to ask, the nature of problems being solved.
I think all of this is stages of grief about the fact that the real king is physics and we have a reasonably good command of physics. Unless AGI unlocks something like rapid nanoassembly and billion-qubit quantum computers, it may simply not change the trajectory significantly. The condition of being a smaller and, as you put it, dopey society compromises "compute advantage". Great American AI will make better robots? Well, it'll likely train better policies in simulation. But China is clearly far ahead at producing robots and can accelerate to tens of millions in little time given their EV industrial base, gather more deployment data, iterate faster, while American startups are still grifting with their bullshit targets. Similar logic applies in nearly every physical domain. Ultimately you need to actually make things. Automated propaganda is… probably not the best idea, American society is too propagandized as is. Cyberwarfare… will American AGI God really be good enough to hack Huawei clusters after their inferior Temu AGI has hunted for vulnerabilities in an airgapped regime for a few months? I think cyberwarfare is largely going dodo in this world, everyone will have an asymmetric defense advantage.
Obviously, that's still the most credible scheme to achieve American hegemony, conquer the light cone etc. etc. I posit that even it is not credible enough and has low EV, because it's an all-or-nothing logic where «all» is getting elusive.
Maybe it can't hack the servers directly if they're airgapped (though I wouldn't underestimate the power of some social-engineered fool bringing in a compromised USB) but it could hack everything around the servers, the power production, logistics, financing, communications, transport, construction. I doubt the servers even are airgapped, modern data centers are saturated with wireless signals from Wi-Fi peripherals, IoT sensors, and private LTE/5G networks. The modern economy is a giant mess of countless digital parts.
I think people underestimate the power of 'nation of geniuses in a datacentre', even without any major breakthroughs in physics, I think mere peak human-level AIs at scale could wipe the floor with any technological power without firing a shot. In cyber there is no perfect defence, only layers of security and balancing risk mitigation v cost. The cost of defending against a nation of geniuses would be staggering, you'd need your own nation of geniuses. Maybe they could find some zero-day exploits. Maybe they could circumnavigate the data centre and put vulnerabilities in the algorithms directly, find and infiltrate the Chinese version of Crowdstrike? Or just raze the Chinese economy wholesale. All those QR code payments and smart city infrastructure can be vulnerabilities as well as strengths.
China's already been kind of doing this 'exploit large high IQ population' with their own massive economic cyberwarfare program. It works, it's a smart idea. 10,000 hackers can steal lots of secrets, could 10 million wreck a whole country's digital infrastructure? You may have read that short story by Ci Xin Liu about the rogue AI program that just goes around causing human misery to everyone via hacking.
I believe that the physical domain is trumped by the virtual. Even nuclear command and control can potentially be compromised by strong AIs, I bet that wherever there is a complex system, there will be vulnerabilities that humans haven't judged cost-efficient to defend against.
I think it's funny that we've both kinda swapped positions on AI geopolitics over time, you used to be blackpilled about US hegemony until Deepseek came along... Nevertheless I don't fully disagree and predicting the future is very hard, I could well be wrong and you right or both of us wrong.
Eh, I think Pokemon and vending machines are good tasks. It's long-form tasks that matter most, weaving all those beautiful pearls (maths ability or physics knowledge) into a necklace. We have plenty of pearls, we need them bound together. And I don't think 3.2 does as well as Claude Code, at least not if we go by the 'each 5% is harder than the 5%' idea in these benchmarks.
You misunderstood my point. I am saying that hacking as such will become ineffectual in a matter of years. Automated SWEs make defense drastically advantaged over offense due to information asymmetry in favor of the defender and rapid divergence in codebases. This “superhacker AGI” thing is just lazy thinking. How long do you think it takes, between open source AIs that win IOI&IMO Gold for pennies, and formally verified kernels for everything, in a security-obsessed nation that has dominated image recognition research just because it wanted better surveillance?
A very American belief, to be sure.
IMO Gold is an important signal but not that significant in and of itself, again, it's longer-term capabilities that matter.
To proof a complex system against hacking, you'd need ASI. This is a superhuman feat, no humans have ever written a provably secure system that actually does useful work as opposed to just being a toy proof of concept.
By the time these kernels come out and are deployed, it's pointless to hack the datacentre.
Thankfully, verifying proofs is easier than generating them, so we're about to find out if this is true.
Even finding all the things you'd need to secure is a nightmarish task. The CPU's physical structure, the microcode, the BIOS, the lower levels of the OS, a myriad of applications... You'd need a completely accurate, top to bottom model of the whole system: thousands of devices, routers, OSI... You'd then need to rewrite all of it while somehow maintaining proper functionality. Have fun updating the ROM of the management engine! Good odds there are physical flaws in CPUs that either humans are too dumb to uncover or were put there by intelligence agencies for spying purposes, so even if you do all that it still isn't sufficient.
ASI is a bare minimum requirement. Probably ASI + a whole new generation of chips is needed.
You're overrating the irreducible combinatorial complexity (especially given that we can improve modularity when software is this cheep) and underrating the computational efficiency. We're in the regime where 1M of near-frontier tokens goes for $0.3, caching-enhanced prefill is almost free. $300 for billion, $300K for trillion, $300M for quadrillion, $30B for 100 quadrillions. Will likely fall 10x within a year while performance creeps towards peak human programming skill, again. Bytedance is currently processing 50 trillion a day for Doubao, they have a near-Gemini 3 multimodal model (Doubao Seed 1.8).
How much is the entire specification of, say, a Huawei server's full hardware-software stack, all relevant documentation, everything? Maybe a few terabytes if we're obsessive. Blow that up 1000x for experiments and proof generation. A few quadrillions, plus the costs of software execution.
How much is invulnerability to ASI hacking worth? It's worth pretty much everything, given that the US is en route to have ASI and is psychotically attached to its finance-powered hegemony.
What is the alternative? Pretty much just preemptive nuclear strike.
They will be forced to do this.
This is not a 'more tokens' task but a 'more intelligence' task, requiring ultra-long horizons and qualitatively superhuman ability.
It would be far easier to make a fun AAA game. It would be far easier to write a LOTR-tier book series. Humans have at least done those things in the past, individually or collectively. Nobody has ever made an unhackable, actually useable system. A system will have to be considered in its entirety, AI training is complex and can't just be reduced to small pieces to be secured independently of eachother. At minimum all this will have to run together performantly. That is no small feat and cannot be achieved monkey-typewriter style.
If it were merely about spending a few billion dollars and a lot of programmer time wouldn't the Pentagon/NSA be totally secured against cyberattack by now? They're not, even state actors can't do this.
I can't understand the world you're proposing, where Chinese AIs are smart enough to shield the entire Chinese training stack but US AIs are not smart enough to hack them before the shield can be completed. The trend suggests that at any given point in time, US AIs are smarter than their Chinese siblings. So there will be a gap between when this defence-shield can be completed and when the US could launch its attack. The US will likely retain a qualitative and quantitative advantage in AI this whole time.
If the Chinese AI can see 'this software is subtly vulnerable to infiltration, I'll write this replacement to secure it and then fit it in with the rest of the stack while still maintaining performance' why can't an American AI see 'this software is subtly vulnerable to infiltration, I'll infiltrate and exploit it before the upgrade process is complete?'
Why is my 'superhacker AGI' lazy thinking but not your 'superhuman perfect defence + performant AI training stack code-writer AI' not lazy thinking? I agree that it's possible in principle but the former will come before the latter.
If China has 100 quadrillion tokens, then the US will have yet more, they have more compute after all. I doubt Doubao's tokens are worth as much as Gemini or OpenAI's, 'token' could be anywhere on the curve of intelligence and cost.
Maybe the US decides not to hack, maybe somebody cuts a deal, maybe Trump makes some inexplicable decision or maybe AGI isn't a big deal. But I don't see your scenario happening.
Furthermore, there are still hardware issues to consider. There are probably many unfixable flaws that humans aren't smart enough to find like these: https://arstechnica.com/information-technology/2020/03/5-years-of-intel-cpus-and-chipsets-have-a-concerning-flaw-thats-unfixable/
State actors have trash talent, for reasons discussed earlier. A few outliers, but at this point Google has like 100x NSA's capability. This needs Google's capability at 100x the scale of labor. Where we differ is that I think this doesn't require 100x Google's capability.
Because the American AI only has access to internet-connected surfaces, is rate-limited and needs to avoid detection before breach, whereas Chinese AI has root access to the entire stack plus documentation and source code and can examine those subtle vulnerabilities in a massively parallel manner and at its leisure.
The point you keep missing is that at every point, information and time asymmetry exists in favor of the defender, which I think makes up for the observed qualitative and quantitative advantage of the attacker.
I maintain that this is a lazy hope for having an unfair advantage and scarcely different from "we could bomb three gorges dam, we just choose not to".
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link