DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
DaseindustriesLtd
late version of a small language model
21d ago
GLM-4.7 for instance, supposedly it has stats comparable to Opus 4.5.
I don't think GLM is really that high. In my experience it may be more comparable to, like, Xiaomi V2-Flash or Minimax M2.1. Chinese ecosystem is uneven, and GLM team has massive clout thanks to their Tsinghua ties. I believe they're a bit overhyped.
Won't the US enjoy a quantitative and qualitative superiority in AI though, based on the compute advantage, through to at least the 2030s?
It probably will have the advantage, but a) unclear what this advantage gives you practically, and b) the divergence from compounding this advantage keeps getting postponed. Roughly a year ago, Dario Amodei wrote:
R1, which is the model that was released last week and which triggered an explosion of public attention (including a ~17% decrease in Nvidia's stock price), is much less interesting from an innovation or engineering perspective than V3. It adds the second phase of training — reinforcement learning, described in #3 in the previous section — and essentially replicates what OpenAI has done with o1 (they appear to be at similar scale with similar results)8. However, because we are on the early part of the scaling curve, it’s possible for several companies to produce models of this type, as long as they’re starting from a strong pretrained model. Producing R1 given V3 was probably very cheap. We’re therefore at an interesting “crossover point”, where it is temporarily the case that several companies can produce good reasoning models. This will rapidly cease to be true as everyone moves further up the scaling curve on these models. …
Making AI that is smarter than almost all humans at almost all things will require millions of chips, tens of billions of dollars (at least), and is most likely to happen in 2026-2027. DeepSeek's releases don't change this, because they're roughly on the expected cost reduction curve that has always been factored into these calculations. […] This means that in 2026-2027 we could end up in one of two starkly different worlds. In the US, multiple companies will definitely have the required millions of chips (at the cost of tens of billions of dollars). The question is whether China will also be able to get millions of chips.
Well, American companies already have millions of chips. We're nearing 2026. Multiple models trained on those superclusters already got released, RL cost is now in high millions, probably tens if not hundreds of millions for Grok 4 and GPTs, and likely Claudes. Result: Opus is not really far smarter than V3.2, an enhanced version of a year-old model Dario writes about, with total post-training costs around $1M. On some hard math tasks, V3.2 Speciale is not just like 20x cheaper per task but straight up superior to American frontier at the time of release. The gap has, if anything, shrank. Wasn't «gold at IMO» considered a solid AGI target and a smoke alarm of incoming recursive self-improvement not so long ago? V3.2-Speciale gets that gold for pennies, but now we've moved goalposts to Django programming, playing Pokemon and managing a vending machine. Those are mode open-ended tasks but I really don't believe they are indexing general intelligence better.
Maybe we'll see the divergence finally materializing in 2026-2027. But I think we won't, because apparently the biggest bottleneck is still engineering talent, and Americans are currently unable to convert their compute advantage into a technological moat. They know the use cases and how to optimize for user needs, they don't really know how to burn $1B of GPU-hours to get a fundamentally stronger model. There's a lot of uncertainty about how to scale further. By the time they figure it out, China has millions of chips too.
There is an interesting possibility that we are exactly at this juncture, with maturation of data generation and synthetic RL environment pipelines on both sides. If so, we'll see US models get a commanding lead for the next several months, and then it would be ablated again by mid-late 2026.
V3.2 was a qualitative shift, a sign that the Chinese RL stack is now mature and probably more efficient, and nobody paid much attention to it. Miles is former Head of Policy Research and Senior Advisor for AGI Readiness at OpenAI, and he pays attention, but it flew under the radar.
But if AI/AGI/ASI is a big deal, then America enjoys a decisive advantage. Doesn't matter if China has 20 AGI at Lvl 5 if the US has 60 at Lvl 8. I think a significantly more intelligent AI is worth a lot more than cheaper and faster AI in R&D, robotics, cyberwarfare, propagandizing, planning.
Another reason I'm skeptical about compounding benefits of divergence is that it seems we're figuring out how to aggregate weak-ish (and cheap) model responses to get equal final performance. This has interesting implications for training. Consider that on SWE-rebench, V3.2 does as well as «frontier models» in pass@5 regime, and the cost here is without caching; they have caching at home so it's more like $0.1 per run and not $0.5. We see how even vastly weaker models can be harnessed for frontier results if you can provide enough inference. China prioritizes domestic inference chips for 2026. Fun fact, you don't need real HBM, you can make do with LPDDR hybrids.
But all of that is probably secondary to social fundamentals, the volume and kind of questions that are economical to ask, the nature of problems being solved.
In a compute drought, the compute-rich country is king. In an AI race, the compute-rich country is king. China would be on the back foot and need to use military force to get back in the game.
I think all of this is stages of grief about the fact that the real king is physics and we have a reasonably good command of physics. Unless AGI unlocks something like rapid nanoassembly and billion-qubit quantum computers, it may simply not change the trajectory significantly. The condition of being a smaller and, as you put it, dopey society compromises "compute advantage". Great American AI will make better robots? Well, it'll likely train better policies in simulation. But China is clearly far ahead at producing robots and can accelerate to tens of millions in little time given their EV industrial base, gather more deployment data, iterate faster, while American startups are still grifting with their bullshit targets. Similar logic applies in nearly every physical domain. Ultimately you need to actually make things. Automated propaganda is… probably not the best idea, American society is too propagandized as is. Cyberwarfare… will American AGI God really be good enough to hack Huawei clusters after their inferior Temu AGI has hunted for vulnerabilities in an airgapped regime for a few months? I think cyberwarfare is largely going dodo in this world, everyone will have an asymmetric defense advantage.
Obviously, that's still the most credible scheme to achieve American hegemony, conquer the light cone etc. etc. I posit that even it is not credible enough and has low EV, because it's an all-or-nothing logic where «all» is getting elusive.
DaseindustriesLtd
late version of a small language model
21d ago
When the japanese and south koreans copied and then bested american cars and german optics and swiss watches, did the americans and germans and swiss subsequently sink into poverty?
I don't know what the Swiss thought of it, but Americans absolutely had a psychotic meltdown about Japanese competition. I think you don't get how intoxicating the sense of supremacy is. Switzerland is just a nation, its manufactures are just manufactures, it operates on the logic of comparative advantage. Americans have an ideological stake in being Number One.
DaseindustriesLtd
late version of a small language model
21d ago
They spend very little on social safety nets. the end result being that Chinese household consumption is something like 40% of gdp vs 65% in the states.
I simply don't think this is even true, it's more self-serving imperial propaganda to present failures as a moral choice. Most of your consumption value is rent-seeking like high rents. Chinese consumption is not that low, read this. Even Chinese safety net is not as low as is often said, it's on par with other middle-income nations.
You're not doing anyone a favor by being corrupt.
DaseindustriesLtd
late version of a small language model
21d ago
I think this is somewhat incoherent.
Americans are happy to let other nations lead in some industries and rely on them long term. We're happy to buy Korean appliances, Japanese cars, European fine crafted goods and Columbian cocaine. If you want to build out a niche the American empire is happy to let you have it and integrate into the global family. This is not how China acts.
Your narrative is a bit out of date. How will Europeans pay for Chinese imports if China has no need of their exports (in «fine crafted goods», services or anything)? Maybe they just won't, if China can do all that fine crafting cheaper and better. But they will face the same issue with American imports, indeed already are facing:
The Trump tariffs have already hit German exporters hard: over the first nine months of the year, their US exports plunged by 7.4 per cent.
But the prospects in China are if anything even bleaker, creating a “China shock” that is now biting into the bottom lines of globally successful German companies.
Since the start of 2025, Germany is now running a trade deficit in capital goods with China over a rolling 12-month period. That is a first since records began in 2008. Chinese machinery exports to Europe roughly doubled to around €40bn in over six years and may reach €50bn this year, according to industry association VDMA.
Trump’s haphazard trade policies are hurting German industrialists much more than the 15 per cent headline tariff accepted by the EU in July suggests.
A month after the controversial deal, the US expanded an existing 50 per cent duty on metal components to more than 400 additional product categories, including motorbikes, railway cars, cranes and pumps. The charges on steel, alloy and copper come with complex disclosure rules and threats of heavy fines for incomplete declarations.
This hit German companies of many stripes. Farming equipment maker Krone Group, for example, based in Spelle in Lower Saxony, was forced to temporarily halt its US-bound production. The extra tariffs on metal were “very shocking”, recalls Bernard Krone, chair of the family firm with €2.4bn in sales. US farmers will face hefty price increases which could damp demand, he predicts.
Yet while selling goods to the US has become more difficult for German industrialists, competing with China’s rapidly ascending industrial might presents an even greater challenge.
Goods coming out of China are no longer cheaply made, lower-quality knock-offs, if they ever were. “Most of what German Mittelstand firms do these days, Chinese companies can do just as well,” says Thilo Köppe, partner at German consultancy firm Vindelici Advisors who worked in China for more than a decade.
American Hegemony is not about building some happy global family with a division of labor. From software down to extractive industries, American Empire wants to be like Emperor Qianlong said: «our Celestial Empire possesses all things in prolific abundance and lacks no product within its own borders. There was therefore no need to import the manufactures of outside barbarians in exchange for our own produce». Currently, the Chinese economy is pretty export-dependent, but Xi would prefer it to be otherwise – Dual Circulation is a big pillar of his policy, and in purely thermodynamic terms, if literally everything is cheaper in China, you can ignore standard macroecon, largely eschew exports, subsidize domestic demand and make Qianlong's boast a reality.
The main difference is that China got there with industrial policy and human capital, and you're trying to get there with tariffs and coercion and a Wunderwaffe. But the end result is the same for non-live players.
Back then, I asked for what the end game of AGI race is, and you said:
If it is powerful enough to actually do high level engineering work then it instantly obviates China's other major advantage in having a big workforce. If it scales all the way to AGI then forget about it, winning that race is all that matters. Winner gets to be the center of commerce and yes some latitude that comes along with having the most powerful military.
A center of commerce in what sense? The US currently has a very low fraction of international trade in GDP – around the level of Pakistan. The US wasn't a global center of commerce during its ascent either. You pat yourselves on the back for importing some junk but it's not really spreading a lot of your wealth around, it's only large in absolute terms. China is simply already doing what you want to do once you get «AGI», and by 2038, if AGI plans pan out, your narrative will be laughably quaint.
Moreover, what's wrong with that? Both nations are large, decently situated and can, in theory, produce all goods in prolific abundance within their own borders more economically than imports would be; and China is entitled to a larger and more diverse internal market on account of population size. There are some hard natural endowments – Australia has more accessible mineral wealth, Atacama desert has excellent solar resource, I don't know – but commodities are cheap. Maybe they'll become less cheap? What remains scarce after labor and R&D are commodified? Land? Copper? Wombs? We need to think of how the world would operate when major nations are capable of industrial autarky, because modulo some Butlerian Jihad we will have to deal with it anyway.
Finally, what exactly is your concern? The US imports more than it exports not out of some moral commitment to subsidizing globalism, but just because it has very credible IOUs to sell. You basically print USD and export inflation. The EU can't do that. China can't do that. Chinese industrial competence doesn't have a direct effect on that, they cannot deny you the ability to print paper and buy Columbian cocaine. Trying to rationalize the take – you deserve hegemony because what, it'll mean unquestionable military supremacy, hard guarantee of your IOUs, and therefore indefinite ability to exchange goods for paper?
Yeah, I think it's less sustainable than «China is a very large and very productive autarkic country». They've been exactly that for centuries, and the world managed fine. In the limit of this trajectory, they will only need to export enough to cover the raw commodities imports necessary for their internal economic activity. That's not a lot, in dollar terms. The more interesting question is what else we all will be trading in 2038.
DaseindustriesLtd
late version of a small language model
21d ago
That's a funny perspective to me. Russians have negligible industrial base, atrophied Soviet military-industrial complex, and mainly export stuff like oil, gas, grain and fertilizers. We also simply don't have many smart people remaining, and instead have a population of jingoistic TV-watching cattle that needs steady supply of copium in the face of a protracted war that's going badly. Of course we have televised fantasies about Wunderwaffes. The US is, for all its grandeur, similarly a corrupt Soy and Gas Empire that struggles with building physical things relative to its consumption and ambition, we see whining about the Rust Belt, Powerpoints with 6th gen fighter jets and «Trump class battleships», and the whole AGI project is supposed to restore the claim to primacy. And even in Ukraine, Americans heroically grappled with the costs of exporting this or that long range wundermissile or Smart Shell, and thought that their Wunderwaffes like HIMARS, or Palmer Luckey's gadgets, will make more of a difference than commodity drone parts from Aliexpress procured by both sides. They did not.
China is the factory of the world and the source of almost all new process innovation and, say, the bulk of Californian top patent holders. The Chinese are not advertising their ion implantation techniques, procurement plans, fabs, they are quietly doing business and actively avoid international publicity on these matters. No, I don't think this is a Wunderwaffe. Yes, we'll see what they can actually produce in 2026.
Alternative explanation: Jensen Huang won the game of "be the last person to talk to Trump", since he knows Trump is a waffling buffoon and Huang just wants to maximize Nvidia's stock, US security concerns be damned.
This is still denialism of the erosion of fundamentals, I think. Classic stabbed-in-the-back-by-Jews [of Asia] doctrine. Huang founded Nvidia over 30 years ago, I don't believe he's a petty merchant optimizing for quarterly reports.
Minor update on the US-PRC tech competition.
Culture war significance: it matters for the grand strategy understanding and the narrative of the US as the Main Character of History. Personally, I had stopped regularly engaging on this forum when it became clear that the US is, in fact, not such a Main Character (at least for the moment), but just a great power with massive momentum and cultural influence. Not being American, I mainly only care about American cultural affairs insofar as they have global spillover effects. Local legislation news and woke-MAGA strife are overwhelmingly noise for the world, unless they reach some critical volume like peak woke or BLM did. Some American tech, and related politics, is very much not noise. The chip war in particular is very high-signal, so I follow it closely.
It seems something happened behind the scenes after those events in October, when the US Department of Commerce went with the Affiliate Rule, China retaliated with REE+ export controls, and soon enough, by November 1, we've got the usual Trump style Deal. (There's also a subplot with Nexperia/Wingtech, that demonstrates Chinese supply chain power and European ineptitude again, with a similar outcome of the Western actor retreating). Suddenly, on Dec 8, we get the news about Trump permitting the sales of H200 to China (context and understandable rationalist perspective here). China reacts somewhat paradoxically, if your theory of their mind is just «they're desperate for our chips» – as per the FT, «Companies seeking to purchase the H200 would need to submit a request explaining why they cannot use domestically produced chips and undergo an approval process», in continuation of their earlier scrutiny, rejections and negative publicity directed at H20s.
10 days later Reuters breaks the news – which were not quite news for those in the know – about Chinese successes with their EUV effort. The article is somewhat confused, as almost all reporting on Chinese AI and IC tech is; from my private sources, the situation has already moved further on multiple components, like optics and metrology.
What I want to emphasize here is that it's not just trivial «industrial espionage» or IP theft. Their light source project is led by former ASML head of light source technology and «Light source competence owner for metrology in ASML research» Lin Nan. I think that he returned not just for money, nationalism or career opportunities, but because China offered him a more ambitious challenge – he seems interested in solid state lasers, which ASML, constrained by market incentives more than strategic considerations, gave up on. For sure, straightforward IP theft also happens - CXMT's DRAM/HBM progress is apparently propped up by Samsung IP which was, well, illicitly transfered by former employees. And there's very substantial domestic talent pipeline, though people are prone to dismiss their patent/paper counts; they lack brand power, «Changchun Institute of Optics» doesn't have the same zing to it as Zeiss, though you may see it in the news soon.
All in all, China is moving far faster than even I imagined. Now we get reports – straining my credulity, to be honest – that ByteDance doesn't expect Nvidia to sell move than a few hundred thousand cards in China, not because of any trade barriers from either side, but because adequate domestic competition will come online in mid-2026 already. Almost certainly it'll be worse and less power-efficient, at least. But clusters with Chinese hardware are eligible for electricity subsidies, and that may be enough to tip the scales? This logic is corroborated by the surprisingly low leaked price of H200s – just $200.000 for an 8-card module (not sure if that's before of after 25% Trump Tax, but in any case very low, maybe lower than in the US proper, at least pre-tax). Meanwhile that's 5 times more bang for the buck than H20s offered. On the other hand, for now Nvidia is selling old stock; new production is being discussed, but at this rate I don't expect the price to increase. One can reasonably ask if this makes any sense, given that the demand in the US outstrips supply. I think it does, both for complex strategic reasons (mainly ecosystem lock-in, which is in fact a big deal, as I explain here) and simply because the US AI market is becoming a very convoluted circular Ponzi scheme where Nvidia de facto subsidizes companies to buy Nvidia wares. That's more of a potential market meltdown recipe than a revenue source. H200 sales to China, for what it's worth, unambiguously pull in dollars, and both Jensen's fudiciary duty and Trump's deficit-slashing mandate (and to be blunt, likely Trump's corruption) create a strong incentive to greenlight them.
Anyway, what looked like Chinese bluffing and negging at the time the sale of H20s was debated looks more and more like genuine, coherent industrial policy. China is pretty sure it'll have sovereignty in the entire stack of AI development, soon enough, that it will even be capable enough to export its AI hardware products, and the US is acting as if that is likely true – as if the competition is about market share and revenue. They are obviously compute-constrained right now, so DeepSeek V3.2 only catches up to around GPT-5 level, with the usual complaints in the paper. They don't appear to mind this enough to bow and scrape for more American chips at any cost. A large component here is that what they need, they can often rent overseas openly –
a data center near Osaka, operated by Japanese marketing solutions firm Data Section, is effectively dedicated to Tencent. This data center houses 15,000 of Nvidia’s Blackwell (B200) GPUs. Tencent secured access to these GPUs for three years through a $1.2 billion (approximately 1.8 trillion Korean won) contract with Data Section via a third-party entity. Data Section plans to establish additional data centers in Sydney, Australia, with over 100,000 Nvidia GPUs, also primarily serving Tencent.
– but I think it's primarily about confidence in the domestic supply chain.
Long before all these events, in September, we had a debate with @aquota here, when the topic was selling China relatively worthless H20s. (For my previous take on H20s specifically see here).
He argued:
This lock-in effect is just nonsense and has not worked for literally a single firm that has sold out to china. China is not going to forego building their own echo-system and hasn't for any other sector they've found strategically important. […] Our one chance at dominance in this sector is remaining ahead in AI and reaping compound interest on that lead whether it's AGI or simply accelerated AI and chip development. If it's not enough then I just don't buy this fantasy that selling out now is going to give us a better seat in the future.
To which I've replied:
As I've said before, "China" is not omnipotent and cannot create an ecosystem solely through political will and subsidies, they've been trying for decades and it hasn't been working so long as Nvidia was the obvious superior choice. Even now, nobody wants to use CANN if afforded the chance. I think this is how Jensen views this: he's straightforwardly fighting as the CEO of American company Nvidia, not just for line going up in quarterly reports but for enduring global dominance of his stack.
… personally, I believe this [AI race theory] is all deluded and very much in the spirit of last days of Nazi Germany. Both sides will have adequate AI to increase productivity, both will have "AGI" at around the same time, you're not going to have some dramatic inflection point, you will not leave them in the dust as a military or economic power, you'll just slow down global economic growth somewhat, and in the long run end up poorer and have a smaller slice of the global market. That's all.
[…] I guess this is the crux. In your world, where unipolarity is the default trajectory, it makes perfect sense to cling to Pax Americana and play negative-sum games hoping to outlast the opposition. Like, what is the alternative, capitulation, suicide? In my world, China is basically guaranteed to not only exist in 30 years but have comprehensively stronger economy than the US plus closest allies, no matter what you sell or don't sell, buy or don't buy. And the US will have to figure out how to exist, and exist well, without boons of global strategic superiority, in a bipolar world, and hopefully remaining a hegemon in its own backyard. That figuring out has got to begin now.
It seems to me that my read on the situation from back then, both the big picture and its implications for compute strategy, is now shared by both the USG and the CPC. The former is trying to regain its position and revenue in the Chinese GPU market and slow down Huawei/Cambricon/Kunlun/etc. ecosystem development by flooding the zone with mature Nvidia chips that will be adopted by all frontier players (eg DeepSeek again – they have a deep bench of Nvidia-specific talent and aren't willing to switch to half-baked Ascend CANN). The latter is more worried about preventing the US from doing that than about gaining moar FLOPS in the short run.
In conclusion, I want to congratulate Americans again with having found a true peer, for the first time since the decline of the British empire. Germans, Japanese and my own people had failed to provide enough stimulation, so Americans have grown lonely and fat at the top.
Aquota said:
surely you understand the "equals across the sea" isn't an option on the table. That isn't what is in store if we give up all our advantages in this sector.
I do not, in fact, "understand" this. Like, that may be the case and we'll just have Pax Sinica. I'm okay with it but I'm not Sinophilic enough to expect it. Even reduced to "just a great power", the US is poised to remain a historical force.
For now the loss of the indisputable Main Character status is being processed traumatically, with anger, denial and exaggeration of the costs of that loss if it were to really happen. But as its reality sinks in, this trauma may become fertile grounds for some cultural Renaissance in the United States. Less capeshit, more self-awareness. I may even come to care about it for reasons aside from global consequences.
…Of course, we can still entertain the hypothesis that all of the above is some interesting ephemera and this final dash of the Chosen Nation towards AGI-powered Rapture and completion of history is the real story of the times. I won't completely discount it, we shall see.
DaseindustriesLtd
late version of a small language model
24d ago
I suspect that these antisemites would rather blame AI on Jews in the sense that it's a “Jewish trick” to extract money while feeding us “goyslop” under the pretenses of building AGI.
Those who take AI seriously are very terrified of the implications.
Amodei siblings are also connected with Holden Karnofsky, of OpenPhil, which can be reasonably described as some kind of New World Order project (albeit, it seems, grossly unsuccessful).
And there was no pre-WWII history of anti-Jewish pogroms, expulsions, or legal discrimination against Jews in China
This is not entirely correct. For sure, generally Jews were treated well in China (eg in Harbin they only were harassed by local Russian fascists, somewhat humorously not by the Nazi-aligned Japanese who sought to resettle them in Japan, on grounds of taking Protocols of the Elders of Zion at face value and anticipating high ROI from alignment with the Jewish people). But Kaifeng Jews were at the very least forced to assimilate, and probably abandon endogamy.
Of course, China is so vast and has seen so many different peoples that all of that is a complete nothingburger in their national consciousness. Anecdotally, I have the impression that they thought well of Jews (even of negative stereotypes), assuming that this is NGMI whining of whites who complain of Chinese shrewdness and intelligence in the same manner. Chinese themselves experience relationships somewhat similar to Medieval Jewish-Gentile ones across the broader Sinosphere, eg in Malaysia where they are the educated, clannish middleman minority with financial assets but without hard power. Politically, the PRC is consistently pro-Palestinian but it doesn't have much of an actionable component or popular purchase, and the Israel Question is folded into the broader competition with America, often with this lazy Marxist spin about Israel as the bulwark of global imperialism for those who want an ideological case against he US.
On the other hand I've been told by Mainlanders that China got really redpilled on the JQ after the reports of starvation in Gaza. They take starvation extremely seriously, and then pattern-matched the whole post-Oct 7 dynamic onto Japanese occupation. That may color perceptions going forward.
DaseindustriesLtd
late version of a small language model
26d ago
That's some quite superficial thinking. I think you really underestimate how far we could have gone with biological and chemical weapons, for one thing. Modern wars are in no way maximally brutal yet. Big states do avoid the logical endpoint of a race to the bottom where all personnel on both sides is writhing in agony within 72 hours.
DaseindustriesLtd
late version of a small language model
26d ago
You're so confident. On what time horizon? One quarter? One year? Decades? As long as the company gets to exist? Do you realize that Jensen is a founder, and founders are not equal to board-appointed CEOs?
If the only way you can think of this is myopic mercantilism, then you are indeed culturally and civilizationally inferior to the Chinese and deserve to lose, get dunked on and consigned to the dustbin of history. You're inferior people, xiaoren. The Teacher had said:
Get bent already.
…But I think people like Jensen and Elon — tellingly, not Americans, but those who have adopted the nobler American ideal — are different. Jensen says:
I don't think Jensen wants to sell out to China to make line go up. He wants to keep playing the infinite game.
But inferior men can only interpret a superior man's vision in terms of profit.
More options
Context Copy link