DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
DaseindustriesLtd
late version of a small language model
2mo ago
There are so many things wrong with what Trump is doing that I find it silly to write a serious response. Literally an LLM would manage. For one thing, accept Von Der Leyen's offer of mutual tariff drop, that's enough of a “win” for your base and an actual economic boon! Apologize to Denmark and negotiate expanded military presence in Greenland under the existing framework. Offer China a mutual reduction in tariffs for sectors where you actually cannot back up your confidence. Tell Bukele to send back the wantonly arrested innocents for a fair trial. Stop gutting STEM research institutions. Crush or pay off the longshoremen, abolish Jones act. Buy a shitton of equipment for manufacturing drones. Put a few bombers on Guam instead of in Afghanistan, send a garrison onto Taiwan. It's not really complicated, he's done too many errors.
Was it a good idea to help build China into the unrivaled manufacturing and arguably economic colossus that it currently seems to be? I'm pretty sure it wasn't a good idea to try to invade and destroy multiple other countries in the name of "spreading democracy", but maybe you disagree? Was Biden on the right track?
Many questions. Was it a good idea to help build China? Probably not, but was it a bad idea to exploit their growth for salvaging your own one? I guess not again. Invasions? I think that was dumb. Biden? Yes, I think that Biden, or rather the system behind his limp body, was highly effective in reaching at least some subset of relevant goals of the Empire, it was going pretty smoothly. I am surprised to see them so thoroughly vanquished so fast.
How to deescalate? Oh, that's a big one. I think it's psychologically impossible, the US isn't willing to be #2, even if that carries none or minimal material demerits. Neither is Xi willing to give up on his system, or on Taiwan. History will decide.
DaseindustriesLtd
late version of a small language model
5mo ago
Okay, fair. #6 is contrived non sequitur slop, barely intelligible in context as a response to #5, so that has confused me.
In conclusion, I think my preference to talk to people when I want to, to AI when I want to, and use any mix of generative processes I want to, has higher priority than comfort of people who have nothing to contribute to the conversation or to pretraining data and would not recognize AI without direct labeling.
DaseindustriesLtd
late version of a small language model
5mo ago
Now if LLMs had had the OpenAI-tier breakthrough in China that would have been a challenge to the HBD stans, but this development basically aligns with the HBD take on the comparative advantage of Chinese talent in adopting Western stuff and then making marginal improvements with their own intelligence and grit.
The problem is that there haven't been substantial breakthroughs in LLMs in the West too. China runs Transformers and you guys run Transformers. I see Western papers full of unnecessarily clever bullshit that doesn't really work, and I see Chinese papers full of derivative bullshit that barely works. DeepSeek's MLA came out in May, and it remains SoTA cache optimization, and it's actually clever. GRPO, too, was quietly announced and seems to hold up very well despite dozens if not hundreds of cleverer results by "crazy geniuses" in the West (increasingly Indian). Today, the Chinese innovate on exactly the same plane.
I think it's time to admit that the famed Western creativity is mostly verbal tilt plus inflated self-esteem, not an advanced cognitive capability. I'm mildly surprised myself.
DaseindustriesLtd
late version of a small language model
1yr ago
To put it bluntly, I do not feel like pandering to misogynistic copes of people like Aaronson, who imagine themselves "romanceless" or "nice guys" rather than unsettling, mentally unwell, pathetically unmanly and, yes, plain ugly nerds. He is a loser. But on the whole, less of a loser than a penniless drunk conscript who'll get his dick blown off by a Ukrainian suicide drone.
I can only congratulate him for making it to a safe environment and finding a woman who looks past those biological drawbacks and loves him for who he is: a high-IQ prosocial academic with a badly coordinated, potbellied body of a paranoid bullying victim attached.
DaseindustriesLtd
late version of a small language model
2yr ago
Sovereign is she who only fights back.
I think the US Deep State was capable of winning this, just like Russia was capable of winning in Ukraine, in theory, if we were to ignore the actual level of Russian governance and corruption and ability to prosecute the war rationally. I knew of that one and so didn't expect Russia to win, and overestimated the US mainly because I did not account for the immense capacity for self-sabotage.
The US State department isn't staffed by geniuses who can shape the world to their liking.
I think they have enough talented people to do this, it's just those people have lost in internal politics.
simply because the world is too hideously complex a system for someone of any intellect or means to meaningfully manipulate
Manipulating the world is made much easier when you own major causal factors of that world. It doesn't take 200 IQ, though intelligence helps not to manipulate yourself into the ditch. All of great power politics is such manipulation. Suppressing competitors, strengthening allies, capturing international institutions, and yes, it's done by networks of high-agency people, not by vague sentiment of the electorate. Sorry, that's just what we can observe happening.
Nothing is set in stone; despite triumphalist propaganda directed at the public, I think the USG is aware of the problems by now and still has major cards like monopoly in crucial technology (ASML is a de facto American company), global reserve currency and, most of all, global goodwill, everyone anxious to go back to normal. Trump has improved his standing in the Middle East with a single speech. Americans are losing time but they can undo the self-inflicted damage with a few more such pivots, apologize for tone-deaf Greenland-posting, revitalize their alliance networks, actually reindustrialize, implement very liberal issuance of citizenship to all Chinese talent and brain-drain the nation – and that's not all. Maybe the AGI God plan will work out too – after all, the attack on Huawei and broader semiconductor supply chain was a resounding success of the sort I expected, it did delay China by years. Maybe Starship makes Brillant Pebbles a reality and forces China to disarm and sign unequal treaties… The US Hegemony is very much a viable project, except some Americans are in the way.
I recognize that my median prognosis has changed in a way that seems discrediting, but it's basically down to high-noise human factors on the US side.
All of these criticisms can be leveled at the Chinese as well - you've never heard them rant about 5,000 years of civilization?
They do have a strong belief in their civilizational superiority, and this chauvinism and smugness is another reason I was bearish on them. But in assessment of their current relative position they tend to be humble. “Building a world-class navy by 2035” is a typical Chinese goal. “Becoming a moderately prosperous society by 2020”. In 2018, Xi said:
When I met with Chinese and foreign journalists after the First Plenary Session of the 19th CPC Central Committee, I said that the Chinese Communist Party was determined to make a thousand years of greatness for the Chinese nation, and that a hundred years was just the right time to be in its prime. At the same time, I said this with a deep sense of worry. From our history, dynasties existed for more than 400 years in the Xia Dynasty, 600 years in the Shang Dynasty, 300 years in the Western Zhou Dynasty, 500 years in the Eastern Zhou Dynasty, 215 years in the Western Han Dynasty, 195 years in the Eastern Han Dynasty, 290 years in the Tang Dynasty, 277 years in the Ming Dynasty, 268 years in the Qing Dynasty, 15 years in the Qin Dynasty, 61 years in the Three Kingdoms, 167 years in the Northern Song Dynasty, 153 years in the Southern Song Dynasty, 90 years in the Yuan Dynasty, 38 years in the Republic of China, and other small dynasties There are countless blips and dynasties. The Qin Dynasty, Northern Song Dynasty, and Yuan Dynasty were all once unbeatable powers, but soon fell into disrepair. Those longer dynasties were also corrupt, socially unstable, discontented and rebellious, and many of them were left to languish and die. This shows that after a regime is established, it is not easy to maintain prosperity and long-lasting peace. Without self-reflection, vigilance, and effort, even the most powerful regimes can come to the end of the road.
It is now 97 years since the founding of our Party and 69 years since the founding of New China. The Soviet Communist Party has existed for 86 years, and the Soviet Union for 74 years. Our Party’s history exceeds that of the Soviet Communist Party, and our Party has not held national power for as long as the Soviet Union. By the middle of this century, the history of our Party will be close to 130 years, and the history of New China will reach 100 years. Comrade Deng Xiaoping said, “The consolidation and development of the socialist system will require a long historical stage, and it will take several generations, a dozen generations, or even dozens of generations of our people to struggle persistently and diligently.” How many years is that? It has to be calculated in terms of millenniums. This means that it will take a long historical period for us to build socialism with Chinese characteristics well and into. In this long historical process, it is an extremely difficult and risky challenge to ensure that the Chinese Communist Party does not collapse and the Chinese socialist system does not fall. Once upon a time, the Soviet Communist Party was so strong, the Soviet Union was so powerful, but now it has long been “the old country can not look back at the bright moon”. A generation does the work of a generation, but without historical perspective, without a long-term vision, also can not do the things of the moment.
This does not look as hubristic as American Main Character Syndrome to me.
The century of humiliation making them temporarily embarrassed hegemons
China has never held more than tenuous regional hegemony, I think this framing is not reflective of their ambitions and self-perception.
And you think that a world where China is hegemon won't see shit like Trump's exploitative trade war on the regular?
Yes. It's a stupid trade war and it's highly likely that no Tsinghua graduate will be so stupid. That aside, China has an official policy of not pursuing global hegemony. This certainly has no teeth, but Americans don't even have an equivalent toothless commitment.
Not to mention I'm fairly confident I've seen you mock Americans hyping the 'Chinese threat' and making them out to be more competent than they actually are as a motivation for more defense spending.
I've been right about that, Americans do hype up the Chinese military threat excessively, and they don't even build military that'd be useful in countering that threat, it's nearly entirely a grift. $1 trillion will go to more nebulous next-generation prototypes and battling the tyranny of distance in distant bases, not to a buildup of autonomous platforms that can compete in the SCS. Again, assuming Americans keep self-sabotaging.
When have you last been there and in what city? This was like watching Serpentza's sneering at Unitree robots back to back with Unitree's own demos and Western experiments using these bots.
Buses broke down, parts of my quite expensive apartment fell off, litter and human feces were everywhere
I simply call bullshit on it as of 2025 for any 1st tier city. My friends also travel there and work there, as do they travel to and live and work in the US. They report that straight from the gate in JFK, US cities look dilapidated, indeed littered with human feces (which I am inclined to trust due to your massive, easily observable and constantly lamented feral homeless underclass) and of course regular litter, squalid, there is a clear difference in the condition of infrastructure and the apparent level of human capital. I can compare innumerable street walk videos between China and the US, and I see that you guys don't have an edge. I do not believe it's just cherrypicking, the scale of evidence is too massive. Do you not notice it?
And I have noticed that Americans can simply lie about the most basic things to malign the competition, brazenly so, clearly fabricating «personal evidence» or cleverly stiching together pieces of data across decades, and with increasingly desperate racist undertones. Now that your elected leadership looks Middle Eastern in attitude, full of chutzpah, and is unapologetically gaslighting the entire world with its «critical trade theory», I assume that the rot goes from top to bottom and you people cannot be taken at your world any more than the Chinese or Russians or Indians can be (accidentally, your Elite Human Capital Indians, at Stanford, steal Chinese research and rebrand as their own). Regardless, @aqouta's recent trip and comments paint a picture not very matching yours.
I think that if they were truly crushing America in AI, they would be hiding that fact
They are not currently crushing the US in AI, those are my observations. They don't believe they are, and «they» is an inherently sloppy framing, there are individual companies with vastly less capital than US ones, competing among themselves.
When the Deepseek news came out about it costing 95% less to train, my bullshit detectors went off. Who could verify their actual costs? Oh, only other Chinese people. Hmm, okay.
This is supremely pathetic and undermines your entire rant, exposing you as an incurious buffoon. You are wrong, we can estimate the costs simply from token*activated params. The only way they could have cheated would be to use many more tokens but procuring a lot more quality data than the reported 15T, a modal figure for both Western and Eastern competitors on the open source frontier, from Alibaba to Google to Meta, would in itself be a major pain. So the costs are in that ballpark, indeed the utilization of reported hardware (2048 H800s) turns out to even be on the low side. This is the consensus of every technical person in the field no matter the race or side of the Pacific.
They've opensourced most of their infra stack on top of the model itself, to advance the community and further dispel these doubts. DeepSeek's RL pipeline is currently obsolete with many verifiable experiments showing that it's been still full of slack, as we'd expect from a small team rapidly doing good-enough job.
The real issue is that the US companies have been maintaining the impression that their production costs and overall R&D are so high that it justifies tens or hundreds of billions in funding. When R1 forced their hand, they started talking how it's actually "on trend" and their own models don't cost that much more, or if they are, it's because they're so far ahead that they finished training like a year ago, with less mature algorithms! Or, in any case, that they don't have to optimize, because ain't nobody got time for that!
But sarcasm aside it's very probable that Google is currently above this training efficiency, plus they have more and better hardware.
Meta, meanwhile, is behind. They were behind when V3 came out, they panicked and tried to catch up, they remained behind. Do you understand that people can actually see what you guys are doing? Like, look at configs, benchmark it? Meta's Llama 4, which Zuck was touting as a bid for the frontier, is architecturally 1 generation behind V3, and they deployed a version optimized for human preference on LMArena to game the metrics, which turned into insane embarrassment when people found out how much worse the general-purpose model performs in real use, to the point that people are now leaving Meta and specifying they had nothing to do with the project (rumors of what happened are Soviet tier). You're Potemkining hard too, with your trillion-dollar juggernauts employing tens of thousands of (ostensibly) the world's best and brightest.
Original post is in Chinese that can be found here. Please take the following with a grain of salt. Content: Despite repeated training efforts, the internal model's performance still falls short of open-source SOTA benchmarks, lagging significantly behind. Company leadership suggested blending test sets from various benchmarks during the post-training process, aiming to meet the targets across various metrics and produce a "presentable" result. Failure to achieve this goal by the end-of-April deadline would lead to dire consequences. Following yesterday’s release of Llama 4, many users on X and Reddit have already reported extremely poor real-world test results. As someone currently in academia, I find this approach utterly unacceptable. Consequently, I have submitted my resignation and explicitly requested that my name be excluded from the technical report of Llama 4. Notably, the VP of AI at Meta also resigned for similar reasons.
This is unverified but rings true to me.
Grok 3, Sonnet 3.7 also have failed to convincingly surpass DeepSeek, for all the boasts about massive GPU numbers. It's not that the US is bad at AI, but your corporate culture, in this domain at least, seems to be.
But if Chinese research is so superior, why aren't Western AI companies falling over themselves to attract Chinese AI researchers?
How much harder do you want them to do it? 38% of your top quintile AI researchers came straight from China in 2022. I think around 50% are ethnically Chinese by this point, there are entire teams where speaking Mandarin is mandatory.
Between 2019 and 2022, «Leading countries where top-tier AI researchers (top 20%) work» went from 11% China to 28%; «Leading countries where the most elite AI researchers work (top 2%)» went from ≈0% China to 12%; and «Leading countries of origin of the most elite AI researchers» went from 10% China (behind India's 12%) to 26%. Tsinghua went from #9 to #3 in institutions, now only behind Stanford and Google (MIT, right behind Tsinghua, is heavily Chinese). Extrapolate if you will. I think they'll crack #2 or #1 in 2026. Things change very fast, not linearly, it's not so much «China is gradually getting better» as installed capacity coming online.
It's just becoming harder to recruit. The brain drain is slowing in proportional terms, even if it holds steady in absolute numbers due to ballooning number of graduates: the wealth gap is not so acute now considering costs of living, coastal China is becoming a nicer place to live in, and for top talent, more intellectually stimulating as there's plenty of similarly educated people to work with. The turn to racist chimping and kanging both by the plebeians since COVID and by this specific administration is very unnerving and potentially existentially threatening to your companies. Google's DeepMind VP of research left for ByteDance this February, and by now his team in ByteDance is flexing a model that is similar but improves on DeepSeek's R1 paradigm (BD was getting there but he probably accelerated them). This kind of stuff has happened before.
many Western countries are still much nicer places to live than all but the absolute richest areas of China
Sure, the West is more comfortable, even poor-ish places can be paradaisical. But you're not going to move to Montenegro if you have the ambition to do great things. You'll be choosing between Shenzhen and San-Francisco. Where do you gather there's more human feces to step into?
But as I said before in the post you linked, Chinese mind games and information warfare are simply on a different level than that of the more candid and credulous Westerner
There is something to credulousness, as I've consistently been saying Hajnalis are too trusting and innocently childlike. But your nation is not a Hajnali nation, and your people are increasingly draught horses in its organization rather than thought leaders. You're like the kids in King's story of how he first learned dread:
We sat there in our seats like dummies, staring at the manager. He looked nervous and sallow-or perhaps that was only the footlights. We sat wondering what sort of catastrophe could have caused him to stop the movie just as it was reaching that apotheosis of all Saturday matinee shows, "the good part." And the way his voice trembled when he spoke did not add to anyone's sense of well-being.
"I want to tell you," he said in that trembly voice, "that the Russians have put a space satellite into orbit around the earth. They call it . . . Spootnik.” We were the, kids who grew up on Captain Video and Terry and the Pirates. We were the kids who had seen Combat Casey kick the teeth out of North Korean gooks without number in the comic books. We were the kids who saw Richard Carlson catch thousands of dirty Commie spies in I Led Three Lives. We were the kids who had ponied up a quarter apiece to watch Hugh Marlowe in Earth vs. the Flying Saucers and got this piece of upsetting news as a kind of nasty bonus.
I remember this very clearly: cutting through that awful dead silence came one shrill voice, whether that of a boy or a girl I do not know; a voice that was near tears but that was also full of a frightening anger: "Oh, go show the movie, you liar!”
I think Americans might well compete with North Koreans, Israelis and Arabs in the degree of being brainwashed about their national and racial superiority (a much easier task when you are a real superpower, to be fair), to the point I am now inclined to dismiss your first hand accounts as fanciful interpretations of reality if not outright hallucinations. Your national business model has become chutzpah and gaslighting, culminating in Miran's attempt to sell the national debt as «global public goods». You don't have a leg to stand on when accusing China of fraud. Sorry, that era is over, I'll go back to reading papers.
DaseindustriesLtd
late version of a small language model
2mo ago
National Socialism with Chinese Characteristics...
It's a funny joke but really, they're not any more National Socialist than any normal European state was before WWII. They are quite different from historical Nazis. They have a representation for minorities (even repressed ones) and affirmative action, they have legalized gender transition, they employ open furries in the PLA (explicitly as fursuit engineers, to develop next generation combat armor). Their notions of “degeneracy” or “racial hygiene” would be quite alien to Germans. The basic level of care for the ethnic majority is just what a state is supposed to do. And Socialism – that they owe to being literally Marxists, with a big portrait of Marx in their main hall of power and stuff. They're far more Capitalist than the Third Reich was, too. Xi has restored the cult of personality, though. Seriously speaking, it's its own complex thing, and should be considered on its own merits in its own historical context, not as a copy or a pastiche of Western paradigms. When all is said and done they're just a modernized Chinese empire.
You simultaneously mock Europeans for being "not capable of resisting a tiny tribe of natural wordcels"
I apologize. My sarcasm there may have been too confusing. I don't think Jews are solely guilty for the quality of your media. Jews, from what I can tell, genuinely like their sermonizing slop, but so does the audience, and creators are increasingly Gentiles too. I think you just have ran out of gas. Particularly Americans. Your culture is vulgar and plain bad, and you should feel bad about it. Your mavericks are sleazy hustlers at best and psychopaths at worst, and you do not resist your worst impulses to bow before the undeserving strongman. You come up with zany and harmful ideas and then force them upon everybody else. Thus, you are what has to be resisted now, at least until you improve somehow.
You just hate Europeans, particularly the West Europeans, you see them as your enemy and you always have.
I don't hate Europeans. I am disappointed in you. In you collectively and in you, SecureSignals, personally. You are less than what I figured, you don't deliver on the crucial advertised open-mindedness and ability-to-change-opinions features, and you take pride in stuff that's completely meh or plainly disgusting. You're like some purebred dogs. Remarkable, peculiar, WEIRD phenotype, but no spark, or almost never. Disappointing.
and I do not want to see them under Chinese hegemony
And at the rate you're going, you may well see Chinese hegemony. It is indeed unfortunate because the Chinese themselves never had it in them to establish one, I don't think. Too insular, too mercantile, too autistically uncharismatic, and frankly not capable enough to dismantle natural affinities and alliances. They'd have secured their backyard and grew content to have limited trade with barbarians, and this was the scenario I still consider preferable. But a few more iterations of low-IQ, smug WINNING and ROCKING THE BOAT, and who knows, they may have to pick up the crown tossed their way.
And the ironic thing is that all this is because you'd have wanted your own hegemony, because for all the denialism – the dream, the hope of being Intrinsically Racially Superior, crushing lessers under the jackboot, still lives and yearns for confirmation. Alas.
DaseindustriesLtd
late version of a small language model
2mo ago
Losing means a Guangxi Massacre in every American town and city.
It's not clear what must happen for the world to end up like this, but America is a nation of dreamers; I suppose you can effect even this result if you keep pressing on. However, my optimistic theory of American loss is that due to constant bluffing and irresponsible policy epilepsy the USD loses its status as reserve currency, your fraudulent markets deflate, your internal racial contradictions bloom, and after a while you get a lot quieter and less obnoxious as your living standards crash down to roughly Polish level, which is actually very neat and, given your current course, more than you deserve. The traffic to your shores dies down, as mandated by the Great Leader Donald Ieyasu Trump; the rest of humanity, free of the loathsome star-spangled yoke, peacefully trades and gets richer, while you lick your wounded pride and dream of revenge.
A median scenario is that you simply accept the existence of a bigger guy on the block (bloodlessly, or after trying your luck one last time in the South China Sea) and retreat to your hemisphere, living much the same lives as today.
And I suspect that you know this. But it's too painful to imagine such a world, a boring high-probability world where the sky didn't fall, but you're no longer the uncontested Main Character Nation. Visions of massacres and genocides are anesthetic in comparison, they return you to the familiar domain of Marvel movies. Any Avengers-Level Threat, by laws of narrative, ultimately gets defeated, so there really aren't any stakes or hard decisions to make this way.
DaseindustriesLtd
late version of a small language model
2mo ago
I do know this and I wonder how that coexists with the common East Asian respect for the Hebrews. Have they considered playing one great tribe against another? Or learning the Manchu ways to beat them at their own game, like Koreans try with Talmud? I should ask.
DaseindustriesLtd
late version of a small language model
4mo ago
Once again I notice that I am usually right in being rude to people, as their responses demonstrate precisely the character flaws inferred. This is a low-content post in defense of wounded ego, with snappy Marvel-esque one-liners («Won a Nobel prize») and dunks optimized for the audience but next to no interest in engagement on the object level. Yes, ML != LLMs, so what? Are you not talking about Altman and Elon who both clearly put their chips on LLMs? «That was a joke», yeah I get your jokes. Here's one you missed:
data engineering is needed and important! But it's not revolutionary. Data engineering, is the same as its always been.
It's not the same, though, that's the thing. Returning back to my point that has upset you –
Fetishizing algorithmic design is, I think, a sign of mediocre understanding of ML, being enthralled by cleverness. Data engineering carves more interesting structure into weighs.
– I meant concretely that this is why leading companies now prioritize creation of training signal sources, that is: datasets themselves (filtered web corpora, enriched and paraphrased data, purely synthetic data, even entirely non-lingual data with properties that induce interesting behaviors), curricula of datasets, model merging and distillation methods, training environments and reward shaping – over basic architecture research, in terms of non-compute spend and researcher hours; under the (rational, I believe) assumption that this has higher ROI for the ultimate goal of reaching "AGI", and that its fruit will be readily applicable to whatever future algorithmic progress may yield. This goes far beyond ScaleAI's efforts in harnessing Pinoy Intelligence to annotate samples for RLHF and you have not even bothered to address any of this. If you think names of Old Titans are a valid argument, I present Hutter as someone who Gets It, gets that what you have a sufficiently general architecture to approximate is at our stage more interesting in terms of eventual structure than how you achieve this potential generality.
This older paper is a neat illustration too. Sohl-Dickstein and Metz have done a little bit of work in non-LLM algo design if you recall, maybe you'll recognize at least them as half-decent scientists.
Now, as regards poor taste in intellectual disagreements, let's revisit this:
Regardless of whether transformers are a dead-end or not, the current approach isn't doing new science or algo design. Its throwing more and more compute at the problem and then doing the Deepseek approach of finetuning the assembly level gpu instructions to exploit the compute even better so you can throw more compute at it. I doubt, Hinton, Goodfellow, LeCunn, Schimdhubber et al. have any desire to do that. Maybe if xAI did something revolutionary like leave the LLM space or introduce a non-MoE-Transformer model for AGI, then talent of that caliber might want to work there. Currently they exist so Elon can piss all over Altman.
My rudeness was not unprovoked; it was informed by the bolded parts. I saw it as a hubristic, elitist, oblivious, tone-deaf insult towards people – scientists – actually moving the field forward today, rather than 8 or 28 years ago, and I do not care that it's slightly obfuscated or that you lack self-awareness to recognize the douche in the mirror but are eager to chimp out at it as you currently do.
Likewise my entire point, before you jumped into insult me, is that the Big Names in ML/AI are "fetishy algo freaks" They shockingly don't want to do non "mediocre algo butt sniffing" work. And Data Engineering isn't new, it isn't revolutionary, it's great, it works well, but it doesn't require some 1% ML researcher to pull it off. It requires a solid engineering team, some technical know-how, and a willingness to get your hands dirty. But no one is going to get famous doing it. It's an engineering task not a research task. And since research tasks are what people pay the ludicrously big bucks for at tech companies the engineers at xAI aren't being paid some massive king-sized salary...
yes thanks for clarification, that's exactly as I understood you.
I claim that to the extent that «talent of that caliber» shares your conceit that design of clever new algorithmic primitives for ANNs is «exciting new science» whereas data work remains and will remain ScaleAI tier «mere data engineering, same as always», this talent is behind the times, too set in their ways, and is resting on its laurels; indeed this is the same high-level philosophical error or prizing manual structure design over simplicity, generality and scalability that keeps repeating on every revolution in AI, and that Sutton has famously exposed. They are free to work on whatever excites them, publish cute papers for fellow affocionados where they beat untuned mainstream baselines, or just leave the frontlines altogether, and even loudly assert that they have superior taste if they so choose, which in my view is just irrational fetishism plus inflamed ego; I think taste is to be calibrated to actual promise of directions. But indeed, what do I know. You are free to share their presumptions. New scientific talent will figure it out.
Seems like you are asking us to praise ignorance over discovery?
To me it seems like the opposite, we just disagree on what qualifies as discovery or science at all, due to differences in taste.
As an exercise, can you tell me THE engineer at Deepseek who proposed or wrote their Parallel Thread Execution(PTX) code with a citation?
Egoists gonna be egoists.
Zhean Xu probably. But I think everyone on (Chenggang Zhao and Shangyan Zhou and Liyue Zhang and Chengqi Deng and Zhean Xu and Yuxuan Liu and Kuai Yu and Jiashi Li and Liang Zhao) list could ask for a megabuck total comp in a frontier lab now, and expect affirmative response.
DaseindustriesLtd
late version of a small language model
5mo ago
I would welcome such a subhuman overreaction.
DaseindustriesLtd
late version of a small language model
5mo ago
Okay. I think the elderly care is mainly a problem of machine vision and manual dexterity. I believe these guys will solve it in five years tops.
DaseindustriesLtd
late version of a small language model
5mo ago
Wefang is implying basically stereotype threat: that the Chinese don't innovate from 0 to 1 because there's a stereotype that job belongs to the West
Wenfeng.
No, it's not a stereotype threat argument, it's an argument about perceived opportunity cost of exploration vs exploitation which is miscalibrated in the age of large domestic revenue generators. He's not arguing they should be like Whites. He's arguing they can now afford to do what Whites do compulsively, if you will.
Your condescension and willful misinterpretation will be your undoing in this dialogue and outside it.
I look down on WEIRDs for one more reason. You are ultimately tool-like, your mentality is that of servitors and cowering peasants. Your "internal dignity" is inextricably bound to collective judgement, you feel the need to justify your value to some imagined audience, to some Baron, some market or some Moral Community. You are ashamed of brute, terminal-value ethnocentrism the sort of which Judaism preaches, so you need to cling to those spiritualist copes wrapped in HBD lingo. "H-here's why we are Good, why we still deserve a place under the sun, sire!" This exposes you to obvious predation and mockery by High-Skill Immigrants like Count.
On the object level: yes, probably on average the Chinese are indeed less "creative" even with optimal incentives, and this has obvious implications at the tails. (though if we think OpenAI is an impressive example of bold creativity, what about NVidia? What did Jensen "merely improve"? As a CEO, he's roughly in the same league as Altman and Musk, I think). The question – raised by R1 there – is, how many more True Breakthrough innovators do we even need before innovation begins to accrete on itself without human supervision? Maybe just a handful. Again, there's been virtually no fundamental progress in AI since 2017, and we're all doing just fine. It may be that architecturally V3 is more sophisticated and innovative than the modern OpenAI stack. Imagine that. After all, Western geniuses are afraid to show their work these days.
Incidentally, I myself have submitted several minor ideas to DeepSeek; maybe they found use for those, maybe not, but I'll find use for the result of their labor and not cope that they needed my input.
It may be that the mode of production implied by the stage of our technological development makes your race, with all its creative perks and industrial drawbacks, less economically useful than it used to be. This only means you need to move that much faster to find reasons to protect your interests unconditionally, before everyone turns equally economically useless.
DaseindustriesLtd
late version of a small language model
5mo ago
Honestly this feels like a cope to me. There obviously was a breakthrough in LLMs in the West: politically, economically, technologically, culturally. It wasn't born in China, but they obviously have a significant part to play downstream of their undeniable talent pool.
What are you talking about? Have you stopped reading my post there?
It's hard to say Deepseek would have accomplished these things without drafting on OpenAI's introduction of LLMs to the world,
Here's what I think about this. The Chinese are not uncreative. It's worse: they're cowardly, conservative, and avoid doing exploratory shit that seems high-risk, and they buy into your theory of their own inferiority, and steelman it as “good at execution”. As Wenfeng says:
Another reason that domestic large models have rarely dabbled in innovation at the architectural level before is that few people have dared to go against the stereotype that America is better at the technological innovation from 0 to 1, while China is better at the application innovation from 1 to 10. Not to mention that this kind of behavior is very unprofitable -- the usual thinking is that, naturally, in a few months, someone would have made the next generation of models, and then Chinese companies can just follow the leader, and do a good job of application. Innovating the model structure means that there is no path to follow, and there are a lot of failures to go through, which is costly in terms of time and money.
There will be more and more hardcore innovation in the future. It may not be yet easily understood now, because the whole society still needs to be educated by the facts. After this society lets the hardcore innovators make a name for themselves, the groupthink will change. All we still need are some facts and a process.
You are watching these facts come in.
I repeat, I've been a believer in this theory of “fundamental Western progress, incremental Eastern refinement”. Eight years into Transformer era (Ashish Vaswani et al., 2017), I start to doubt it. Whites are just people who are sexually attractive, relatively trustworthy, and provide linear labor to verbal-tilted Brahmins who max corporate KPIs leveraging even more verbal-tilted Ashkenazim like Altman who are good at raising capital.
That's about it at this point.
The most credible, big-brained, innovation-heavy alternative to Transformer was Mamba (Tri Dao, Albert Gu). It also didn't go far. I've read perhaps hundreds of Western papers of purportedly brilliant innovations, they're narcissistic shit that doesn't scale. Sepp Hochreiter is peddling his xLSTM that has no utility, Schmidhuber is making some boastful noises as usual, Sutskever and Karmack are supposedly doing… something. Mistral is dead in the water…
I am not saying this out of racism. I am reporting on what I see happening. All historical inventions and discoveries of note? Yes, those were White work. But time is accelerating. Maxwell's equations seem not far from "muh gunpowder" of the Middle Kingdom now, to my eyes. Do something new, folks. You're losing face.
On the other hand we know OpenAI did not need Deepseek.
Sure, OpenAI needed another company. OpenAI built its legend on scaling up a Google paper. By your own standards, it's not creative brilliance. It's the sort of talent you condescendingly concede Chinese people have.
DaseindustriesLtd
late version of a small language model
1yr ago
Oh come on, this is more American whining. Muh deaths of overdoses, muh Russian election meddling, little old us assaulted on all fronts, won't somebody please spare a thought for the poor hegemon.
The CHIPS act has been about pork and the usual fighting over the spoils from the beginning, its success or failure is of no consequence. China was summarily cut off from modern semiconductor manufacturing and falls behind, new fabs in safe allied countries are being completed, Taiwan is getting reinforced, and AGI seems to be on schedule within 5 years. Yes, could have been done better. But it has gone well enough that advancing petty political agendas took precedence. If there ever is any plausible risk of the US losing control over the global high-end manufacturing chain, I am sure you'll see it going differently.
DaseindustriesLtd
late version of a small language model
1yr ago
Cynicism is a cope of dysfunctional people.
DaseindustriesLtd
late version of a small language model
2yr ago
How many Joyces contend for an average Hugo?
Every second-tier female fanfic writer is invested in her characters' feelings and thoughts and dedicates a big part of the work to spelling that out.
Men, like I said, have other avenues to express their core interests.
DaseindustriesLtd
late version of a small language model
2yr ago
A very German thing to believe. I weep for your people, but really you've been cooked since before both of us were born, so this revolution adds nothing.
DaseindustriesLtd
late version of a small language model
1mo ago
Spy satellites contribute so little to the total mass to orbit that you never even needed SpaceX for that (i don't consider Starlink a primarily national security project, because it's not).
For delivering payloads, including probably international ones, China will begin catching up next year. I do not assume that Americans will be contracting them, no, so in that sense SpaceX is poised to maintain its near-monopoly.
It is believed that the crop of reusable rocket startups is attributable to Robin Li, the founder of Baidu, getting into National People's Congress, and advocating for legalization of private space businesses in 2010s. So far, there have been three Chinese entities that have conducted VTOL tests for reusable rockets.
- China Aerospace Science and Technology Corporation (CASC), June 23, 2024
- LandSpace, September 11, 2024
- Space Epoch, May 29, 2025
There are others which are further behind.
Technologically, they are several iterations behind, but strategically I'd say they save significant advantages over the current SpaceX (a usual feature of Chinese fast-following). For example Space Epoch Yuanxingzhe-1 is basically a small Starship (or a better, thicker Falcon-9, if Falcon-9 were designed today). Stainless steel, metholox, will naturally plug into the existing and state-subsidized logistics, including military facilities that currently produce aviation parts (as a small point, Falcon's extreme height-to-width ratio is obviously suboptimal and downstream of American highway standards, but China had no problem building dedicated roads). LandSpace Zhuque-3 VTVL-1 is similar (they can boast of the first metholox engine to make it to orbit).
But as you rightfully notice, it's not clear if this will have much effect on the SpaceX bottom line, since Americans can saturate their cadence anyway. In all likelihood it will only unnerve some people in Washington as a symbolic thing.
I see you took this pretty personally.
All I have to say is that top AI research companies (not ScaleAI) are already doing data engineering (expansively understood to include training signal source) and this is the most well-guarded part of the stack, everything else they share more willingly. Data curation, curricula, and yes, human annotation are a giant chunk of what they do. I've seen Anthropic RLHF data, it's very labor intensive and it instantly becomes clear why Sonnet is so much better than its competitors.
They clearly enjoy designing "algos", and the world clearly respects them greatly for that expertise.
Really glad for them and the world.
Past glory is no evidence of current correctness, however. LeCun with his «AR-LLMs suck» has made himself a lolcow, so has Schimidhuber. Hochreiter has spent the last few years trying to one-up the Transformer and fell to the usual «untuned baseline» issue, miserably. Meta keeps churning out papers on architectures; they got spooked by DeepSeek V3 which architecture section opens with «The basic architecture of DeepSeek-V3 is still within the Transformer (Vaswani et al., 2017) framework» and decided to rework the whole Llama 4 stack. Tri Dao did incredibly hard work with Mamba 1/2 and where is Mamba? In models that fall apart on any long context eval more rigorous than NIAH. Google published Griffin/Hawk because it's not valuable enough to hide. What has Hinton done recently, Forward-Forward? Friston tried his hand at this with EBMs and seems to have degraded into pure grift. Shazeer's last works are just «transformers but less attention» and it works fine. What's Goodfellow up to? More fundamental architecture search is becoming the domain of mentally ill 17yo twitter anons.
The most significant real advances in it are driven by what you also condescendingly dismiss – «low-level Cuda compiler writing and server orchestration», or rather hardware-aware Transformer redesigns for greater scalability and unit economics, see DeepSeek's NSA paper.
This Transformer is just a paltry, fetish, "algo".
Transformer training is easy to parallelize and it's expressive enough. Incentives to find anything substantially better increase by OOM year on year, so does the compute and labor spent on it, to no discernible result. I think it's time to let go of faulty analogies and accept the most likely reality.
DaseindustriesLtd
late version of a small language model
5mo ago
Sounds like they need LLM writing assistance more than anyone, then.
DaseindustriesLtd
late version of a small language model
5mo ago
I am quite happy with my analytical work that went into the prompt, and R1 did an adequate but not excellent job of expanding on it.
But I am done with this discussion.
DaseindustriesLtd
late version of a small language model
5mo ago
I was not aware that this is a forum for wordcels in training, where people come to polish their prose. I thought it's a discussion platform, and so I came here to discuss what I find interesting, and illustrated it.
Thanks for keeping me updated. I'll keep it in mind if I ever think of swinging by again.
This is of course a projection of your own tribalism and your own deluded moral framework.
Your problem is that your only guiding light, the only salvation you see for your people, is Nazism, and Nazism is still quite degenerate and NGMI. I won't talk of its moral merits, it's just strategically bad because it's aestheticized desperation and refuge from hopelessness in animalistic impulses. A stronk chieftain (high agency!), will to power (rock the boat!), blood-based tribal identity, vibes over facts… in effect, reject modernity, retvrn by rolling back the evolutionary clock 9000 years, to where an average European was a fat bipolar slob with 65 IQ. Nazism was swiftly crushed by Capitalism and Communism. 80 years later, they remain the dominant forces on the planet and continue their dialectic and coevolution. You like to think that Judaism is still more important, the root of all evil. Well, it's underrated for obvious reasons, I'll give you that, but Earth is a big place, and your struggle with Joos is ultimately quite parochial.
I have observed many sincere Nazis over the years and most are suicidal. It doesn't have to be this way. Accept that the dream of Aryan greatness is dead, but you can live if you accept this world on its own terms, where your people have some advantages and disadvantages entirely irrespective of “jewish manipulation” or “suicidal empathy” or what have you, and need to manage them soberly. In particular this requires a good understanding of where you stand relative to that huge chunk of humanity in East Asia. One approach is to cope with 4chan gifs of tortured dogs and industrial accidents, or the book of Ralph Townsend. Another is to grow the fuck up.
More options
Context Copy link