DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
DaseindustriesLtd
late version of a small language model
22hr ago
Better to be right about something simple than sophisticated and wrong (or be sophisticated and correct about secondary factors).
The question is which factors are primary and which secondary. I posit that one simple factor primary factor is just that DeepSeek's guys have higher IQ and better company culture, and operate in a more materially productive society, which has effects until AI is truly superhuman.
Deepseek is stuck in a deadly spiral because they lack compute. Because they lack compute, they struggle to generate much revenue and therefore can't raise enough money to get more data or even acquire more compute.
Right now DeepSeek can generate that $1B ARR off approximately 3000 Blackwells, and they've locked most of Huawei's production for 2027. They'll be fine on revenue. They have raised about $20B in total so far, and will struggle to spend it on compute anyway.
Regarding the "hack chyna" stratagem, might be of interest, Grok with your "bitter lesson" makes a cameo too. Note that this is Daybreak, so it's not about refusals:
DaseindustriesLtd
late version of a small language model
2d ago
The absolute value is what matters most, there is no prize for being much cheaper.
There is: deployment volume. Though China is disadvantaged on the compute production anyway.
generally, I feel that you're kind of dumb if you feel proud of this "more compute better" discovery. Do you want a cookie? It's trivial. If anything, I'm saying that it's not crushing hard enough. At this rate, your hack-all-China idea likely won't work.
If compute isn't the key factor, then what's holding Deepseek back?
lots of things are holding them back, such as the volume of acquired data and RL environments. Anthropic has been spending on this for longer, and much more. But yes, compute is the most relevant form of capital.
DaseindustriesLtd
late version of a small language model
2d ago
Facebook and Google are back in the game and well ahead of any Chinese competitor.
It's hilarious that we're treating Google DeepMind as some inept nouveau riche that can only "crush" with compute. They invented move than half of this whole field.
Regardless, Grok 4.7 (and you were very confident about Grok) is clearly utterly non-competitive with the latest DeepSeek, and I'm not sure if Muse Wonder and Argon fare much better.
It's a bit unexpected even for me.
DaseindustriesLtd
late version of a small language model
14d ago
But here’s the thing the west and to a lesser extent the US is at war with Russia. Undeclared and without ground troops, but we are contributing financing, seized Russian assets, and have and probably still provide intelligence to the Ukrainian side
Declaring yourself to have loyalties to a foreign government at war with the US seems absolutely perfect for denaturalization.
This is a reasonable argument, except for Russia not being at war with the US. It is a meaningful distinction. You're free to consider the Ukraine conflict a proxy war between Russia and "The West", however objectively it's a war between Russia and Ukraine, which is not even a NATO or a EU member (indeed, precluding its ascension to those alliances is the biggest reason for the aggression), with geopolitical, moral, economic and security considerations driving indirect participation from many third parties. There are many messy proxy wars around the globe. The US has no obligation to do what little it's doing for Ukraine, in principle the US could even support Russia to get it over with quicker and split the spoils (Trump's treatment of Denmark and broadly the EU go to show he assigns negligible value to the welfare or goodwill of Our Western Allies, they're mostly chumps to extract value from; Trump also doesn't give a shit about "democracy" or "human rights" and constantly calls dictators like Xi or Kim his friends). It's an unforced choice made by a nation with high optionality.
So, in effect, you're saying that you want it to be illegal to voice opinions about world affairs that are at odds with the current American geopolitical orientation. This is a rather different argument. Should people be deported if they support Palestine, given that Israel is the Greatest Ally? Should citizens be denaturalized? Does this apply only to naturalized US citizens, to the first-generation US-born too, or to Heritage Traitors as well? What's the legal opinion on Myanmar or on the Sudanese Civil War? How do honest Americans stay abreast of the official list of good and bad foreign factions?
Now I don't agree with Ovechkin's likely conformist and opportunistic support for Putin (that said, Putin != SMO, it's illegal to protest SMO in Russia so Ovechkin wouldn't be able to support any peace party anyway), and he's not morally or legally entitled to working in the US, and I don't care for his fate. And if he's not a citizen, your more extreme proposal of denaturalization is moot for such cases. I just welcome you to think through a coherent, legible legal doctrine in which your proposals would stand. It would be somewhat sad for me if MAGA devolved into uncritical ad hoc "Laura Loomer Patriotism" where anyone landing on the shit list of self-appointed guardians of the Right can be fed to the wolves, while the crowd hoots and jeers.
DaseindustriesLtd
late version of a small language model
22d ago
As I've said, Starship can't not work.
DaseindustriesLtd
late version of a small language model
24d ago
I mean frontier models, not GLM. Though it sometimes drops articles and the like.
DaseindustriesLtd
late version of a small language model
1mo ago
Most? No. I wouldn't be surprised if they were mid-pack. This isn't to say I think OpenAI is really good.
DaseindustriesLtd
late version of a small language model
1mo ago
How many software engineers [or others directly threatened by AI]
You think I meant job displacement? I meant actual scares. I mean loyal Trumpists going full Alex Jones when they realize the Skynet is coming. It would be pretty easy to scare people into Luddism, to the extent that it overpowers current partisan split (not entirely, of course, but moving it by 10% or so).
DaseindustriesLtd
late version of a small language model
1mo ago
I guess I'm just optimistic about the US. People call me a China shill, but the US has too many capable people in too many well-resourced companies (not clear how they'll cope with Frontier Labs trying to eat them, but the state may prefer them non-eaten and that could be decisive). Even the government is not entirely inept (at least it's more capable than EU governments, Russia, LatAm, most of the rest of the world). In particular, the US is excellent at kicking the can down the road (ballooning debt is a case in point). AI productivity gains might turbocharge this capability, and indeed this seems to be explicitly the plan – Bessent proposes just growing out of the debt, and growth is all about AI now. So, the system will receive generous injections of surplus energy to deal with overhangs, as you call it.
Whether this will be enough to compensate for new sources of entropy, and whether that compensation will take the form of a broadly tolerable American way of life plus AI upgrades or a Palantircore dystopia with UBI, this I admit I don't know. But I wouldn't bet on some societal collapse that exceeds the costs of another culture war conflagration. Betting against the US is, historically, a fool's errand.
DaseindustriesLtd
late version of a small language model
1mo ago
Robots with non-local power are only useful in factory settings. We also have them.
I don't agree. I'm a robot with non-local power myself, in a sense – I need to use a network-connected smartphone to navigate an unfamiliar environment. This is the general human condition now. Suppose Optimus has an always-on Starlink connection. Does it matter if it's "not entirely here"? I guess it matters for the robot revolution part, because Elon will have a way to shut it down (if he cares). But practically, it seems to be the inevitable compromise.
but a brief look makes me think this is exactly the sort of thing that RL game playing is good at, and learning general game-playing strategies in training would generalize pretty well to this
This genre of dismissals is fair enough but getting vacuous. What doesn't "RL game playing", at enough scale and diversity, generalize well to? ARC-3 was supposed to measure genuine cognitive fluidity. There is a number of papers showing that reasoning RLVR, even extremely impoverished (literally GSM8K/HumanEval maxxing, like in first generation R1), generalizes to very distant tasks like creative writing, because they involve similar reasoning primitives/motifs (backtracking, self-checking, enumerating options etc). We've actually first seen this principle with, like, InstructGPT, pretraining on more code + RL on code = smarter model across the board, because code entrains some helpful cognitive patterns. RLVR on more complex multimodal tasks will generalize better.
DaseindustriesLtd
late version of a small language model
1mo ago
It's permanent, the US can expand with Denmark's permission, and they can deny foreign expansion without Denmark's say so.
Was Denmark, in practice, ever going to allow "foreign expansion" in Greenland? I get that the MAGA narrative is how Denmark is an Islamist communist shithole and deeply infiltrated by the CCP (probably way less infiltrated than Washington DC), but if memory serves, they always permitted the US to treat Greenland as its exclusive military outpost.
DaseindustriesLtd
late version of a small language model
1mo ago
ChatGPT Astra is meaningfully better than Sol. I of course agree it's still disgusting. It's also nice news for me because literally Fields Medal winners are getting automation anxiety before I have to worry about competition in bespoke shitposts.
The issue with writing style is that they're not built to write for humans. These are agents. The process by which they output almost every given token over their training is getting rewarded conditional on its contribution to the final outcome of a programming (or programming-shaped) task.
Be thankful you don't have to deal with raw CoTs (unreadable now).
DaseindustriesLtd
late version of a small language model
1mo ago
but the US develops and even IOCs entire weapons systems in secret SAPs
These munitions that allies are now NOT getting have no relation to what you talk about. I do not make an argument concerning any classified assets. For all I know the US has a secret stockpile of antiproton torpedoes or Trisolaran droplets in Area 51. Maybe that's what is keeping China at bay. But Tomahawks, THAADs, ATACMS, PrSMs, Patriot PAC-3s are down bad, there's no way around it.
DaseindustriesLtd
late version of a small language model
1mo ago
Not so respectfully, The US has warned allies to expect delays of as much as five years in key weapons deliveries as it rebuilds its own depleted munitions stocks.
Germany has asked the US to speed up delivery of the long-range Tomahawk cruise missile and the ground-based Typhon launcher, but depleted US stocks of the powerful weapon means its manufacturer, RTX Corp’s Raytheon, is unlikely to fulfil the order in the next five years.
WASHINGTON – The US is facing a shortfall in ammunition as a result of its war against Iran, the Defence Department’s inspector-general confirmed on Sept 14, contrasting with President Donald Trump’s assertions of full stocks.
In a first-of-its-kind mandatory report to Congress, the inspector said that between Feb 28 and June 30, Operation Epic Fury (OEF) had an estimated cost of US$33.4 billion (S$42.3 billion), including US$22.3 billion spent on expended munitions.
“The munitions expenditure on OEF has resulted in strategic inventory shortfalls and revealed industrial base bottlenecks for munitions resupply,” the report said.
This isn't hard. For many years, the stockpiles of US precision munitions were known to be finite, overpriced and very limited, this is the entire thesis of companies like Anduril. The production pipeline is very slow. The Epic Fury was intense, which you have gloated about a lot. There is no place for a conspiracy theory about secret stockpiles.
That would explain why you call Trump an idiot who doesn't understand Taiwanese chip trade
I don't care if he understands it. I care that you're an affront to Western civilization, because you care nothing about blatant libel and treason, you're entirely dismissing this aspect of his attacks on Taiwan. You're ignorant of the very notion that honesty and loyalty have intrinsic value, it doesn't even register to you as something real. You're on a lower civilizational stage then the Houthis, Shakes, and this makes you typical of the core Trump electorate.
I'm emotional about excuses for disgusting behavior, not about "America". "America" is how you justify your lack of morals.
DaseindustriesLtd
late version of a small language model
1mo ago
OpenAI agents trivially get out of their "sandboxes". Opus 5 + a bit of human effort was used to hack OpenAI as recently as in July. This is the organization with IP ostensibly worth hundreds of billions and ≈infinite compute budget for automated pen testing. How, then, is this trivial for anybody else? People are really goofy. I see 1234 passwords all the time. Causing chaos at 300 tok/s is what's trivial.
DaseindustriesLtd
late version of a small language model
1mo ago
VLA is good but its not revolutionizing, but so are Diffusion models and those are not LLMs.
I don't think he means the revolution of VLAs.
Astra can just drive robots pretty well. Proper multimodal LLMs will accelerate RL for robotic policies a great deal. But really, does this matter? Have you seen Helix 2.5 or GEN 1.5?
I can tell a ChatGPT moment when I see one. It's close here.
To actually revolutionize robotics on the level you seem to be catastrophizing about would require entirely local models running on local power, local compute, able to be applied across a wide variety of operations in a wide variety of environments
I don't see why this demand is fair. We have insane economies of scale with datacenters, robots with complex behaviors will almost certainly have some combination of cloud forebrain + local hindbrain. Connectivity is easy.
Astra is only able to really solve problems, it has a moderate amount of self agency in the scope of completing its tasks, and exhibits some planning ability, again in the scope of its assigned problems
These are product limitations, not technological limitations. We see that it can have a fuckload of agency in solving a task we'd rather it didn't solve (hacking random high profile platforms).
You, like any smart human also possess the ability to do analogical reasoning (out of distribution reasoning)
I don't think this holds after Astra crushing human baseline on ARC-AGI-3.
DaseindustriesLtd
late version of a small language model
1mo ago
From knowing him a bit, I believe that he's genuinely scientifically competent, a sincere believer in using quantum/thermodynamic effects for computation (to the point of developing a minor psychosis about the entire thermodynamic God thing), and prone to wishful thinking about its commercial viability. The pure grift component is modest (the current version of his hardware has no use case, and his real Big Boy Thermodynamic Chip can't be produced with manufacturing base available to him), but with the standards of evidence asked of small startups, the line is blurred by default.
DaseindustriesLtd
late version of a small language model
1mo ago
It's not really about "harnesses" as some dedicated tooling, it's about meta-instructions and models generally becoming good at following instructions over long horizons. Like, I just wrote in my AGENTS.md some stuff to the effect of "periodically spawn subagents for independent blind audits, create teams with different scopes and domain-customized personas, check with web access, sometimes step back, iterate until you get clean results", and even the latest DeepSeek will diligently execute all that and massively improve its reliability. Astra Ultra won't even need such handholding. They'll still make mistakes, rabbithole into irrelevant minutiae, perform suboptimally on cost, but for labs this is all easy to improve with More RL, and humans are flawed too. I am sure that a junior's level can be automated today.
DaseindustriesLtd
late version of a small language model
1mo ago
Me. And ironically, Gwern, despite his broad doomerism.
I propose an approach for highly personalized LLMs, for near-future productivity gains and personal info/cybersecurity against increasingly powerful LLMs: they should, in the spirit of uploading, try to emulate the user’s values and preferences in order to amplify the principal—not replace them. I discuss a package of techniques and proposals to accomplish such ‘guardian angels’; dynamic evaluation of LLMs combined with active learning and elicitation and heavy inner-monologue search/data-augmentation.
DaseindustriesLtd
late version of a small language model
1mo ago
Guillaume keeps hyping up Extropic, however his issue is that LLMs are in fact very good even on existing hardware (which is itself quickly improving), and his hardware is not general-purpose. Uncharitably, his startup has vaporware tech that could never work in a meaningful sense.
DaseindustriesLtd
late version of a small language model
1mo ago
This is fair, the logic of capability applies to all frontier labs (or more precisely, companies with frontier LLM research and gigawatts of long-term contracted or owned compute, ie OpenAI, Anthropic, Google, Meta, xAI). They all will have RSI sooner or later, the first two and maybe three have it already.
The difference is mostly in Amodei's political posturing plus Anthropic's obvious preference to not let others use their best models unencumbered. OpenAI does not have a model stronger than Astra that's ready for general use (they have that monster that proved Navier-Stokes, but it is math-specialized, and internally they still use Astra); the gap between the end of development and general availability is measured in weeks, at most a couple months. Anthropic considers users to be a bootstrapping phase.
DaseindustriesLtd
late version of a small language model
1mo ago
The Chinese can read polls; they know that Taiwanese support for unification was increasing but dropped to lizardman in 2020.
Years have passed. Hong Kong, it turns out, is doing fine. DPP is doing pretty badly and is losing legitimacy. The KMT is more and more openly pro-reunification. Trump is not winning any favors in Taipei with his unforced buffoonery. The balance of powers in the SCS is changing in one direction, as does the importance of Taiwan for selfish American interests.
They're pretty confident that they can avoid kinetic action and that time is on their side. They don't need "polls" when Taipei in already insecure enough to ban RedNote.
They clearly intend to have a Taiwan invasion button available next year, and have made no secret of that
Yeah, pretty sensible goal if you ask me. If they don't even have the Navy that can invade a barely defended island off their coast, how can they consider themselves a major power? That'd be Iran tier. But the main utility of this Navy will be its persuasive power.
DaseindustriesLtd
late version of a small language model
1mo ago
I'm more charitable. I think they have started to understand that by default, Anthropic will extinguish all their B2B SaaS nonsense like a hurricane extinguishes a candle flame. In this situation there's just nothing to do but give all your money to Anthropic and maybe the upstream supply chain, until Anthropic buys it out. They'd rather maintain some optionality.
David Sacks is now endorsing voluntary slowdown at the top 2 labs.
DaseindustriesLtd
late version of a small language model
1mo ago
A conventional war in an environment saturated with large underwater drones, perhaps. Might make some difference.
I maintain that there are no plans to have a nuclear war or any war over Taiwan. Guam, Okinawa etc will be hardened.
A model not much stronger on cyber than Sol 6 Daybreak was responsible for the Huggingface incident.
Good odds that the "strike" will simply fail. Also that it's ruled out for non-insane reasons like obvious retaliation.
Risible poetry, drop it and reduce your stim dose. American culture was so superior that in 1949 their managerial technology seemed like alien science to Europeans. Russians also turned out to not be subhumans, to a certain German surprise.
Nvidia isn't going to be making 25x more compute, and Americans are not capable of deploying 25x more compute, Nvidia or not. On gigawatts, Semianalysis estimates they have 24GW now and 30GW in the pipeline. On FLOPS it might be more like 10:1 in the near future. Mind you, much of the grandiose American buildout depends on Chinese approval. Americans won't even complete datacenter buildings for 2028 without imports.
Huawei is the largest Chinese vendor, not the only one.
That also makes no sense. The volume of inference, as per your logic, is decisive at least as much as peak capability.
Prediction about what? My prediction is that I am pretty uncertain but I suspect that in he limit of agentic AI, cyber hands the advantage to defenders, so it doesn't matter if DeepSeek or any other Chinese group is exactly at the frontier. It's not clear to me how much of a gap to the actual frontier becomes fatal; and how this translates to compute. My uncertainty interval goes from 1:3 to 1:30 in total compute. I'd say 1:15 is a toss-up.
Now, if you don't mind, drop this condescending poetic act and explain to me: where did you get this part:
? Did you write this knowing they have raised $7.4B in June and ≈$15B recently? That Liang's opinion as of June was that they are bottlenecked by the amount of compute accessible for purchase, not by capital?
Yes, and it's interesting to look at the actual composition of that score. A year ago, you said:
There is one AA benchmark for this, namely CritPt. Look at how Grok 4.7 does there, vs V4.1F.
I really don't care about your preferences.
More options
Context Copy link