BigObjectPermanenceShill
No bio...
User ID: 4286
Could you at least add more substance than "Opus changed tokenizer, therefore, as I've already said, AI is a bubble"?
I don't pretend that AGI is some clean concept. For me, it means a very banal thing: an AI that can reliably replace a human worker. "I can point an LLM at a knowledge work task and it'll do it". Or at least: it'll commit to an honest humanlike attempt to do the job, it won't run out of context length, won't hallucinate something superficially related, won't trip on its own shoelaces; it'll reason about the problem, identify what it lacks, collect the necessary data, maybe do some trials in a scratchpad of sorts, consistently orient towards truth and common sense, do its best and then admit to me if something was still genuinely beyond its ability.
5.2 was a very big jump over 5/5.1 and it showed, in my opinion, a very powerful awareness of problems, an ability to contextualize and deconstruct them. 5.4 and the upcoming 5.5 clearly continue this trend. They've figured something out and I believe it's on the path to AGI as defined above, modulo technological details that seemingly won't be a long-term blocker.
Will it cure blindness and reverse aging?
Will anything? Will human scientists? I don't know. Plenty of things that human-level intelligence has so far proven unable to solve. But so long as science is knowledge work, yes I expect AGI to do it at least as well as we do.
I don't understand this claim. Who "we"? Most people learn almost everything they know about economically valuable complex domains from textbooks, manuals, teacher's answers and such second-hand information, and then polish it with on-site instructions and increasingly long-range, open-ended training. They don't build much in the way of their own "techniques and theories" and there's not a world of difference from what LLMs now do. Maybe you're overestimating how much they depend on pretraining at this point. Well, it's believed that >50% of compute in some of the last-generation models goes towards RL, not pretraining on human data.
And as I've said in the opening post: we have literally just seen an LLM employ a technique no human mathematician had thought of using in this specific context, to solve a problem that had remained unsolved since 1968 – over half a century! It wasn't some Riemann hypothesis tier challenge, but it wasn't exactly obscure either, smart professional mathematicians had been working on it for years before GPT 5.4 Pro came and did this. Moreover, GPT does this reliably. In the comments you can see Terence Tao, arguably the guy with the greatest knowledge of "techniques and theories" of math on the planet Earth, an expert of such level that he actively avoids getting roped into solving other people's frontier research level problems, seriously engage with GPT's work:
Thanks! So there does seem to be something special about the original von Mangoldt process - the associated invariant measure ν is extremely smooth (in the Archimedean sense), being asymptotic to 1/nlogn , while all the variants of this measure pick up arithmetic factors such as 1∏pvp(n)!
- A little surprising to me that removing individual primes instead of prime powers makes it less likely to have prime multiplicity, but I'll chalk it up to one of the numerous probability paradoxes that arise when one tries to compare various weighted expectations. But these factors mean that one cannot immediately solve #1196 by using these processes instead of the von Mangoldt one, as the invariant measure is no longer asymptotic to 1/nlogn
- So in some sense the AI was "lucky" in finding the one approach that actually worked; it would be interesting to publish the traces to see if there was a lot of brute force involved in trying nearby approaches which didn't quite work.
……
Arb Research has kindly shared with me ten separate runs of GPT 5.4 Pro on this problem #1196 (with a request not to use internet search). From a quick reading, it appears that 8 of them claimed successes, with the other 2 rating the claim as plausible. Interestingly, several of the successful runs actually obtained the sharper formula ∑n≤Aν(n)≤1 that was also derived here, with ν essentially the Mellin transform of 1/ζ(s)
- Almost all of the runs latched on to the approach of constructing a random chain with a good hitting probability (many runs referred to this as the "Lubell method", after the Lubell of the LYM inequality).
Another notable fact is that none of the runs highlighted the von Mangoldt process that was a prominent feature of the original run (and none of them mention flow networks either). Runs 4 and 7 have an interesting alternate construction of the upward divisibility chain in terms of exponential clocks in the prime factorization indices that actually looks rather tractable to work with; I will need to study this construction further when I have more time.
Basically it seems that for this particular type of problem there are several natural ways to proceed that make the problem actually quite tractable; the literature had managed to focus on a somewhat suboptimal approach in which the opening move was to transfer the problem to a continuous setting, but the AI runs consistently stayed in the discrete world and managed to utilize various existing tools from discrete mathematics (mostly centering around methods relating to the LYM inequality) to reach a solution.
So I don't know. Where's this inherent limit on complexity that you're talking about? What in our culture is truly irreducibly complex, if not math that can surprise Terence Tao?
This is getting a bit comical, don't you think?
Intuitively, it seems unlikely that the result will be any better than what you started with. And apparently both experiments and mathematics indicates that what happens is "model collapse," i.e. with each iteration the new model performs worse.
Yes, this follows from data processing inequality.
Assuming that's all true, it follows that LLMs must be missing some essential attribute possessed by human brains. Because we apparently picked ourselves up by our bootstraps and created from scratch all the text which is used to create LLMs.
No. It applies just as well to humans. And humans did not build a civilization by thinking really hard at a corpus of word sequences. Oh, we tried this too, to an extent, and got wonders like Sophistry, Rabbinical Judaism, Medieval Scholasticism, Marxism and Rationalism. But we mostly progressed by receiving environmental feedback, filtering the generated data and preferentially training on validated fraction. Similar logic can be applied to LLMs (or any ML artifacts). This is why the basic trick of the current paradigm is RLVR (reinforcement learning with verifiable rewards). You finetune a model on successful trajectories, then you give it tasks and update towards policy that has generated correct conclusions. The primary source of updates is the model itself, steered by an external verifier. In principle they can do this fully autonomously, by building an ontology of possible tasks that can be algorithmically verified, coding these verifiers, and generating (eg relying on web search) queries against these tasks.
Even under very rudimentary realistic assumptions, generated data improves model performance.
OK but do you agree that "Anthropic has slightly altered their tokenizer in a 0.1 update for Opus" is not really "controversial issues that fall along set tribal lines"? Which tribe has a strong position on Anthropic's tokenization design choices?
I think mods should intervene… somehow, because these posts are getting too frequent, too obviously agenda-laden, and aren't even remotely about the culture war (though AI discussion as such is necessary). It's becoming one guy's AI Bad blog.
Look man, it seems that the Opus 4.7 tokenizer change functionally amounts to them forcing each whitespace be a separate token rather than part of any subword, removing all whitespace-containing subwords from the vocab; it does not change the compression rate for whitespace-free languages. I do not know why Anthropic did that, but my hypothesis is that they've found in experiments that this is better in some valuable scenarios, such as related to analyzing code for vulnerabilities; trained Claude Mythos with it; and now are pushing Opus further via distillation from Mythos (this is suggested by it being weirdly different, and them saying they now focus on GraphWalks, which Mythos is doing really great on, for evaluating long-context performance).
For logprob distillation, you ideally need identical vocabulary (there are copes for inter-tokenizer logprob matching, but better just change the student model's tokenizer and heal it).
As a datapoint in the timeline of AI progress, it's a total nothingburger, a non-news.
Anthropic's move here (combined with them handicapping Opus 4.6 a few weeks ago) seems to clearly be an attempt to achieve profitability.
Do you realize that while this is bad for users, it's not that good for Anthropic? The compute and memory cost per a sequence of 1 million tokens is the same whether these tokens encode 1 million or 500 thousand English words. It doesn't improve the profit margin. Of course, now that everyone's codebase is functionally like 40% "larger", they are selling more tokens to their captive clientele for each plaintext-identical request. But this would be such an awkward growth hack. And on Claude Plan, cache is free anyway, so their margins could even shrink.
For everyone here, nut perhaps especially the AGI believers, have your feelings changed at all over the last few months?
Yes. After GPT 5.2 I've become a bit paranoid that we will have AGI before 2028 and are totally unprepared. Recent events such as GPT 5.4 autonomously solving Erdos #1196 with a trick that no human mathematician expected corroborate my feeling.
Pride is a rough master, and that velvet glove / participation trophy empire thing that the US was doing with Europe just sparked resentment.
If you think the US is currently treating Europe with more respect than China does and THAT explains the decline of loyalty to Atlanticist project, this speaks to your enormous, delusional entitlement. Just for one thing, Xi gives kingly receptions to European dignitaries, while Trump says they're kissing his ass and publicly relishes refusing their pleas for lenience. I suppose he operates on your theory too.
Seeing upvote/downvote ratio, I guess little more can be said.
No, they're just quislings who were already sucking up to China because China is an asshole that thinks they're subhuman.
I hope you realize this sounds a bit unhinged.
Europeans here have commented plenty on the asinine degree of hatred the MAGA crowd had suddenly developed for them. I think it's a reaction akin to jealousy, and it's always ugly and pitiful. Little more to say. Except:
China is retrenching and reconsidering Taiwan plans after watching their tech fail the last four months
Nothing of the sort happened, the only active "Chinese tech" I'm positively sure has been defeated was HQ-2 that dates to, like, Cultural Revolution. Xi is currently negotiating with KMT representative, in China. They're feeling really comfortable. You seem to inhabit a thick MAGA bubble. This is unhealthy. Try to check out alternative sources, and preferably not the outgroup but some professional ones.
To the best of my knowledge, the terms are as follows:
1—Commitment to non-aggression
2—Continuation of Iran’s control over the Strait of Hormuz
3—Acceptance of uranium enrichment
4—Lifting of all primary sanctions
5—Lifting of all secondary sanctions
6—Termination of all UN Security Council resolutions
7—Termination of all Board of Governors resolutions
8—Payment of compensation to Iran
9—Withdrawal of U.S. combat forces from the region
10—Cessation of war on all fronts, including against Hezbollah in Lebanon
They do not say anything about collecting the transit fee (good global diplomacy), but seem to directly demand reparations from the aggressor parties. Not to mention all the other stuff, especially #9 and #3. (Sanctions/resolutions items and war items are I think reasonable/obvious given the situation). This is much more hardcore than what they've been asking for yesterday, in response to Kushner-Witkoff's undisclosed 15 points list, allegedly a capitulation-with-extra-steps ultimatum. And their old 10 points were already bold, including the bit about $2M per ship that they dropped in favor of… well, all this.
I have seen no evidence that Trump is unaware of the specific content of the list. He's reposting Iranian statement, which is devoid of details.
Needless to say, the new list amounts to strategic defeat for the US. I don't see how Trump can accept it in good faith (nevermind Israel; Israelis are seething or coping, and of course will in any case proceed with the usual lower-level conflict, just as Iran will, seeing its preference for maintaining its proxies). So it's another 2 weeks of market manipulation, threats and TACO, if not straight up back to war, I guess. This is surely far from over.
But the sheer fact that Trump was pushed to go from "tonight, a whole civilization will die" to using Iranian list instead of his son-in-law's is telling enough. Beyond this point, this won't be a clean win for the US no matter what.
Trump has caused hundreds of billions in damage with Liberation Day tariffs, attacked Denmark and forced EU allies to orient towards China, basically wrecked NATO by this point, foolishly escalated against the same China and got humiliated in Busan, exposing American industrial ineptitude (particularly to Korea), is about to lose Taiwan, and is in the process of shaving off 1% or so off the global GDP growth. That's just the big foreign policy stuff I care about, domestic policy is discussed daily here.
That loyalists conveniently forget such issues or reframe them into WINS is unsurprising.
Donald Trump did not trick his way into office and then surprise everyone by acting in a manner unbecoming of a president
He apparently surprised Catturd and other major boosters who were celebrating NO MORE FOREIGN WARS upon his election. That said, they've quickly pivoted, now foreign wars are Based.
The people of the United States of America decided to elect someone who breaks all the rules of what a president is 'supposed' to do. They don't care what the president is 'supposed' to do, they care whether the president is doing what they want him to do.
This is mostly sophistry, but I suppose you are making a sharp point: the problem is not Trump, it's the American people.
Four years of ineffectual flailing due to political sabotage, 4 years out of power, and now a bit over a year of uninterrupted and unprecedented shitshow. The list of his follies is very long, but an American consumer is very rich, so can ignore it for a time. Still, it hasn't been a long time. What did you mean by ten years?
Can't argue with gigachad responses.
Yeah, that's the kind of thing people said when Dubya was compared to his father, whether it be the vomiting incident or the broccoli one. Yes, each Republican will be worse than the last, I understand.
Just so I understand, too:
While attending a banquet hosted by Japanese prime minister Kiichi Miyazawa on January 8, 1992, U.S. president George H. W. Bush fainted after vomiting onto Miyazawa's lap at around 20:20 JST. The incident took place at the Naikaku Sōri Daijin Kōtei in Tokyo, the Prime Minister's personal residential quarters. Doctors later attributed the incident to a case of acute gastroenteritis.
"During his tenure as the 41st president of the United States, George H. W. Bush frequently mentioned his distaste for broccoli, famously saying: "I do not like broccoli. And I haven't liked it since I was a little kid. And my mother made me eat it."
And now:
Tuesday will be Power Plant Day, and Bridge Day, all wrapped up in one, in Iran. There will be nothing like it!!! Open the Fuckin’ Strait, you crazy bastards, or you’ll be living in Hell - JUST WATCH! Praise be to Allah. President DONALD J. TRUMP […] A whole civilization will die tonight, never to be brought back again. I don’t want that to happen, but it probably will.
Is that it? Is your reaction to a claim that the latter is qualitatively worse than the former just an eyeroll about those pearl-clutching libs with TDS who nitpick at gaffes and bully awkward Republican presidents?
Have you considered that it is in fact possible for your side to be getting considerably worse over time, or rather, that specifically Trump is worse than Dubya?
Do you have any absolute frame of reference, or is it just anchored to the current intensity of lib chatter?
It seems to be common knowledge that the Left had in some ways gotten worse over time. Do you say it's a priori implausible the same has happened on your team?
No, they get others (including Hezbollah)
I presume you mean Beirut in the 1980s. Wow, you sure have been around for a long time.
- yes, it is hyperbole in the case of Iran. This has been covered enough. Iranians have, in fact, never had the genocide of Americans or even collapse of the American state to be their policy.
- Iran cannot credibly threaten death to America, or even Israel, for it has no means to inflict such death. On the other hand, the US can credibly send Iran to the Stone Age, or whatever, with near-complete impunity.
- I do have double standards. An ostensible vanguard of the Western Civilization cannot conduct itself like an insular theocracy it says is ruled by "crazy mullahs".
- This victim posturing is exactly what I've been talking about, it's unbecoming of a superpower.
Chuck Norris jokes as policy announcements.
Does it not tire you to maintain this disaffected persona chuckling at "lib" overreaction? Do you really believe that the proper treatment of POTUS's words is "unhinged and obviously non-credible bullshitting", but also that this kind of person deserves to be POTUS?
It so happens that if you are too stupid and malicious to consider consequences, you can do a hell of a lot as the president of the United States of America. You can do great things indeed. The greatest. It's a tremendous force, this office.
For example, you can kill a whole civilization.
You can realign everything into a crooked parody of itself. You can throw a temper tantrum and wreck international treaties, replacing them with your preferred club of third world kleptocrats. Easy-peasy. The hard part is getting into that office, but thankfully even the smart Americans have grown tired of treating things seriously, and so happily elected a random moron.
If you have to go all the way to Jackson…
But America was a bit player back then. I am not sure even Jackson would have been such a jackass in charge of the global hegemon.
Personally, this image of "Iranian Shia Muslims" being insane brown terrorists has been one of the casualties of the war for me. Was it always just Israeli-American propaganda, MemriTV cherry-picking, or merely a product of general American ignorance about the broader world, "everyone in the ME is Arab, except Jews who are white"? They don't really look the part. They don't do suicide bombings like Sunnis, they don't deal in over-the-top theatrics, they state conditions and try to execute on those. Throughout the war, Iranians have communicated much more similarly to how I'd expect a European nation – say, Sweden, or Denmark (we've just had a dry run with Denmark, come to think of it) – under attack to communicate, compared to the coalition of Moral Clarity. "This is the Middle East" is a bad excuse for the American side, so bad in fact that it vindicates their desire to have nuclear deterrent. Same logic as Israel uses. You can't expect to be left alone by such people without WMDs on the table.
They don't seem as religiously unhinged, too. Very little talk of Apocalypse.
Very funny to see people trying to normalize Trump's behavior by comparing him to Dubya, or doing the usual "but the libs", or "TDS". We're not in Kansas any more, this isn't about stutters or being uncouth, Dubya had nothing on this guy, irrespective of bad faith criticism and outright slander by his opponents. Trump 2.0 is a sui generis in American history, whether in his ineptitude, corruption, or malice (or, yes, aesthetics). And the only TDS I see at this point is unquestioning, defensive loyalty to Trump that has become completely untethered from his faithfulness to any policy line – except, perhaps, owning said libs, making them seethe.
But perhaps that's all there was to it from the start? Conservatives felt bullied, mocked. Conservatives were fed Romney-style polite retreat or, if they grew intellectually curious, Moldbuggian defeatism about Cthulhu swimming left inexorably, like a law of nature, about the Elite Human Capital destined to convert their kids into hating their bloodline. Conservatives wanted to know how it feels to be on the other side of the boot – just once. And so long as Trump grants them this wish, so long as the moment of ecstasy continues uninterrupted, he can do no wrong.
This is pretty much how Palestinians saw Oct 7. When you're convinced, rightly or wrongly, that you're a desperate underdog, no manner of retaliation feels unjust or unwise – punching up, as libs like to put it. Of course, Israeli politics are the same, perfected, elevated to the core of religious doctrine – eternal righteous lashing-out of a cornered rat, since the Holocaust, since Titus, since the Pharaoh. Iranians are lashing out against the Satan Duo and the entire global economy now. Russians imagined themselves boxed in by NATO, post-colonial third worldists chafe under the White Man's revealed superiority and asserted colonial sins, the Chinese are fantasizing about revenge for the Century of Humiliation… Perhaps the root of evil is plebeian resentment as such. Elites that can feel it themselves are an unacceptable hazard, and political systems that reward fanning and exploiting this loser sentiment are powder kegs. What can stop this? Fukuyamists hoped it'll be the sheer sedating comfort of the unfathomably rich liberal democratic order. Didn't work. Christianity offered some lofty words about forgiveness, but Christianity is now a garish Easter Bunny mascot that ushers in gleeful war crimes on Passover. Praise be to Allah, I guess.
Why is the $500B being discussed as a news? Trump had announced his plans for a $1.5T military budget long ago, in early January iirc (was a bit hard to find).
To quote it in full (I've just been told "Don't pay any attention to anything on Truth Social ever and you'll probably have a clearer view of world events" but just this once):
After long and difficult negotiations with Senators, Congressmen, Secretaries, and other Political Representatives, I have determined that, for the Good of our Country, especially in these very troubled and dangerous times, our Military Budget for the year 2027 should not be $1 Trillion Dollars, but rather $1.5 Trillion Dollars. This will allow us to build the “Dream Military” that we have long been entitled to and, more importantly, that will keep us SAFE and SECURE, regardless of foe. If it weren’t for the tremendous numbers being produced by Tariffs from other Countries, many of which, in the past, have “ripped off” the United States at levels never seen before, I would stay at the $1 Trillion Dollar number but, because of Tariffs, and the tremendous Income that they bring, amounts being generated, that would have been unthinkable in the past (especially just one year ago during the Sleepy Joe Biden Administration, the Worst President in the History of our Country!), we are able to easily hit the $1.5 Trillion Dollar number while, at the same time, producing an unparalleled Military Force, and having the ability to, at the same time, pay down Debt, and likewise, pay a substantial Dividend to moderate income Patriots within our Country!
PRESIDENT DONALD J. TRUMP
Now Trump is not very numerate, so he presumably misunderstood how the tariff revenue changes the long-term balance, and he's seemingly still committed to at least not blowing up the debt further; so something had to give.
And yes, this is about China, and anyone else who might pose a challenge (as can be seen, even the decapitated Iran is something of a challenge). China has been steadily increasing their defense spend by 7% a year, for the last decade or more, and their 2026 target is ≈$277B, keeping it well under 2% of GDP. In 2027, accordingly, we may expect almost $300B. Given the differences in productivity, in fighting age male population, the level of corruption, and the share of US DoD (DoW) spend on veteran benefits and such, a 70% overmatch was too close for comfort.
Would you prefer it if I said that the difference is just that Europe is finally doing realpolitik on its own? They're sometimes too slow, for sure, as Macron puts it.
Proactively provide evidence in proportion to how partisan and inflammatory your claim might be
After moralism, rules lawyering? I don't think this rule is being enforced with any sort of consistency, and after Europe's response is, "America, control Iran better! and other such claims I don't think you get to ask for it to be enforced. But to be clear, by "you" I mean "Israel + the US", since you clearly are operating as a coalition.
We have certainly killed fewer than the Iranians own government did a couple months back
Nation states are defined by having a monopoly on violence within their jurisdiction. Your opinion that the Iranian regime is not legitimate (whereas your own regime is sacred and must not be violated by foreigners), and that it is acceptable to demand submission to a foreign-backed attempt at violent revolution, is just that, an opinion; it doesn't give you the right to kill any extra civilians. Moreover you're not even doing it at the right time, should have joined in while protests were actually happening, when Iranians were killing cops and getting killed in turn.
I see estimates at > 1000 civilians dead, no idea how valid that is, except for the now-indisputable Tomahawk strike on girls' school (which Trump rather pathetically denied). I have seen footage of massive destruction that includes clearly residential housing, and the way these things go, that should have cumulatively taken some hundreds. The high estimates of 30-40-60 thousand killed in protests appear to be simply made up as well.
The Iranians I see who can still get the occasional internet access say that they aren't afraid of the bombs, they're afraid of the bombs stopping
Might be the most bizarre war ever, the way Americans all suddenly have contacts in Tehran and are grasping at straws, straining to hear the voices of unbroken opposition, to convince themselves that they're liberators, even as their Secretary of War is plainly relishing the opportunity to break a nation.
Though this reminds me, people liked to see "all the pretty Iranian woman trying to learn Trump's YMCA dance a few weeks ago". Since then, the most viral of these pretty women had her cousin "killed in a war". She objects: not a war, a rescue operation! "The reason I lost my cousin is only and only the Islamic regime and no one else!". So yes, there is something to your point.
Nobody asks now how many of these dissidents or incredibly Zionist diaspora monarchists (very funny movement, imagine wanting this guy to be your king) are children of SAVAK, or astroturfed by Israel, or not even real. What they say soothes your conscience, so they get to speak for the will of the Iranian people as a whole. The poor Iranian people now apparently need their entire civilian infrastructure wiped out to Truly rise up against the brutal regime. We'll see how it goes.
Don't pay any attention to anything on Truth Social ever and you'll probably have a clearer view of world events.
I don't actually read Truth Social, and I'd prefer if the President of the United States of America didn't use it as his platform of choice, but reality is often disappointing.
The military is ran by a visibly incompetent Fox News host with a drinking problem, who's such s good person his mother condemns him for abuse of women.

We have a good idea of how to train AI to solve mathematical problems, of virtually unbounded complexity. In the course of this, AI clearly learns "techniques" as shown here, if not "theories". I don't think King's prowess is theory-driven either, but in any case we don't have a good idea of how to train AI to be a good prose writer. We have some ideas, but are unlikely to act on them. There's not much money to be made in it, and plenty of highly motivated enmity – AI is already widely hated. and yes, autoregressive generation for the prompt "write like King" is not like King actually writing a novel. We have such tricks though.
My point is, it's not a general principle that AI will only rehash human techniques in some uninspired "probabilistic" way. If there is a hill to climb, such that "good" and "bad" outputs with regard to the problem statement can be distinguished, AI can bumble its way up the hill and also find new tricks. We've seen this before LLMs, with AlphaGo and move 37, we're starting to see it with LLMs.
Human mind runs entirely on probabilistic mush. Neural networks were invented as approximation of our own approximate learning. But probabilistic decision processes can have clear enough decision boundaries that they become able to operate with "abstractions", "symbols" or "theories". They also remain able to fail. For example, you are failing to update on evidence, because you haven't been trained to take input like "Terry Tao is surprised" seriously and think it's infinitely less interesting than your preconceived notions, basically some dweeb noise. Unlike an LLM, you can update at lifetime, so maybe you'll reread the above post and see how it contradicts your position.
More options
Context Copy link