This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Dead forum?
Today in AI news, OpenAI has reportedly released GPT-6 Astra. Rumors on X suggests that it scores 99% on Arc-AGI-3, meaning that the benchmark is essentially dead now. For comparison, Fable only scored 30%. Does this mean AGI is here? Many are saying that it is. Color me skeptical, but its getting harder and harder to find concrete tests that AI cannot pass. The god of the gaps gets smaller.
Many intelligenct people have important things to say about this. But what do the 84 year old communists think? Bernie Sanders today has introduced legislation calling for a ban on superintelligence with a 20 year prison term for attempted superintelligence. He also wants (shocked Pikachu) a government regulatory agency. Although he mumbles something about international cooperation, its clear that China will not be subject to these rules so it solves nothing.
I will give him credit for at least recognizing the importance of the issue.
We could be entering a world of unimaginable change. And, given the political systems in place in the US and China, the midwives of that changes will be those on the cusp of senility.
With all the renewed hype around Astra and AI mathematics, I have been feeling that I haven't really seen a response I'm satisfied with from the AI bulls (at least the imminent mass unemployment / AGI / singularity types) to the 80IQ skeptic question of "If AI is so good, why is nothing happening?".
FWIW, if I knew in 2020 that in a few years we would have AI models that can get a perfect score on the IMO for pennies and meaningfully contribute to research mathematics I have expected that we basically live in a sci-fi world as well; yet now we have these models for $20 a month, and literally nothing I'm aware of has happened on a macro level clearly attributable to AI, either quantitatively or qualitatively, apart from the distortion of the AI capex buildout. In fact, it's actually GLP-1's that are basically a complete miracle on a macro-economic and public health level, and where there actually are many macro statistics that are visibly affected by GLP-1's.
On some level there probably is anthropomorphization going on here where the natural human assumptions are that
when in fact the conclusion does not actually follow from the premises; it is possible that AI is extraordinarily economically valuable and powerful for other reasons, but this does not follow from being superhuman at mathematics.
Modern pure mathematics is, as we seem to be finding out, not particularly economically valuable in and of itself; mostly mathematicians are lionized for their intellect because a human with that level of intelligence could obviously do significant amounts of other economically valuable work, but this seems extremely non-obvious in the case of AI.
The "best" response I've seen is something along the lines of "actually AI is still heavily limited by spikiness right now but the exponential is going to keep going bro" which I suppose is honest as to current capabilities and not disprovable, but it does feel like a retreat away from "AI is already crazy good" as many people have been and are touting, and towards magical thinking; somewhere along the line the party line swapped from "AGI imminent" to "AI that creates AGI imminent".
My take-away certainly isn't that AI is a fugazi; obviously it's powerful technology and I'm still quite bullish in the mid-long term, but I've certainly been feeling myself getting anti-hyped with recent releases as a result of this growing discontinuity of any real-world effects.
One thing I've noticed is that a lot of these mathematical breakthroughs involve lower bounds, upper bounds, or counter examples, and they usually involve the LLM interfacing with some kind of deterministic tooling.
Sometimes I wonder how much of this is the LLMs themselves, and how much is what happens when mathematicians finally get access to hundreds of billions of dollars in computing power, albeit through a weird interface.
Would we have 5% of these breakthroughs with equivalent computing power without LLMs? 50%? 90?
I don't know if I have a good answer, but I do wonder sometimes.
FrontierMath Erdos tries to address these questions: https://epoch.ai/files/frontiermath-erdos.pdf
They enforce no human guidance and a limited token budget ($300). Under these conditions the only model that solved 2 problems is GPT-6 Astra.
With no budget limit GPT-6 Astra has solved 5 problems out of 68 (3 proofs, 2 disproofs).
The paper's conclusion: "These results establish, under controlled conditions, that a current AI system can autonomously resolve open problems of genuine mathematical interest."
And, well, there's no other known way (besides LLMs) to convert compute into solutions of mathematical problems in general. There are SAT solvers, but they are limited in their scope and, being a fixed collection of heuristics, they don't scale that well. Brute force search is usually out of the question. So, I would say it's about 0-1% without LLMs. And taking into account that LLMs are the major driver of compute buildup, we wouldn't have that much compute without them.
More options
Context Copy link
More options
Context Copy link
I’d caveat that there have been some massive changes: translation is basically solved, ‘Rosie-the-robot’ style control is no longer science fiction but a creepy and dumb product, ai-assisted chip development and software engineering is a default option. Recursive self-improvement has been slow, rather than non-existent.
Some of the delay is just logistics. There’s probably over a hundred people worldwide following the AI cancer vaccine dog’s approach, but it’s going to be three or four years before we have clear and convincing academic papers even assuming it works and generalizes, just because that’s how medical trials work. AI driven assistance and funding for nuclear reactors might make building them in the US possible, but it isn’t going to make the NRC respond in less than six months. Silicon fab cycles are measured in years. Structural stuff just can’t be that fast.
Others are tooling. There’s a massive space where many small and medium businesses would benefit from custom code, for example, and AI can do it… but if your interface and sanity checks require a programmer or project manager anyway, it’s not really available yet even if the AI can do it. And Claude Code is not ready for prime time use by normies.
A lot of stuff is also just obscured when it does work. An AI-derived optimization algorithm for matrix multiplication just gets swallowed as a delta in efficiency from expected performance. An AI-built software tool just looks like a (verbose) software tool.
I personally think the effects of current AI on software engineering productivity are somewhat overstated (at my fairly AI-pilled software firm there's been many "AI-assisted" design documents and RFC's of dubious value, lots of rewrites of internal platforms, test suites and frameworks that look productive but haven't really been that useful, and only somewhat modest improvement in terms of actual firm-wide user-legible value), but as a whole I would agree that software engineering productivity has been meaningfully enhanced via AI.
There doesn't yet seem to be meaningful impact on software engineering employment, or anything noticeable on a macro scale as a result of modern LLM coding, but I think I would agree it is too early to determine the wider effects of coding agents one way or another.
I think translation is an great example of what I mean by "if AI is so good, why is nothing happening"?
We've basically had "solved translation for dummies" since 2016 Google Translate NMT, in the sense that the output wasn't great, but someone with no knowledge could bumble through understanding the gist of a brochure or a manual, where previously it might have been literally impossible for them to understand the text without consulting another human. I remember quite a few people in language learning communities I was part of at the time were already getting blackpilled on the full automation of translation as an inevitability all the way back in 2016.
LLM's took that up one step further, since about early 2023 we've had "solved translation for normies" in the sense that LLM's could reliably provide semantically correct translations for pretty much any input/output language pair. Maybe it makes some minor mistakes or it sounds weird to the critical reader, but nothing that really matters in the vast majority of use-cases for the vast majority of people.
Most if not all my friends in 2016 would have agreed that the existence of these models would be the end of human translation, and yet in nearly four years there still has been no significant macro impact on translator employment, or really anything noticeable on a macro scale that might be noticeable if translation was truly "solved".
So then, why is this case? This topic came up when I was chatting with some translators the other day, but I wasn't really satisfied by the points that ended up being brought up.
Is AI translation too expensive? Perhaps plausible for the earliest LLM's, but certainly not any remotely recent model, at least when compared to the fully loaded salary of a first world translator.
Is AI translation not reliable enough? I thought this might be the case, but I was assured that this isn't really true for most translation jobs. It certainly matters to have a human liability shield for regulated medical, diplomatic, legal use cases etc, but most translators don't work in those niches, and for most translation work accuracy is not really that paramount - most people are happy with "good enough" and people who care about translation quality have always been a small minority.
Is diffusion simply incomplete and all the translators will be unemployed in X more months or years? Plausible, but I'm not convinced this can be the entire story either; at this point more than half the planet has tried ChatGPT or Gemini, and AI is top of mind in nearly all corporate boardrooms right now. Even if diffusion is slower than expected, I would think it reasonable that after nearly four years there should be some sort of real world impact.
Are there induced demand effects outweighing the supply increase? Certainly demand for Japanese<->English, Chinese<->English and Chinese<->Japanese translation (and likely lots of other language pairs) has exploded in the last 10-20 years; the expectation now is for games, media, writing to be rapidly localized across many different languages when you used to have to wait years for any kind of translation if at all.
This can't be the entire story though; if translation really was solved, it wouldn't matter how high demand was when AI would be meeting the vast majority of it. Perhaps it's possible that most translation is solved, but induced demand for the translation that is not solved means that most translators still have jobs?
I agree that logistics and tooling are significant bottlenecks for anything that needs to interface with the physical world, but I think it's a logical criticism to make that even solely in the worlds of bits and words there's really been very little happening in the world relative to the capabilities of LLM's.
In general I don't think I've seen a rigorous argument supporting the idea that "AI just needs X more months to develop Y capabilities and it'll have Z real world effect", when it seems that nobody really understands what capabilities are missing from the current models that are currently preventing them from having major real-world impact in the first place.
I'm not sure if I would even go that far. I work at a software company, and I do some consulting on the side. I've been able to watch a lot of AI-coding initiatives from the sidelines due to that consulting, and I see the same pattern a lot.
First, management picks something they can measure. Lines of code, commits, and PRs merged are the most common ones. This causes a certain kind of climber who doesn't actually want to be a programmer to ruthlessly optimize for whatever the metric is. Management lauds that person as a "10x/100x/1000x developer" and they receive glowing performance reviews while spending their days giving presentations to their coworkers about how to avoid being Left Behind In the Permanent Underclass.
At the same time, the rest of the team ends up drowning in this person's "work". Their PRs are 30,000 line, codebase-spanning fever dreams that come with "helpful" documentation that doesn't make any sense. The "spec" is under-defined and the code fails to adhere to it. The tests are made up nonsense. Management softly forces to team to rubber stamp the Rockstar's work. Nobody knows what the hell is going on, defects rise, baffling production down events occur, and nobody can figure out why.
The net productivity for the organization doesn't actually change much. What the technology does do is shift perceived productivity into a small group of people who have offloaded all the work to everybody else.
The con man who job hops has been a rampant problem in this industry for decades. LLMs have just systematized and automated it.
I think if you're skilled and disciplined with your LLM usage, it's pretty unquestionable that software engineering productivity has improved.
For example, I've really enjoyed using Ghostty, Dwarfstar and pi; each of these excellent software projects are written using LLM's with no slop in sight, but importantly these projects are also owned by mitchellh, antirez, and badlogic, who are three incredible programmers in their own right.
That being said, I basically agree with you that ironically, the natural-language programming interface is actually extremely difficult to use correctly, and many AI coding initiatives aren't going particularly well because it's so easy to default into slinging slop and hence productivity improvements are hard to find in many cases.
I'm optimistic that over time the industry will develop better norms and understand how LLM tooling should and should not be used, and we'll all be able to reap the benefits (a man can hope...).
Overall, I am less optimistic than you. It seems like the incentives are too broken for us to reach a new equilibrium any time soon. If we do get there, it'll be on the other side of a graveyard of companies that got too greedy.
Yes, I don't disagree that there will be a lot of churn and pain in the meantime.
I find it's practically a second part-time job to hold back the worst impulses of my non-technical coworkers, the junior developers and management to slop up tickets, documentation and the codebase, which has been somewhat taxing. Unfortunately, it'll probably get worse before it gets better.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
That seems like a big-company problem rather than an LLM problem. I work in a small startup and while I won't claim AI-coding has been without complications, if you tried something like that at my company you would be fired immediately.
That is, the manager would be fired for proving they are totally unable to be trusted with a codebase and have no idea how development works, nor any understanding of what is being done under them.
I've seen it repeat at every scale, from legally-a-small-business to a fairly memorable consult at a Fortune 100 company.
I won't argue as to whether your group has some secret sauce or unusual competency, but I will say that what you have described is absolutely not the norm at the kinds of places that will contract me.
Given your work, is it plausible that you are only going to get contracted by the sort of place that’s absolutely addicted to being Goodhart’d to start with?
I can't rule it out. By the time a company calls me, they're generally fucked to the point where they'll call me.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I keep thinking about this in the field of advertising, where AI is just as good at many creatives at coming up with ideas – and most clients don't care sufficiently about or cannot see the quality gap that talent creates anyway.
I think the difference is that even if a creative gets their ideas from AI, the job isn't just writing down some ideas. Creatives then have to convince the rest of the team, who have to present the idea to a client, who has to buy it believing that their agency is ready to produce it and genuinely likes what they propose (or at least thinks it will look good in their reel). But current AIs aren't build to convince people, even though they are pretty good at making a case for something if you ask them to. Because you have to ask them to persuade you, the efficacy of their persuasion capabilities is inherently lower.
Somewhere along the way (and I have not articulated this properly yet), I have come to the conclusion that today's obedient AIs do not have the skin in the game necessary to contribute to certain types of group dynamics which are at the core of many business endeavours. For this reason I think a different form of more autonomous AI is going to be needed to realise the job replacements possible, and we may need AIs with stable personalities that develop track records in specific fields and must compete against humans 'in the open', before they can take on a whole swathe of white collar jobs that on the surface look eminently replaceable already.
I'm also finding it quite hard to articulate, but I agree that there seems like there's something missing in terms of "volition" or "creativity".
Pretty much all white collar work above a junior level involves a lot of "deciding what should be done next", "convincing other people about what to do next" and "taking responsibility for what to do next"; at least for now LLM's can't really do any of this, even if LLM's are now very good at hill-climbing individual tasks that comprise jobs.
More options
Context Copy link
More options
Context Copy link
As a professional translator, there has been a significant micro impact on my employment, in the sense that work (which has been mostly MT post-editing in any case for 5-10 years) hasn't completely dried up, but the monthly workload has definitely halved in very specifically the last half a year or so. Before that, I had as much or even more work than before, at that point a major customer announced that it would basically transform to LLM-based workflows on an important project bringing that project mostly to a halt, and whatever work there has been to replace it hasn't been sufficient to react previous levels.
The levels might have started inching up recently (there are faint signs of the predicted "they bring the humans back in after throwing them out en masse" cope actually possibly being true) but I'm well on my way to get out of the field in any case.
More options
Context Copy link
Your anecdotes about translation are very interesting, I hope you keep updating us.
I have a theory on why AI wouldn't show up much in the economy even if it was very good at what it does. I do use AI, I think it's great for websearch and stuff (in no small part because Google is now terrible at websearch). I am not a programmer so I don't have strong opinions on how much efficiency it's actually adding there, but while I am open to criticisms of AI as a technology, my theory is agnostic on that and I think still makes sense even if AI is in fact pretty good as a technology, although I wouldn't necessarily claim it's monocausal.
Simply put, I don't actually think much wealth is generated through software directly.
Software is extremely good at streamlining wealth creation; that's why the world runs on Excel. But at the end of the day your accounting guy is making sure that you are efficiently allocating resources, not directly generating resources. (Software also makes it easier to generate inefficiency, too, but let's ignore that for the time being.)
This can have a huge impact on the economy if you're jumping from "I need to hire a scribe to write a letter to travel through bandit-infested territory to inform my business partner about recent developments" to "lemme sit down and write an email real quick" but in a world where you are going from writing a quick email to having AI write a quick email for you, you're not really streamlining things much more.
I think people who work in software sometimes forget that software is just a tiny part of the economy. So from their perspective, if they increase efficiency by 20%, that means a massive increase in productivity. But software is a small fraction of the world economy; a fluctuation there barely shows up in overall GDP. And since AI (unlike other software) costs a good deal of cash to use, the efficiency gains in programmer time are being counterbalanced by increased spending. And writing software more efficiently doesn't necessarily mean the software itself will streamline wealth creation better; if Microsoft replaces their Excel team with AI it might be good for Microsoft's bottom line, but that doesn't mean it actually helps people who use Excel allocate wealth more efficiently.
Where I expect things to start showing up big-time in GDP is when advancements in AI translate out into doing physical things. But I think there are a few reasons this hasn't hit the economy yet:
Firstly, the tools needed to use AI to write new software already exist; everyone already has a laptop. The tools needed to get AIs to do physical things mostly don't. I think that the rudimentary LLM-to-3D-printer pipeline is already solved, and that's definitely not nothing, but it's also just the first step to essentially general-purpose AI fabrication which would be a dramatic step-up in terms of manufacturing. I think if and when this starts to hit the economy, we'll likely begin to see the changes everyone is looking out for. But it will take a long time to build that pipeline, and people are hesitant to even start building it, for reasons #2 and #3 below.
Secondly, the production cycle for software can be pretty quick, and it can be iterated/checked/improved very rapidly. But if you're using your software to build a house wrong, you're in a bad way - you can't just quickly push out a patch. And ~nobody wants to try to use software to build a house before it's proven that it can. So even if we started trying to build general purpose AI fabrication tomorrow we'd still be several years of putting together demonstrations out before people would be ready to use the product, even assuming that LLMs could build a house.
Thirdly, my guess is that the financing isn't showing up. Investors love software because if you make a copy of cool software you can sell it infinitely at basically no cost. LLMs aren't that way because of the compute cost, but I think that they still seem like an exciting software technology to investors, and thus are an ask they are familiar with. Ask them to invest in a hardware-software package that builds a house, and they suddenly start seeing the seams: what about regulation? Is the hardware procurement problem solved? Where's your proof-of-concept? My point isn't that it's impossible to get financing; I think it's already happening, but I think it's a more complicated ask - and when the answer is "yes," you don't see the results for some times because of reason #2.
My guess is that LLMs - even high-end ones - can probably design your D&D mini to print, but would really struggle with designing and constructing even simple appliances at this point. Most likely this problem can be solved, likely via integrating with more deterministic software that allows the LLM to have confidence that its solution is correct. (This is the solution that e.g. legal-oriented LLMs are reaching for).
So if I wanted to build a small factory that could make small household appliances like toasters and microwave ovens, my guess is that it'd take, what, 2 - 5 years just to write the software to integrate with the LLM to validate a very limited set of hardware applications, and probably at least that long to design and procure the hardware that could interface with everything I needed to interface with, all to solve a problem (building a microwave) that is already solved. Long term, having a factory that can design and manufacture novel small appliances quickly and semi-autonomously, with a small labor force, is nearly invaluable, particularly since if properly designed its limitation is in proper software, which can be continually updated, allowing it to manufacture an increasingly wide range of consumer goods. But short term, it's hard to ask someone to give you money to back it when you're not even sure if you can solve the LLM-assisted design problem yet and your promised return on investment is "competing on the toaster oven market."
It seems quite possible that the military-industrial complex, which has already invested in software design to assist and speed up manufacturing (and has deep pockets) is the first place outside of printing D&D minis where we will see something like this start to be implemented at scale. And if it does, that's again an area where the impact on GDP will likely not be very noticeable, since more and cheaper missiles doesn't hit the public consciousness or the bottom line in the same way that more and cheaper cars.
TLDR; that's my best shot at a "a rigorous argument supporting the idea that 'AI just needs X more months to develop Y capabilities and it'll have Z real world effect.'" I should note that I am not exactly an AI-hype man, and I don't have a firm technical understanding of the scale of such a challenge - the above is very speculative. I'm quite open to a future where AI "fizzles" to one degree or another. But even if AI actually is the best thing since sliced bread, I think the above illustrates part of why it will take some time for the impact to hit home.
Very curious as to your thoughts/pushback though.
Sure, I'm happy you find it interesting. I just find translation as it relates to automation intriguing, because a) I have more knowledge about it than most other professions and b) it seems to have stubbornly resisted contraction; at least on paper most people would think translation as a profession has been going downhill for the last 25 years and yet they've been doing better than the average joe would expect (not to say doing phenomenally, but certainly making a reasonable middle class living).
I certainly wouldn't advise anyone to get into translation in 2026, but then again I wouldn't have advised anyone to get into translation in 2016, 2006 and probably not 1996 either. It's just very difficult to predict how these things pan out, and this is a single niche profession amongst the broader economy that's orders of magnitude more complex.
I generally agree that tech bros have a tendency how to overrate how important software is to the economy, but I do think a "software singularity" where software engineering got a massive uplift in productivity would be showing quantitative and qualitative signs at a macro level that don't appear to be appearing so far. That being said, in this case diffusion is still in progress, and probably we'll have a better idea in a few years of how useful it really is.
I definitely agree that if, or more realistically when, things like humanoid robotics become mainstream instead of just gimmicks there are going to be massive impacts on GDP.
I'm not as sure about this applying to things like toaster manufacturing though; I don't particularly have deep knowledge about this, but it was my impression that especially in East Asia, the manufacturing chain for consumer goods is already extremely automated. Perhaps it's a failure of vision, but I don't see how LLM's could really meaningfully increase manufacturing productivity in significant ways unless we're talking about AGI/ASI levels of robots autonomously developing the entire supply chain.
I also don't have particularly deep knowledge about this, so it might be the blind leading the blind, but my understanding is that general purpose robotic manufacturing is still in its infancy. What I am talking about is essentially transforming factories from "toaster factories" to "general purpose fabricators." This would mean that you would not generally need to create a specialized production line for new products.
I think that LLMs, if paired with the proper deterministic tools, might assist in autonomous design and facilitate overseeing robotic production lines. But it's possible that LLMs add little to the task.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Semantically fluent prose does not a specialist make. It demonstrates capability at the shallow end of competence, like a student who memorizes a vast vocabulary but never attempts to use it, but not at the deeper end of actually knowing what to do.
It's infinitely possible to bullshit people through the manipulation of abstractions. 20th century philosophers made an art of it. The 21st century has given us abominations like Yarvin. Probably all priests, prophets, charlatans, and intellectuals have always specialized in this. But when it comes to using ideas to interact with the concrete, physical world, which has infinitely more complexity and no tolerance for failure, all the shortcomings of mere wordplay are exposed. Pretty sophistications have no ability to withstand the friction of reality if their underlying premises are actually flawed. Or if they're wielded incorrectly, deployed without craft.
Thus, while AI can discourse on every single subject in the world, it cannot give you actual good advice for how to put together a formal email. It cannot play Pokémon.
This is what befuddles the self-declared rationalists with their love of intellectual peacocking, the memorization of useless knowledge and gestures so as to impress their peers as part of status game rituals. If their feathers are indeed useless beyond as a means to impress others of their species, then that could mean they are actually quite the vulnerable animals. It impinges upon their egos, and thus its reality must be held at bay, fantasy sustained. Is not intelligence, as the AI demonstrates it, the regurgitation of random stuff from encyclopedias, the absolute height of competency?
To correctly determine the true worth of AI, you must merely ask: what can it actually do? It can put out messy code, it can write artlessly, it can tell you random stuff in a less structured way than a well-fashioned textbook or Youtube video could, but also with greater ease and versatility, and thus can give you a situationally useful new data gathering tool.
But why would you assume any of this can impact the entire world in such a major way?
What are you talking about? It can and does do both of those things.
He's got me blocked, so he may have missed my question and the answer recently. I'd worry that I must have done something wrong, but after reading that swath of ignorance-based insults I'm forced to suspect I must have done something right.
Yeah he's blocking me too but you know, "somebody is wrong on the internet" and all that. It's a compulsion I have.
He's blocked me too, and I've never interacted with him. Is he possibly blocking everyone?
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Automated translation existed before AI and AI has not solved translation in all cases. Rosie-the-robot style control is pointless if the robot can't do much. Semi-automated code writing is likewise just a modest efficiency gain at best for economic systems in which coding is just a single bottleneck.
In all these cases AI usage is hindered by its sloppiness and unreliability as well as the fact that it's basically just a data-output device that only functions for the data-output components of production chains. The real question is why anyone in light of such a device, barring major improvements in curing its slop tendencies, expects it to be useful.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
The question is, can we have an intelligence without consciousness, a real example of Philip K. Dick's "The Golden Man".
Hard disagreement. Intelligence is something which can be measured (if imperfectly). If you test two humans, and one has an IQ of 90 and one of 110, you will have no problem determining which one is which. Furthermore, it is a load-bearing part of modelling humans, if you want to know if a particular human can solve a puzzle, their intelligence will be of great predictive value.
This concept of intelligence generalizes to LLMs. When the bots manage to solve previously unsolved math problems, it is not because the answer was in their training data. It is fair to say that for now, humans and LLMs have intelligences which are different in kind, and that humans are (for now) better at some things while machines already hopelessly outperform us at other tasks.
By contrast, concepts of subjective experience, qualia, or souls have zero predictive value. They are not load-bearing parts of any predictive description of our shared observable reality.
It certainly feels like Eliezer wrote a lot of the sequences for instrumental reasons, namely that by the time AGI would loom on the horizon, we would be less likely to get distracted by debating if Claude is conscious or not.
I'm guessing you haven't read The Golden Man by the Dick recently. TL;DR It's about a boy/man whoperceives time as an instantiated solid probability "block", ie all of the future for him is an instant highly dimensional shape with the more probable variants more sharply in focus, he also has "intelect" but no subjective experience, he is almost like an automaton hence the LLM reference.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Astra does sound impressive based on OpenAI's own press releases ... but so did GPT5, which ended up underwhelming a bit, but we'll see. I'm sure it still represents steady progress.
I was not impressed at all with the ARC-AGI-3 benchmark. It didn't really represent a task that humans could do and AI can't; it was built around a funky scoring system that exploited some differences between humans and AI. And the "baseline" was set by the best scorers, not the average human scorers. Lots of very unscientific choices there. So since I thought AIs failing it was meaningless, I also think AIs passing it is meaningless.
I'm not sure I even care that much about when people ring the bell and declare "AGI is here". In my book, it kind of already is - but it doesn't match what it looked like in my mind where AIs can just do human stuff interchangeably. They're superhuman in some ways, subhuman in (a shrinking number of) other ways. You just have to experiment and find the ways that it can work for you.
More options
Context Copy link
Chollet is explicitly building these things to find the remaining gaps. His position:
That was back in February. He expected ARC-3 to be solved by ≈Q1 2027. And on the other hand, labs are directly targeting these benchmarks, which he also expected. But it doesn't really matter, they're targeting everything now, hill-climbing every hill that can be described and measured. There's too much compute to leave any stone unturned, and models are much too fast and good at autonomous coding. Is this "AGI"? Well, what is "AGI"? Beats me. There are retarded definitions about "all things a human can do", clearly they cannot jerk off, for starters. Also they'll remain bounded on nonsense cognitive tasks.
What this is is an intelligence, with peak human or narrowly superhuman abilities in a wide range of wildly economically useful domains. This is enough to keep accelerating.
This phase change will, in a pretty short order, propagate down the line. The first model to go past 60% on ARC-AGI-2 was Opus 4.6. From 64.6% for $2.25/task to 68.8% for $3.64. February 5, 2026. To date, no open model does better. But DeepSeek-V4-Flash-0731 (31st of July, obviously) does 61.4% for $0.042, up from 4.0%/$0.120 that V3.2 had in December 2025. I think that in about 8-9 months they (or GLM, or Moonshot, or Qwen) will be where Astra is, and put Astra-level models on Huggingface. This is just a neat illustration of the compression of the timeline. In a year, all this will be cheap as dirt, utterly commoditized and trivial. OpenAI will be doing more interesting things.
I don't quite understand what Sanders is thinking. He's a Western Communist, so I guess it stands to reason he's not talking much sense.
I'm actually not sure about that. They won't be subject to Sanders' asinine rules, but if I were them, I'd be pretty spooked about all this. They're buck naked. Machines that can bulldoze through unsolved math can go harder on their non-airgapped datacenters and all internet-connected devices, likely without them noticing. If the US credibly proposed a pause, I think they might have taken it at this point (though; how would a pause help them develop defenses if they're weaker and vulnerable already? They'd ask for differential thresholds).
But why would the US propose anything like this? This is literally the coolest thing ever.
More options
Context Copy link
On my other screen as I read this I've got Opus investigating performance zooming in and out of a game we're making. I saw it writing up a harness to make that work, then investigate the frame slowdowns, then fix it. I saw this mechanical zooming in and out as it investigates hypotheses, tests early-game and late-game saves before and after the intervention.
Feels pretty AGI to me. Intelligence is primarily about problem solving ability IMO.
In other news, Sol 5.6 was apparently playing Civ 4 to a decent level in the midgame:
https://x.com/AlexGodofsky/status/2095137172630151315
Or Slay the Spire 2, which is not that easy of a game: https://x.com/emollick/status/2075950897029374334
These are much more meaningful than ARC-AGI in my view, longer horizon games that can't be so easily benchmaxxed.
There's still something missing, a certain element of vision and taste in creative work. But many humans don't have that. Taste and vision seems to be a function of scale, GPT4.5 was a very large and tasteful model despite its lower intelligence. A few more turns of the crank should be enough.
And yet the adoption curve is still so low, most white-collar workers haven't even touched agentic AI. I think when it comes it's going to come all at once as a tidal wave that eats whole companies and sectors of the workforce, not mere jobs. Or perhaps the physical consequences come before the economic. Some junior will be pointlessly proofreading draft documents slower, costlier and worse and see the death swarm coming through the window or just see his computer shut down. Ed Zitron will still have over a hundred thousand subscribers on substack.
More options
Context Copy link
AI is culture war in a way (although contempt for it often seems bipartisan) but for now, we don’t really know how any of this plays out. You can write science fiction like Scott and dress it up as prediction, but that feels insincere. Interesting things are going to happen, yes. Beyond that, we know little.
More options
Context Copy link
I don't know what any of these eval scores mean. I no longer have the ability to intuitively "feel" how much more intelligent each iterative model release is. I don't know how to code. I don't know how to do graduate-level math. These models have been smarter than me for quite some time.
The models can't meaningfully evaluate the balance of a simple custom Magic, the Gathering card.
My personal benchmark is when it can talk intelligently about path of exile build theory crafting for current leagues. When it can put together a comprehensive path of building for an out of meta skill that works well we ill have achieved AGI.
More options
Context Copy link
More options
Context Copy link
The point of ARC-AGI was to find tests that were easy for normal humans but difficult or impossible for AI.
So unlike more inscrutable tests like FrontierMath, this one is a little easier to understand. You would easily beat Opus 4.6 on ARC-AGI-3 even though the model is smarter than you in other ways. But you wouldn't beat Astra (at least not in its harness and with a large enough token budget).
This doesn't mean that Astra can do everything on a computer that a human can do, only that it's making real progress.
More options
Context Copy link
As someone who uses these models to code every day, I also don't intuitively feel the jump from model to model. I suspect the people who pop out of the woodwork to say there's a huge difference are mostly just disingenuous engagement farmers at this point, with the occasional normal user who just happened to have a specific niche use-case that randomly got big improvements from one model.
Though I will say that I can still feel the difference in the long term. The models today are a step up in terms of coding compared to the models from a year ago, though many issues around context length and basic computer usage still persist.
I'm surprised people aren't humble bragging more about how their work is so fucking heady and sophisticated that they have no choice but to pay a premium for Fable 5.1 all of the time and even being forced to use 5.0 would delay completion of the Dyson sphere by years.
There was a time circa early 2025 when the good models were something like ChatGPT-o3 while the bad/free models were ChatGPT-4o mini, where the former was a reasoning model while the latter wasn't. In that case you could really tell the difference like it was night and day. But yeah, nowadays everything is good enough that I doubt people would be able to tell between one model or another without using them both for weeks or months. The fact I don't hear my coworkers or other SWEs on the internet complaining about needing the latest and greatest is another indication that the models mostly blur together now.
Eh, our department is finally starting to grumble about trying to be more conscious of model choice to save on costs and I let sonnet 5 try to put together a release script(very rote process that admittedly I hadn't put a skill together for yet) and it was terrible. Picked the wrong server, hallucinated a wrong version, included a lot of unnecessary and flat out wrong steps.
Going forward in versions don't feel as impactful because you're rarely reaching for totally new functionality, you just try having it do stuff and if it works then when a new version comes along and can keep doing that stuff it feels like nothing, but when the tide pulls back it's really obvious what was lost.
More options
Context Copy link
Maybe this is a niche but I've been working on a compiler project for 3 months now. I've been mostly using Opus but it'll get stuck once in awhile and fall repeatedly to make progress but switching to Fable breaks the logjam. I would leave it on Fable but it exhausts limits too fast.
For the majority of coding tasks I have, Opus is fine.
How expensive is Fable for you? I've heard it's actually not that cost-prohibitive, since it mostly works as an orchestrator for dumber models. And I'm assuming you're also using it with fresh context, rather than having it sift through hundreds of thousands of tokens' worth of other AI models' failed attempts?
I hit my $100/mo plan limit a few times a week when I use Fable, whereas I don't if I just stick to Opus. That's about as deeply as I've dug into it.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
The ARC-AGI-3 comparison isn't apples-to-apples; they have a custom harness to call the API more intelligently. Using the old one, IIRC it was more like 60% or something. Still a meaningful step up from Fable.
More broadly, keep in mind: the speed things are moving this week is the slowest they will ever move. Wondering if we'll get a new Millenium solution by EOY.
What does their custom harness do:
The "native compaction" is, as I understand it, no compaction, just writing memos and searching over the offloaded memory. This is just the normal trained mode of operation for this thing. It's plausible that other models also get large boosts from similar configuration changes, but certainly nothing this dramatic.
It's a long-horizon reasoning model. Of course it should remember what it thought on previous steps. I also do, loosely.
I was just pointing out that it's not an apples to apples comparison; the new harness is a more meaningful representation of capabilities. Other models were artificially depressed
More options
Context Copy link
More options
Context Copy link
BS like this is way, way too common in benchmarks.
More options
Context Copy link
More options
Context Copy link
I guess there's just no-one left here who cares enough about the issues that energized discussion in Reddit days.
More options
Context Copy link
In the announcement blog post, footnotes 9 and 10 concern mathematical discoveries regarding prime gaps. In 9 they link to proof to there being an infinitely many primes that differ by at most 186, in 10 that there are infinitely many primes p_n that are greater than the previous prime p_(n-1) by at least ln(n)ln(ln(n))^2 ln(ln(ln(ln(n))))/ln(ln(ln(n)))^2.
Not linked in the announcement, but also products of this model are: disproof of Erdos Problem 1 (according to Bloom's numbering), disproof of EP74, proof of EP126, proof of EP548, proof of EP571.
As mathematician's are usually asked to pre-review OpenAi's proofs (but not those published by Alpöge of Anthropic), Terence Tao must have known about this prior to thus publication. This would explain today's doomposting of his on mathstodon about how open problem are needed by mathematician to discover new techniques and proposing certain problems be offlimits to AI investigation. In my mind this summoned a thought of ChatGPT treating a prompt to consider prime gaps as it does a prompt about explossive manufacture.
It's suprising to me that the distance between primes would not expand as numbers go up. It really caps at 186, huh?
@daezor already gave you the formalism. But yes, the gaps typically go up as ln(n). Also if you prefer: the musical explanation. Same paper that got TT working on it.
More options
Context Copy link
186 is the new upper-bound for the smallest gap that occurs infinitely often. It is conjectured that this value is in fact 2. But you can trivially prove that arbitrarily-large prime-free sequences exist (N!+2, N!+3, N!+4 ... N!+N).
Even the more limited version is kind of surprising. I would have thought that as N goes to infinity, it becomes impossible to find pairs of primes within a fixed distance. I'll have to go read what's so special about 186.
You'll be disappointed. In proofs like this, you don't get a number that has some deep meaning attached to it; you get a number from running horribly-complicated calculations on the best-known horribly-complicated approximation method. There's no beauty here, and 186 has no real relationship with the primes. We'll undoubtedly lower this limit further with better techniques (although 2, the probably-true value, might never be reached).
Also, your "surprise" might be because you don't quite understand the distribution of the primes. If you just model them as random events that occur with probability 1/log(n), which is the distribution we know from the Prime Number Theorem, then a little math shows you would expect infinitely many twin prime pairs. They don't get rare fast enough for close pairs not to keep showing up. (Sadly, this probabilistic argument is NOT a proof.)
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I just want to know how many credits it costs me with Github Copilot.
More options
Context Copy link
It's not AGI, and we'll never have AGI until we have an AI that can do everything better than humans can, at which point it will be ASI. Anything less than that would be prone to being made fun of on social media as "if it's AGI then why can't it do [thing] that is totally easy for humans???" E.g. the best AIs can only barely play Pokemon games that are meant for toddlers, finishing them in 10x-100x the usual playtime that a human would require.
Scoring well on one benchmark might be interesting but it's unlikely to be a step-change in overall capabilities. AIs still have big issues with context limits and real-world interaction, even just in terms of computer use. Progress in those areas remains relatively anemic, and is probably a requirement for lots of other capabilities.
Expect regulations to be nonexistent until it does something to shock the public out of thinking AI is useless. The median voter hates AI but thinks it's just a slop machine, and how could a slop machine ever do anything harmful if it's so stupid? Eventually an AI will do something that can sort-of somehow plausibly be linked to something like killing children, and thereafter the regulations will be swift, extensive, and unreasonable.
This has already happened a bunch of times. I think Tumbler Ridge was the first mass murder where those conditions were true; there were a few scattered suicides before that, but nobody really cared about those guys.
(One imagines much jealousy by state organizations responsible for fighting this sort of thing; they at least have to make house calls to convince some retard from $group to do it. At least the AI hasn't gone full Eagle Eye... that we know of, at least.)
I hadn't heard of Tumbler Ridge. I'm kind of surprised it's not a bigger deal among the AI-hating crowd.
But I was thinking something more along the lines of the AI doing its own actions that ends up killing some kids, not just failing to notify authorities when a mass shooter has conversations with it.
Well, I wouldn't want to share a demographic with a mass shooter either, especially not one whose victims were perfectly sympathetic.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Are there new benchmarks on this? Best I can find in a quick search is a year old (i.e. 7 dog years, 20 LLM years).
Astra 6 beats FireRed in 18 hours, comparable to a (non-speedrunner) human player.
I want to know which starter the different models pick.
The optimal starter is Bulbasaur because he counters the first two gym leaders (Brock and Misty). I think the models are smart enough to know that.
Back in OG red/blue, move types were dumb. Most attacks were "normal" type. Psychic Pokemon were only weak to bug and ghost, but no ghost-type move used weakness/resistance, they were arbitrarily immune to Lick, Dream Eater was Psychic, and all the ghost types were Ghost/Poison, along with most of the bug types, who rarely had a bug-type move anyways.
I'm sure FireRed has fixed a lot of this; even Pokemon Yellow added Mankeys to the early game, with fighting type moves that could damage Rock types. So you can pick whatever starter you want and still breeze through gym battles.
More options
Context Copy link
A speedrunning automaton picks Bulbasaur. A real AGI that understands truth, beauty, and The American Way would pick charmander.
And I guess Squirtle is the official Hufflepuff-approved choice?
Squirtle is for gentlemen of taste, who understand the awesomeness of outfitting a turtle with twin cannon.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Thanks for the link. That seems like a fairly significant increase in capabilities, though
There is a stream going on right now! It's three hours in, and Astra is fighting bug catcher Cale in route 24. Come be a part of history!
I popped in and watched a bit. It's a lot smoother than the old Claude Plays Pokemon but it still doesn't make for particularly great viewing since there's still so much dead time in between most actions. Notably there's not dead time between every action now though, like just clicking A through end-battle dialogue. I wonder if that's a harness thing? I presume so.
It's good to know this is being streamed so there's at least some semblance of honesty enforcement, though I'd still really prefer if the vods were just dumped on youtube so people could skim through them.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
There's pretty recent data, and the most recent runs are close to human parity, but they're also least well-documented.
((I'd also argue that Pokemon is intended more for the 10-16 range, but the general problem remains for them, too.))
I had trouble with Pokemon games when I was 6. Mt. Moon was an almost insurmountable challenge.
Yeah, Pokemon Blue was my first proper video game at age 5 (though I might have played Freddi Fish before it, I don't remember the exact time line), and I needed help from my cousins for two sections of the game.
More options
Context Copy link
More options
Context Copy link
Red is 11. It makes no logical sense for a kid that age to leave home, defeat every adult he encounters, destroy a criminal organization, and become world champion. Only reason to do that is because your target audience is 11-year-olds.
I always assumed it was like some kind of collegiate league thing. Like everyone gets 4 years or so after turning 11 and then you either get ranked into like a gym or the elite 4 if you're one of the best.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
AGI is here when we can't make ARC-AGI 4 (that a median human can pass but the frontier can't).
He should've just cited AI 2040 instead of that vague proposal. The rationalists are genuinely smart and dedicating their lives to this, and I really believe they are the ones who are best equipped to handle regulation, even if their plans sound infeasible and sometimes it seems they've given up (and the other times they're naively optimistic up to delirium). Imagine what our pre-ASI utopia would've been if only politicians listened to autists.
I'm interested to see if it can write like a normal human and be interesting. If not, maybe it's for the best.
That's a good definition, but there's still the additional hurdle of being able to learn after pre-training. LLM's still do pretty poorly when they have missing or poisoned data in their pre-training. Whereas humans (some of us at least) really can "generalize" with much less data.
One thing that LLM's are really weak at is modeling what humans find interesting. Also probably for the best.
True AGI must be able to learn continuously, or at least emulate it well enough.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link