This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Dead forum?
Today in AI news, OpenAI has reportedly released GPT-6 Astra. Rumors on X suggests that it scores 99% on Arc-AGI-3, meaning that the benchmark is essentially dead now. For comparison, Fable only scored 30%. Does this mean AGI is here? Many are saying that it is. Color me skeptical, but its getting harder and harder to find concrete tests that AI cannot pass. The god of the gaps gets smaller.
Many intelligenct people have important things to say about this. But what do the 84 year old communists think? Bernie Sanders today has introduced legislation calling for a ban on superintelligence with a 20 year prison term for attempted superintelligence. He also wants (shocked Pikachu) a government regulatory agency. Although he mumbles something about international cooperation, its clear that China will not be subject to these rules so it solves nothing.
I will give him credit for at least recognizing the importance of the issue.
We could be entering a world of unimaginable change. And, given the political systems in place in the US and China, the midwives of that changes will be those on the cusp of senility.
With all the renewed hype around Astra and AI mathematics, I have been feeling that I haven't really seen a response I'm satisfied with from the AI bulls (at least the imminent mass unemployment / AGI / singularity types) to the 80IQ skeptic question of "If AI is so good, why is nothing happening?".
FWIW, if I knew in 2020 that in a few years we would have AI models that can get a perfect score on the IMO for pennies and meaningfully contribute to research mathematics I have expected that we basically live in a sci-fi world as well; yet now we have these models for $20 a month, and literally nothing I'm aware of has happened on a macro level clearly attributable to AI, either quantitatively or qualitatively, apart from the distortion of the AI capex buildout. In fact, it's actually GLP-1's that are basically a complete miracle on a macro-economic and public health level, and where there actually are many macro statistics that are visibly affected by GLP-1's.
On some level there probably is anthropomorphization going on here where the natural human assumptions are that
when in fact the conclusion does not actually follow from the premises; it is possible that AI is extraordinarily economically valuable and powerful for other reasons, but this does not follow from being superhuman at mathematics.
Modern pure mathematics is, as we seem to be finding out, not particularly economically valuable in and of itself; mostly mathematicians are lionized for their intellect because a human with that level of intelligence could obviously do significant amounts of other economically valuable work, but this seems extremely non-obvious in the case of AI.
The "best" response I've seen is something along the lines of "actually AI is still heavily limited by spikiness right now but the exponential is going to keep going bro" which I suppose is honest as to current capabilities and not disprovable, but it does feel like a retreat away from "AI is already crazy good" as many people have been and are touting, and towards magical thinking; somewhere along the line the party line swapped from "AGI imminent" to "AI that creates AGI imminent".
My take-away certainly isn't that AI is a fugazi; obviously it's powerful technology and I'm still quite bullish in the mid-long term, but I've certainly been feeling myself getting anti-hyped with recent releases as a result of this growing discontinuity of any real-world effects.
I’d caveat that there have been some massive changes: translation is basically solved, ‘Rosie-the-robot’ style control is no longer science fiction but a creepy and dumb product, ai-assisted chip development and software engineering is a default option. Recursive self-improvement has been slow, rather than non-existent.
Some of the delay is just logistics. There’s probably over a hundred people worldwide following the AI cancer vaccine dog’s approach, but it’s going to be three or four years before we have clear and convincing academic papers even assuming it works and generalizes, just because that’s how medical trials work. AI driven assistance and funding for nuclear reactors might make building them in the US possible, but it isn’t going to make the NRC respond in less than six months. Silicon fab cycles are measured in years. Structural stuff just can’t be that fast.
Others are tooling. There’s a massive space where many small and medium businesses would benefit from custom code, for example, and AI can do it… but if your interface and sanity checks require a programmer or project manager anyway, it’s not really available yet even if the AI can do it. And Claude Code is not ready for prime time use by normies.
A lot of stuff is also just obscured when it does work. An AI-derived optimization algorithm for matrix multiplication just gets swallowed as a delta in efficiency from expected performance. An AI-built software tool just looks like a (verbose) software tool.
I personally think the effects of current AI on software engineering productivity are somewhat overstated (at my fairly AI-pilled software firm there's been many "AI-assisted" design documents and RFC's of dubious value, lots of rewrites of internal platforms, test suites and frameworks that look productive but haven't really been that useful, and only somewhat modest improvement in terms of actual firm-wide user-legible value), but as a whole I would agree that software engineering productivity has been meaningfully enhanced via AI.
There doesn't yet seem to be meaningful impact on software engineering employment, or anything noticeable on a macro scale as a result of modern LLM coding, but I think I would agree it is too early to determine the wider effects of coding agents one way or another.
I think translation is an great example of what I mean by "if AI is so good, why is nothing happening"?
We've basically had "solved translation for dummies" since 2016 Google Translate NMT, in the sense that the output wasn't great, but someone with no knowledge could bumble through understanding the gist of a brochure or a manual, where previously it might have been literally impossible for them to understand the text without consulting another human. I remember quite a few people in language learning communities I was part of at the time were already getting blackpilled on the full automation of translation as an inevitability all the way back in 2016.
LLM's took that up one step further, since about early 2023 we've had "solved translation for normies" in the sense that LLM's could reliably provide semantically correct translations for pretty much any input/output language pair. Maybe it makes some minor mistakes or it sounds weird to the critical reader, but nothing that really matters in the vast majority of use-cases for the vast majority of people.
Most if not all my friends in 2016 would have agreed that the existence of these models would be the end of human translation, and yet in nearly four years there still has been no significant macro impact on translator employment, or really anything noticeable on a macro scale that might be noticeable if translation was truly "solved".
So then, why is this case? This topic came up when I was chatting with some translators the other day, but I wasn't really satisfied by the points that ended up being brought up.
Is AI translation too expensive? Perhaps plausible for the earliest LLM's, but certainly not any remotely recent model, at least when compared to the fully loaded salary of a first world translator.
Is AI translation not reliable enough? I thought this might be the case, but I was assured that this isn't really true for most translation jobs. It certainly matters to have a human liability shield for regulated medical, diplomatic, legal use cases etc, but most translators don't work in those niches, and for most translation work accuracy is not really that paramount - most people are happy with "good enough" and people who care about translation quality have always been a small minority.
Is diffusion simply incomplete and all the translators will be unemployed in X more months or years? Plausible, but I'm not convinced this can be the entire story either; at this point more than half the planet has tried ChatGPT or Gemini, and AI is top of mind in nearly all corporate boardrooms right now. Even if diffusion is slower than expected, I would think it reasonable that after nearly four years there should be some sort of real world impact.
Are there induced demand effects outweighing the supply increase? Certainly demand for Japanese<->English, Chinese<->English and Chinese<->Japanese translation (and likely lots of other language pairs) has exploded in the last 10-20 years; the expectation now is for games, media, writing to be rapidly localized across many different languages when you used to have to wait years for any kind of translation if at all.
This can't be the entire story though; if translation really was solved, it wouldn't matter how high demand was when AI would be meeting the vast majority of it. Perhaps it's possible that most translation is solved, but induced demand for the translation that is not solved means that most translators still have jobs?
I agree that logistics and tooling are significant bottlenecks for anything that needs to interface with the physical world, but I think it's a logical criticism to make that even solely in the worlds of bits and words there's really been very little happening in the world relative to the capabilities of LLM's.
In general I don't think I've seen a rigorous argument supporting the idea that "AI just needs X more months to develop Y capabilities and it'll have Z real world effect", when it seems that nobody really understands what capabilities are missing from the current models that are currently preventing them from having major real-world impact in the first place.
Your anecdotes about translation are very interesting, I hope you keep updating us.
I have a theory on why AI wouldn't show up much in the economy even if it was very good at what it does. I do use AI, I think it's great for websearch and stuff (in no small part because Google is now terrible at websearch). I am not a programmer so I don't have strong opinions on how much efficiency it's actually adding there, but while I am open to criticisms of AI as a technology, my theory is agnostic on that and I think still makes sense even if AI is in fact pretty good as a technology, although I wouldn't necessarily claim it's monocausal.
Simply put, I don't actually think much wealth is generated through software directly.
Software is extremely good at streamlining wealth creation; that's why the world runs on Excel. But at the end of the day your accounting guy is making sure that you are efficiently allocating resources, not directly generating resources. (Software also makes it easier to generate inefficiency, too, but let's ignore that for the time being.)
This can have a huge impact on the economy if you're jumping from "I need to hire a scribe to write a letter to travel through bandit-infested territory to inform my business partner about recent developments" to "lemme sit down and write an email real quick" but in a world where you are going from writing a quick email to having AI write a quick email for you, you're not really streamlining things much more.
I think people who work in software sometimes forget that software is just a tiny part of the economy. So from their perspective, if they increase efficiency by 20%, that means a massive increase in productivity. But software is a small fraction of the world economy; a fluctuation there barely shows up in overall GDP. And since AI (unlike other software) costs a good deal of cash to use, the efficiency gains in programmer time are being counterbalanced by increased spending. And writing software more efficiently doesn't necessarily mean the software itself will streamline wealth creation better; if Microsoft replaces their Excel team with AI it might be good for Microsoft's bottom line, but that doesn't mean it actually helps people who use Excel allocate wealth more efficiently.
Where I expect things to start showing up big-time in GDP is when advancements in AI translate out into doing physical things. But I think there are a few reasons this hasn't hit the economy yet:
Firstly, the tools needed to use AI to write new software already exist; everyone already has a laptop. The tools needed to get AIs to do physical things mostly don't. I think that the rudimentary LLM-to-3D-printer pipeline is already solved, and that's definitely not nothing, but it's also just the first step to essentially general-purpose AI fabrication which would be a dramatic step-up in terms of manufacturing. I think if and when this starts to hit the economy, we'll likely begin to see the changes everyone is looking out for. But it will take a long time to build that pipeline, and people are hesitant to even start building it, for reasons #2 and #3 below.
Secondly, the production cycle for software can be pretty quick, and it can be iterated/checked/improved very rapidly. But if you're using your software to build a house wrong, you're in a bad way - you can't just quickly push out a patch. And ~nobody wants to try to use software to build a house before it's proven that it can. So even if we started trying to build general purpose AI fabrication tomorrow we'd still be several years of putting together demonstrations out before people would be ready to use the product, even assuming that LLMs could build a house.
Thirdly, my guess is that the financing isn't showing up. Investors love software because if you make a copy of cool software you can sell it infinitely at basically no cost. LLMs aren't that way because of the compute cost, but I think that they still seem like an exciting software technology to investors, and thus are an ask they are familiar with. Ask them to invest in a hardware-software package that builds a house, and they suddenly start seeing the seams: what about regulation? Is the hardware procurement problem solved? Where's your proof-of-concept? My point isn't that it's impossible to get financing; I think it's already happening, but I think it's a more complicated ask - and when the answer is "yes," you don't see the results for some times because of reason #2.
My guess is that LLMs - even high-end ones - can probably design your D&D mini to print, but would really struggle with designing and constructing even simple appliances at this point. Most likely this problem can be solved, likely via integrating with more deterministic software that allows the LLM to have confidence that its solution is correct. (This is the solution that e.g. legal-oriented LLMs are reaching for).
So if I wanted to build a small factory that could make small household appliances like toasters and microwave ovens, my guess is that it'd take, what, 2 - 5 years just to write the software to integrate with the LLM to validate a very limited set of hardware applications, and probably at least that long to design and procure the hardware that could interface with everything I needed to interface with, all to solve a problem (building a microwave) that is already solved. Long term, having a factory that can design and manufacture novel small appliances quickly and semi-autonomously, with a small labor force, is nearly invaluable, particularly since if properly designed its limitation is in proper software, which can be continually updated, allowing it to manufacture an increasingly wide range of consumer goods. But short term, it's hard to ask someone to give you money to back it when you're not even sure if you can solve the LLM-assisted design problem yet and your promised return on investment is "competing on the toaster oven market."
It seems quite possible that the military-industrial complex, which has already invested in software design to assist and speed up manufacturing (and has deep pockets) is the first place outside of printing D&D minis where we will see something like this start to be implemented at scale. And if it does, that's again an area where the impact on GDP will likely not be very noticeable, since more and cheaper missiles doesn't hit the public consciousness or the bottom line in the same way that more and cheaper cars.
TLDR; that's my best shot at a "a rigorous argument supporting the idea that 'AI just needs X more months to develop Y capabilities and it'll have Z real world effect.'" I should note that I am not exactly an AI-hype man, and I don't have a firm technical understanding of the scale of such a challenge - the above is very speculative. I'm quite open to a future where AI "fizzles" to one degree or another. But even if AI actually is the best thing since sliced bread, I think the above illustrates part of why it will take some time for the impact to hit home.
Very curious as to your thoughts/pushback though.
Sure, I'm happy you find it interesting. I just find translation as it relates to automation intriguing, because a) I have more knowledge about it than most other professions and b) it seems to have stubbornly resisted contraction; at least on paper most people would think translation as a profession has been going downhill for the last 25 years and yet they've been doing better than the average joe would expect (not to say doing phenomenally, but certainly making a reasonable middle class living).
I certainly wouldn't advise anyone to get into translation in 2026, but then again I wouldn't have advised anyone to get into translation in 2016, 2006 and probably not 1996 either. It's just very difficult to predict how these things pan out, and this is a single niche profession amongst the broader economy that's orders of magnitude more complex.
I generally agree that tech bros have a tendency how to overrate how important software is to the economy, but I do think a "software singularity" where software engineering got a massive uplift in productivity would be showing quantitative and qualitative signs at a macro level that don't appear to be appearing so far. That being said, in this case diffusion is still in progress, and probably we'll have a better idea in a few years of how useful it really is.
I definitely agree that if, or more realistically when, things like humanoid robotics become mainstream instead of just gimmicks there are going to be massive impacts on GDP.
I'm not as sure about this applying to things like toaster manufacturing though; I don't particularly have deep knowledge about this, but it was my impression that especially in East Asia, the manufacturing chain for consumer goods is already extremely automated. Perhaps it's a failure of vision, but I don't see how LLM's could really meaningfully increase manufacturing productivity in significant ways unless we're talking about AGI/ASI levels of robots autonomously developing the entire supply chain.
I also don't have particularly deep knowledge about this, so it might be the blind leading the blind, but my understanding is that general purpose robotic manufacturing is still in its infancy. What I am talking about is essentially transforming factories from "toaster factories" to "general purpose fabricators." This would mean that you would not generally need to create a specialized production line for new products.
I think that LLMs, if paired with the proper deterministic tools, might assist in autonomous design and facilitate overseeing robotic production lines. But it's possible that LLMs add little to the task.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link