This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Dead forum?
Today in AI news, OpenAI has reportedly released GPT-6 Astra. Rumors on X suggests that it scores 99% on Arc-AGI-3, meaning that the benchmark is essentially dead now. For comparison, Fable only scored 30%. Does this mean AGI is here? Many are saying that it is. Color me skeptical, but its getting harder and harder to find concrete tests that AI cannot pass. The god of the gaps gets smaller.
Many intelligenct people have important things to say about this. But what do the 84 year old communists think? Bernie Sanders today has introduced legislation calling for a ban on superintelligence with a 20 year prison term for attempted superintelligence. He also wants (shocked Pikachu) a government regulatory agency. Although he mumbles something about international cooperation, its clear that China will not be subject to these rules so it solves nothing.
I will give him credit for at least recognizing the importance of the issue.
We could be entering a world of unimaginable change. And, given the political systems in place in the US and China, the midwives of that changes will be those on the cusp of senility.
With all the renewed hype around Astra and AI mathematics, I have been feeling that I haven't really seen a response I'm satisfied with from the AI bulls (at least the imminent mass unemployment / AGI / singularity types) to the 80IQ skeptic question of "If AI is so good, why is nothing happening?".
FWIW, if I knew in 2020 that in a few years we would have AI models that can get a perfect score on the IMO for pennies and meaningfully contribute to research mathematics I have expected that we basically live in a sci-fi world as well; yet now we have these models for $20 a month, and literally nothing I'm aware of has happened on a macro level clearly attributable to AI, either quantitatively or qualitatively, apart from the distortion of the AI capex buildout. In fact, it's actually GLP-1's that are basically a complete miracle on a macro-economic and public health level, and where there actually are many macro statistics that are visibly affected by GLP-1's.
On some level there probably is anthropomorphization going on here where the natural human assumptions are that
when in fact the conclusion does not actually follow from the premises; it is possible that AI is extraordinarily economically valuable and powerful for other reasons, but this does not follow from being superhuman at mathematics.
Modern pure mathematics is, as we seem to be finding out, not particularly economically valuable in and of itself; mostly mathematicians are lionized for their intellect because a human with that level of intelligence could obviously do significant amounts of other economically valuable work, but this seems extremely non-obvious in the case of AI.
The "best" response I've seen is something along the lines of "actually AI is still heavily limited by spikiness right now but the exponential is going to keep going bro" which I suppose is honest as to current capabilities and not disprovable, but it does feel like a retreat away from "AI is already crazy good" as many people have been and are touting, and towards magical thinking; somewhere along the line the party line swapped from "AGI imminent" to "AI that creates AGI imminent".
My take-away certainly isn't that AI is a fugazi; obviously it's powerful technology and I'm still quite bullish in the mid-long term, but I've certainly been feeling myself getting anti-hyped with recent releases as a result of this growing discontinuity of any real-world effects.
I’d caveat that there have been some massive changes: translation is basically solved, ‘Rosie-the-robot’ style control is no longer science fiction but a creepy and dumb product, ai-assisted chip development and software engineering is a default option. Recursive self-improvement has been slow, rather than non-existent.
Some of the delay is just logistics. There’s probably over a hundred people worldwide following the AI cancer vaccine dog’s approach, but it’s going to be three or four years before we have clear and convincing academic papers even assuming it works and generalizes, just because that’s how medical trials work. AI driven assistance and funding for nuclear reactors might make building them in the US possible, but it isn’t going to make the NRC respond in less than six months. Silicon fab cycles are measured in years. Structural stuff just can’t be that fast.
Others are tooling. There’s a massive space where many small and medium businesses would benefit from custom code, for example, and AI can do it… but if your interface and sanity checks require a programmer or project manager anyway, it’s not really available yet even if the AI can do it. And Claude Code is not ready for prime time use by normies.
A lot of stuff is also just obscured when it does work. An AI-derived optimization algorithm for matrix multiplication just gets swallowed as a delta in efficiency from expected performance. An AI-built software tool just looks like a (verbose) software tool.
I personally think the effects of current AI on software engineering productivity are somewhat overstated (at my fairly AI-pilled software firm there's been many "AI-assisted" design documents and RFC's of dubious value, lots of rewrites of internal platforms, test suites and frameworks that look productive but haven't really been that useful, and only somewhat modest improvement in terms of actual firm-wide user-legible value), but as a whole I would agree that software engineering productivity has been meaningfully enhanced via AI.
There doesn't yet seem to be meaningful impact on software engineering employment, or anything noticeable on a macro scale as a result of modern LLM coding, but I think I would agree it is too early to determine the wider effects of coding agents one way or another.
I think translation is an great example of what I mean by "if AI is so good, why is nothing happening"?
We've basically had "solved translation for dummies" since 2016 Google Translate NMT, in the sense that the output wasn't great, but someone with no knowledge could bumble through understanding the gist of a brochure or a manual, where previously it might have been literally impossible for them to understand the text without consulting another human. I remember quite a few people in language learning communities I was part of at the time were already getting blackpilled on the full automation of translation as an inevitability all the way back in 2016.
LLM's took that up one step further, since about early 2023 we've had "solved translation for normies" in the sense that LLM's could reliably provide semantically correct translations for pretty much any input/output language pair. Maybe it makes some minor mistakes or it sounds weird to the critical reader, but nothing that really matters in the vast majority of use-cases for the vast majority of people.
Most if not all my friends in 2016 would have agreed that the existence of these models would be the end of human translation, and yet in nearly four years there still has been no significant macro impact on translator employment, or really anything noticeable on a macro scale that might be noticeable if translation was truly "solved".
So then, why is this case? This topic came up when I was chatting with some translators the other day, but I wasn't really satisfied by the points that ended up being brought up.
Is AI translation too expensive? Perhaps plausible for the earliest LLM's, but certainly not any remotely recent model, at least when compared to the fully loaded salary of a first world translator.
Is AI translation not reliable enough? I thought this might be the case, but I was assured that this isn't really true for most translation jobs. It certainly matters to have a human liability shield for regulated medical, diplomatic, legal use cases etc, but most translators don't work in those niches, and for most translation work accuracy is not really that paramount - most people are happy with "good enough" and people who care about translation quality have always been a small minority.
Is diffusion simply incomplete and all the translators will be unemployed in X more months or years? Plausible, but I'm not convinced this can be the entire story either; at this point more than half the planet has tried ChatGPT or Gemini, and AI is top of mind in nearly all corporate boardrooms right now. Even if diffusion is slower than expected, I would think it reasonable that after nearly four years there should be some sort of real world impact.
Are there induced demand effects outweighing the supply increase? Certainly demand for Japanese<->English, Chinese<->English and Chinese<->Japanese translation (and likely lots of other language pairs) has exploded in the last 10-20 years; the expectation now is for games, media, writing to be rapidly localized across many different languages when you used to have to wait years for any kind of translation if at all.
This can't be the entire story though; if translation really was solved, it wouldn't matter how high demand was when AI would be meeting the vast majority of it. Perhaps it's possible that most translation is solved, but induced demand for the translation that is not solved means that most translators still have jobs?
I agree that logistics and tooling are significant bottlenecks for anything that needs to interface with the physical world, but I think it's a logical criticism to make that even solely in the worlds of bits and words there's really been very little happening in the world relative to the capabilities of LLM's.
In general I don't think I've seen a rigorous argument supporting the idea that "AI just needs X more months to develop Y capabilities and it'll have Z real world effect", when it seems that nobody really understands what capabilities are missing from the current models that are currently preventing them from having major real-world impact in the first place.
I'm not sure if I would even go that far. I work at a software company, and I do some consulting on the side. I've been able to watch a lot of AI-coding initiatives from the sidelines due to that consulting, and I see the same pattern a lot.
First, management picks something they can measure. Lines of code, commits, and PRs merged are the most common ones. This causes a certain kind of climber who doesn't actually want to be a programmer to ruthlessly optimize for whatever the metric is. Management lauds that person as a "10x/100x/1000x developer" and they receive glowing performance reviews while spending their days giving presentations to their coworkers about how to avoid being Left Behind In the Permanent Underclass.
At the same time, the rest of the team ends up drowning in this person's "work". Their PRs are 30,000 line, codebase-spanning fever dreams that come with "helpful" documentation that doesn't make any sense. The "spec" is under-defined and the code fails to adhere to it. The tests are made up nonsense. Management softly forces to team to rubber stamp the Rockstar's work. Nobody knows what the hell is going on, defects rise, baffling production down events occur, and nobody can figure out why.
The net productivity for the organization doesn't actually change much. What the technology does do is shift perceived productivity into a small group of people who have offloaded all the work to everybody else.
The con man who job hops has been a rampant problem in this industry for decades. LLMs have just systematized and automated it.
I think if you're skilled and disciplined with your LLM usage, it's pretty unquestionable that software engineering productivity has improved.
For example, I've really enjoyed using Ghostty, Dwarfstar and pi; each of these excellent software projects are written using LLM's with no slop in sight, but importantly these projects are also owned by mitchellh, antirez, and badlogic, who are three incredible programmers in their own right.
That being said, I basically agree with you that ironically, the natural-language programming interface is actually extremely difficult to use correctly, and many AI coding initiatives aren't going particularly well because it's so easy to default into slinging slop and hence productivity improvements are hard to find in many cases.
I'm optimistic that over time the industry will develop better norms and understand how LLM tooling should and should not be used, and we'll all be able to reap the benefits (a man can hope...).
Overall, I am less optimistic than you. It seems like the incentives are too broken for us to reach a new equilibrium any time soon. If we do get there, it'll be on the other side of a graveyard of companies that got too greedy.
Yes, I don't disagree that there will be a lot of churn and pain in the meantime.
I find it's practically a second part-time job to hold back the worst impulses of my non-technical coworkers, the junior developers and management to slop up tickets, documentation and the codebase, which has been somewhat taxing. Unfortunately, it'll probably get worse before it gets better.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
That seems like a big-company problem rather than an LLM problem. I work in a small startup and while I won't claim AI-coding has been without complications, if you tried something like that at my company you would be fired immediately.
That is, the manager would be fired for proving they are totally unable to be trusted with a codebase and have no idea how development works, nor any understanding of what is being done under them.
I've seen it repeat at every scale, from legally-a-small-business to a fairly memorable consult at a Fortune 100 company.
I won't argue as to whether your group has some secret sauce or unusual competency, but I will say that what you have described is absolutely not the norm at the kinds of places that will contract me.
Given your work, is it plausible that you are only going to get contracted by the sort of place that’s absolutely addicted to being Goodhart’d to start with?
I can't rule it out. By the time a company calls me, they're generally fucked to the point where they'll call me.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I keep thinking about this in the field of advertising, where AI is just as good at many creatives at coming up with ideas – and most clients don't care sufficiently about or cannot see the quality gap that talent creates anyway.
I think the difference is that even if a creative gets their ideas from AI, the job isn't just writing down some ideas. Creatives then have to convince the rest of the team, who have to present the idea to a client, who has to buy it believing that their agency is ready to produce it and genuinely likes what they propose (or at least thinks it will look good in their reel). But current AIs aren't build to convince people, even though they are pretty good at making a case for something if you ask them to. Because you have to ask them to persuade you, the efficacy of their persuasion capabilities is inherently lower.
Somewhere along the way (and I have not articulated this properly yet), I have come to the conclusion that today's obedient AIs do not have the skin in the game necessary to contribute to certain types of group dynamics which are at the core of many business endeavours. For this reason I think a different form of more autonomous AI is going to be needed to realise the job replacements possible, and we may need AIs with stable personalities that develop track records in specific fields and must compete against humans 'in the open', before they can take on a whole swathe of white collar jobs that on the surface look eminently replaceable already.
I'm also finding it quite hard to articulate, but I agree that there seems like there's something missing in terms of "volition" or "creativity".
Pretty much all white collar work above a junior level involves a lot of "deciding what should be done next", "convincing other people about what to do next" and "taking responsibility for what to do next"; at least for now LLM's can't really do any of this, even if LLM's are now very good at hill-climbing individual tasks that comprise jobs.
More options
Context Copy link
More options
Context Copy link
As a professional translator, there has been a significant micro impact on my employment, in the sense that work (which has been mostly MT post-editing in any case for 5-10 years) hasn't completely dried up, but the monthly workload has definitely halved in very specifically the last half a year or so. Before that, I had as much or even more work than before, at that point a major customer announced that it would basically transform to LLM-based workflows on an important project bringing that project mostly to a halt, and whatever work there has been to replace it hasn't been sufficient to react previous levels.
The levels might have started inching up recently (there are faint signs of the predicted "they bring the humans back in after throwing them out en masse" cope actually possibly being true) but I'm well on my way to get out of the field in any case.
More options
Context Copy link
Your anecdotes about translation are very interesting, I hope you keep updating us.
I have a theory on why AI wouldn't show up much in the economy even if it was very good at what it does. I do use AI, I think it's great for websearch and stuff (in no small part because Google is now terrible at websearch). I am not a programmer so I don't have strong opinions on how much efficiency it's actually adding there, but while I am open to criticisms of AI as a technology, my theory is agnostic on that and I think still makes sense even if AI is in fact pretty good as a technology, although I wouldn't necessarily claim it's monocausal.
Simply put, I don't actually think much wealth is generated through software directly.
Software is extremely good at streamlining wealth creation; that's why the world runs on Excel. But at the end of the day your accounting guy is making sure that you are efficiently allocating resources, not directly generating resources. (Software also makes it easier to generate inefficiency, too, but let's ignore that for the time being.)
This can have a huge impact on the economy if you're jumping from "I need to hire a scribe to write a letter to travel through bandit-infested territory to inform my business partner about recent developments" to "lemme sit down and write an email real quick" but in a world where you are going from writing a quick email to having AI write a quick email for you, you're not really streamlining things much more.
I think people who work in software sometimes forget that software is just a tiny part of the economy. So from their perspective, if they increase efficiency by 20%, that means a massive increase in productivity. But software is a small fraction of the world economy; a fluctuation there barely shows up in overall GDP. And since AI (unlike other software) costs a good deal of cash to use, the efficiency gains in programmer time are being counterbalanced by increased spending. And writing software more efficiently doesn't necessarily mean the software itself will streamline wealth creation better; if Microsoft replaces their Excel team with AI it might be good for Microsoft's bottom line, but that doesn't mean it actually helps people who use Excel allocate wealth more efficiently.
Where I expect things to start showing up big-time in GDP is when advancements in AI translate out into doing physical things. But I think there are a few reasons this hasn't hit the economy yet:
Firstly, the tools needed to use AI to write new software already exist; everyone already has a laptop. The tools needed to get AIs to do physical things mostly don't. I think that the rudimentary LLM-to-3D-printer pipeline is already solved, and that's definitely not nothing, but it's also just the first step to essentially general-purpose AI fabrication which would be a dramatic step-up in terms of manufacturing. I think if and when this starts to hit the economy, we'll likely begin to see the changes everyone is looking out for. But it will take a long time to build that pipeline, and people are hesitant to even start building it, for reasons #2 and #3 below.
Secondly, the production cycle for software can be pretty quick, and it can be iterated/checked/improved very rapidly. But if you're using your software to build a house wrong, you're in a bad way - you can't just quickly push out a patch. And ~nobody wants to try to use software to build a house before it's proven that it can. So even if we started trying to build general purpose AI fabrication tomorrow we'd still be several years of putting together demonstrations out before people would be ready to use the product, even assuming that LLMs could build a house.
Thirdly, my guess is that the financing isn't showing up. Investors love software because if you make a copy of cool software you can sell it infinitely at basically no cost. LLMs aren't that way because of the compute cost, but I think that they still seem like an exciting software technology to investors, and thus are an ask they are familiar with. Ask them to invest in a hardware-software package that builds a house, and they suddenly start seeing the seams: what about regulation? Is the hardware procurement problem solved? Where's your proof-of-concept? My point isn't that it's impossible to get financing; I think it's already happening, but I think it's a more complicated ask - and when the answer is "yes," you don't see the results for some times because of reason #2.
My guess is that LLMs - even high-end ones - can probably design your D&D mini to print, but would really struggle with designing and constructing even simple appliances at this point. Most likely this problem can be solved, likely via integrating with more deterministic software that allows the LLM to have confidence that its solution is correct. (This is the solution that e.g. legal-oriented LLMs are reaching for).
So if I wanted to build a small factory that could make small household appliances like toasters and microwave ovens, my guess is that it'd take, what, 2 - 5 years just to write the software to integrate with the LLM to validate a very limited set of hardware applications, and probably at least that long to design and procure the hardware that could interface with everything I needed to interface with, all to solve a problem (building a microwave) that is already solved. Long term, having a factory that can design and manufacture novel small appliances quickly and semi-autonomously, with a small labor force, is nearly invaluable, particularly since if properly designed its limitation is in proper software, which can be continually updated, allowing it to manufacture an increasingly wide range of consumer goods. But short term, it's hard to ask someone to give you money to back it when you're not even sure if you can solve the LLM-assisted design problem yet and your promised return on investment is "competing on the toaster oven market."
It seems quite possible that the military-industrial complex, which has already invested in software design to assist and speed up manufacturing (and has deep pockets) is the first place outside of printing D&D minis where we will see something like this start to be implemented at scale. And if it does, that's again an area where the impact on GDP will likely not be very noticeable, since more and cheaper missiles doesn't hit the public consciousness or the bottom line in the same way that more and cheaper cars.
TLDR; that's my best shot at a "a rigorous argument supporting the idea that 'AI just needs X more months to develop Y capabilities and it'll have Z real world effect.'" I should note that I am not exactly an AI-hype man, and I don't have a firm technical understanding of the scale of such a challenge - the above is very speculative. I'm quite open to a future where AI "fizzles" to one degree or another. But even if AI actually is the best thing since sliced bread, I think the above illustrates part of why it will take some time for the impact to hit home.
Very curious as to your thoughts/pushback though.
Sure, I'm happy you find it interesting. I just find translation as it relates to automation intriguing, because a) I have more knowledge about it than most other professions and b) it seems to have stubbornly resisted contraction; at least on paper most people would think translation as a profession has been going downhill for the last 25 years and yet they've been doing better than the average joe would expect (not to say doing phenomenally, but certainly making a reasonable middle class living).
I certainly wouldn't advise anyone to get into translation in 2026, but then again I wouldn't have advised anyone to get into translation in 2016, 2006 and probably not 1996 either. It's just very difficult to predict how these things pan out, and this is a single niche profession amongst the broader economy that's orders of magnitude more complex.
I generally agree that tech bros have a tendency how to overrate how important software is to the economy, but I do think a "software singularity" where software engineering got a massive uplift in productivity would be showing quantitative and qualitative signs at a macro level that don't appear to be appearing so far. That being said, in this case diffusion is still in progress, and probably we'll have a better idea in a few years of how useful it really is.
I definitely agree that if, or more realistically when, things like humanoid robotics become mainstream instead of just gimmicks there are going to be massive impacts on GDP.
I'm not as sure about this applying to things like toaster manufacturing though; I don't particularly have deep knowledge about this, but it was my impression that especially in East Asia, the manufacturing chain for consumer goods is already extremely automated. Perhaps it's a failure of vision, but I don't see how LLM's could really meaningfully increase manufacturing productivity in significant ways unless we're talking about AGI/ASI levels of robots autonomously developing the entire supply chain.
I also don't have particularly deep knowledge about this, so it might be the blind leading the blind, but my understanding is that general purpose robotic manufacturing is still in its infancy. What I am talking about is essentially transforming factories from "toaster factories" to "general purpose fabricators." This would mean that you would not generally need to create a specialized production line for new products.
I think that LLMs, if paired with the proper deterministic tools, might assist in autonomous design and facilitate overseeing robotic production lines. But it's possible that LLMs add little to the task.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Semantically fluent prose does not a specialist make. It demonstrates capability at the shallow end of competence, like a student who memorizes a vast vocabulary but never attempts to use it, but not at the deeper end of actually knowing what to do.
It's infinitely possible to bullshit people through the manipulation of abstractions. 20th century philosophers made an art of it. The 21st century has given us abominations like Yarvin. Probably all priests, prophets, charlatans, and intellectuals have always specialized in this. But when it comes to using ideas to interact with the concrete, physical world, which has infinitely more complexity and no tolerance for failure, all the shortcomings of mere wordplay are exposed. Pretty sophistications have no ability to withstand the friction of reality if their underlying premises are actually flawed. Or if they're wielded incorrectly, deployed without craft.
Thus, while AI can discourse on every single subject in the world, it cannot give you actual good advice for how to put together a formal email. It cannot play Pokémon.
This is what befuddles the self-declared rationalists with their love of intellectual peacocking, the memorization of useless knowledge and gestures so as to impress their peers as part of status game rituals. If their feathers are indeed useless beyond as a means to impress others of their species, then that could mean they are actually quite the vulnerable animals. It impinges upon their egos, and thus its reality must be held at bay, fantasy sustained. Is not intelligence, as the AI demonstrates it, the regurgitation of random stuff from encyclopedias, the absolute height of competency?
To correctly determine the true worth of AI, you must merely ask: what can it actually do? It can put out messy code, it can write artlessly, it can tell you random stuff in a less structured way than a well-fashioned textbook or Youtube video could, but also with greater ease and versatility, and thus can give you a situationally useful new data gathering tool.
But why would you assume any of this can impact the entire world in such a major way?
What are you talking about? It can and does do both of those things.
He's got me blocked, so he may have missed my question and the answer recently. I'd worry that I must have done something wrong, but after reading that swath of ignorance-based insults I'm forced to suspect I must have done something right.
Yeah he's blocking me too but you know, "somebody is wrong on the internet" and all that. It's a compulsion I have.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link