This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Dead forum?
Today in AI news, OpenAI has reportedly released GPT-6 Astra. Rumors on X suggests that it scores 99% on Arc-AGI-3, meaning that the benchmark is essentially dead now. For comparison, Fable only scored 30%. Does this mean AGI is here? Many are saying that it is. Color me skeptical, but its getting harder and harder to find concrete tests that AI cannot pass. The god of the gaps gets smaller.
Many intelligenct people have important things to say about this. But what do the 84 year old communists think? Bernie Sanders today has introduced legislation calling for a ban on superintelligence with a 20 year prison term for attempted superintelligence. He also wants (shocked Pikachu) a government regulatory agency. Although he mumbles something about international cooperation, its clear that China will not be subject to these rules so it solves nothing.
I will give him credit for at least recognizing the importance of the issue.
We could be entering a world of unimaginable change. And, given the political systems in place in the US and China, the midwives of that changes will be those on the cusp of senility.
With all the renewed hype around Astra and AI mathematics, I have been feeling that I haven't really seen a response I'm satisfied with from the AI bulls (at least the imminent mass unemployment / AGI / singularity types) to the 80IQ skeptic question of "If AI is so good, why is nothing happening?".
FWIW, if I knew in 2020 that in a few years we would have AI models that can get a perfect score on the IMO for pennies and meaningfully contribute to research mathematics I have expected that we basically live in a sci-fi world as well; yet now we have these models for $20 a month, and literally nothing I'm aware of has happened on a macro level clearly attributable to AI, either quantitatively or qualitatively, apart from the distortion of the AI capex buildout. In fact, it's actually GLP-1's that are basically a complete miracle on a macro-economic and public health level, and where there actually are many macro statistics that are visibly affected by GLP-1's.
On some level there probably is anthropomorphization going on here where the natural human assumptions are that
when in fact the conclusion does not actually follow from the premises; it is possible that AI is extraordinarily economically valuable and powerful for other reasons, but this does not follow from being superhuman at mathematics.
Modern pure mathematics is, as we seem to be finding out, not particularly economically valuable in and of itself; mostly mathematicians are lionized for their intellect because a human with that level of intelligence could obviously do significant amounts of other economically valuable work, but this seems extremely non-obvious in the case of AI.
The "best" response I've seen is something along the lines of "actually AI is still heavily limited by spikiness right now but the exponential is going to keep going bro" which I suppose is honest as to current capabilities and not disprovable, but it does feel like a retreat away from "AI is already crazy good" as many people have been and are touting, and towards magical thinking; somewhere along the line the party line swapped from "AGI imminent" to "AI that creates AGI imminent".
My take-away certainly isn't that AI is a fugazi; obviously it's powerful technology and I'm still quite bullish in the mid-long term, but I've certainly been feeling myself getting anti-hyped with recent releases as a result of this growing discontinuity of any real-world effects.
I’d caveat that there have been some massive changes: translation is basically solved, ‘Rosie-the-robot’ style control is no longer science fiction but a creepy and dumb product, ai-assisted chip development and software engineering is a default option. Recursive self-improvement has been slow, rather than non-existent.
Some of the delay is just logistics. There’s probably over a hundred people worldwide following the AI cancer vaccine dog’s approach, but it’s going to be three or four years before we have clear and convincing academic papers even assuming it works and generalizes, just because that’s how medical trials work. AI driven assistance and funding for nuclear reactors might make building them in the US possible, but it isn’t going to make the NRC respond in less than six months. Silicon fab cycles are measured in years. Structural stuff just can’t be that fast.
Others are tooling. There’s a massive space where many small and medium businesses would benefit from custom code, for example, and AI can do it… but if your interface and sanity checks require a programmer or project manager anyway, it’s not really available yet even if the AI can do it. And Claude Code is not ready for prime time use by normies.
A lot of stuff is also just obscured when it does work. An AI-derived optimization algorithm for matrix multiplication just gets swallowed as a delta in efficiency from expected performance. An AI-built software tool just looks like a (verbose) software tool.
I personally think the effects of current AI on software engineering productivity are somewhat overstated (at my fairly AI-pilled software firm there's been many "AI-assisted" design documents and RFC's of dubious value, lots of rewrites of internal platforms, test suites and frameworks that look productive but haven't really been that useful, and only somewhat modest improvement in terms of actual firm-wide user-legible value), but as a whole I would agree that software engineering productivity has been meaningfully enhanced via AI.
There doesn't yet seem to be meaningful impact on software engineering employment, or anything noticeable on a macro scale as a result of modern LLM coding, but I think I would agree it is too early to determine the wider effects of coding agents one way or another.
I think translation is an great example of what I mean by "if AI is so good, why is nothing happening"?
We've basically had "solved translation for dummies" since 2016 Google Translate NMT, in the sense that the output wasn't great, but someone with no knowledge could bumble through understanding the gist of a brochure or a manual, where previously it might have been literally impossible for them to understand the text without consulting another human. I remember quite a few people in language learning communities I was part of at the time were already getting blackpilled on the full automation of translation as an inevitability all the way back in 2016.
LLM's took that up one step further, since about early 2023 we've had "solved translation for normies" in the sense that LLM's could reliably provide semantically correct translations for pretty much any input/output language pair. Maybe it makes some minor mistakes or it sounds weird to the critical reader, but nothing that really matters in the vast majority of use-cases for the vast majority of people.
Most if not all my friends in 2016 would have agreed that the existence of these models would be the end of human translation, and yet in nearly four years there still has been no significant macro impact on translator employment, or really anything noticeable on a macro scale that might be noticeable if translation was truly "solved".
So then, why is this case? This topic came up when I was chatting with some translators the other day, but I wasn't really satisfied by the points that ended up being brought up.
Is AI translation too expensive? Perhaps plausible for the earliest LLM's, but certainly not any remotely recent model, at least when compared to the fully loaded salary of a first world translator.
Is AI translation not reliable enough? I thought this might be the case, but I was assured that this isn't really true for most translation jobs. It certainly matters to have a human liability shield for regulated medical, diplomatic, legal use cases etc, but most translators don't work in those niches, and for most translation work accuracy is not really that paramount - most people are happy with "good enough" and people who care about translation quality have always been a small minority.
Is diffusion simply incomplete and all the translators will be unemployed in X more months or years? Plausible, but I'm not convinced this can be the entire story either; at this point more than half the planet has tried ChatGPT or Gemini, and AI is top of mind in nearly all corporate boardrooms right now. Even if diffusion is slower than expected, I would think it reasonable that after nearly four years there should be some sort of real world impact.
Are there induced demand effects outweighing the supply increase? Certainly demand for Japanese<->English, Chinese<->English and Chinese<->Japanese translation (and likely lots of other language pairs) has exploded in the last 10-20 years; the expectation now is for games, media, writing to be rapidly localized across many different languages when you used to have to wait years for any kind of translation if at all.
This can't be the entire story though; if translation really was solved, it wouldn't matter how high demand was when AI would be meeting the vast majority of it. Perhaps it's possible that most translation is solved, but induced demand for the translation that is not solved means that most translators still have jobs?
I agree that logistics and tooling are significant bottlenecks for anything that needs to interface with the physical world, but I think it's a logical criticism to make that even solely in the worlds of bits and words there's really been very little happening in the world relative to the capabilities of LLM's.
In general I don't think I've seen a rigorous argument supporting the idea that "AI just needs X more months to develop Y capabilities and it'll have Z real world effect", when it seems that nobody really understands what capabilities are missing from the current models that are currently preventing them from having major real-world impact in the first place.
Semantically fluent prose does not a specialist make. It demonstrates capability at the shallow end of competence, like a student who memorizes a vast vocabulary but never attempts to use it, but not at the deeper end of actually knowing what to do.
It's infinitely possible to bullshit people through the manipulation of abstractions. 20th century philosophers made an art of it. The 21st century has given us abominations like Yarvin. Probably all priests, prophets, charlatans, and intellectuals have always specialized in this. But when it comes to using ideas to interact with the concrete, physical world, which has infinitely more complexity and no tolerance for failure, all the shortcomings of mere wordplay are exposed. Pretty sophistications have no ability to withstand the friction of reality if their underlying premises are actually flawed. Or if they're wielded incorrectly, deployed without craft.
Thus, while AI can discourse on every single subject in the world, it cannot give you actual good advice for how to put together a formal email. It cannot play Pokémon.
This is what befuddles the self-declared rationalists with their love of intellectual peacocking, the memorization of useless knowledge and gestures so as to impress their peers as part of status game rituals. If their feathers are indeed useless beyond as a means to impress others of their species, then that could mean they are actually quite the vulnerable animals. It impinges upon their egos, and thus its reality must be held at bay, fantasy sustained. Is not intelligence, as the AI demonstrates it, the regurgitation of random stuff from encyclopedias, the absolute height of competency?
To correctly determine the true worth of AI, you must merely ask: what can it actually do? It can put out messy code, it can write artlessly, it can tell you random stuff in a less structured way than a well-fashioned textbook or Youtube video could, but also with greater ease and versatility, and thus can give you a situationally useful new data gathering tool.
But why would you assume any of this can impact the entire world in such a major way?
What are you talking about? It can and does do both of those things.
He's got me blocked, so he may have missed my question and the answer recently. I'd worry that I must have done something wrong, but after reading that swath of ignorance-based insults I'm forced to suspect I must have done something right.
Yeah he's blocking me too but you know, "somebody is wrong on the internet" and all that. It's a compulsion I have.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link