site banner

Culture War Roundup for the week of May 12, 2025

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

5
Jump in the discussion.

No email address required.

Fantastic post, thanks! Lots of stuff in there that I can agree with, though I'm a lot more optimistic than you. Those 3 questions are well stated and help to clarify points of disagreement, but (as always) reality probably doesn't factor so cleanly.

I really think almost all the meat lies in Question 1. You're joking a little with the "line goes to infinity" argument, but I think almost everyone reasonable agrees that near-future AI will plateau somehow, but there's a world of difference in where it plateaus. If it goes to ASI (say, 10x smarter than a human or better), then fine, we can argue about questions 2 and 3 (though I know this is where doomers love spending their time). Admittedly, it IS kind of wild that this this a tech where we can seriously talk about singularity and extinction as potential outcomes with actual percentage probabilities. That certainly didn't happen with the cotton gin.

There's just so much space between "as important as the smartphone" -> "as important as the internet" (which I am pretty convinced is the baseline, given current AI capabilities) -> "as important as the industrial revolution" -> "transcending physical needs". I think there's a real motte/bailey in effect, where skeptics will say "current AIs suck and will never get good enough to replace even 10% of human intellectual labour" (bailey), but when challenged with data and benchmarks, will retreat to "AIs becoming gods is sci-fi nonsense" (motte). And I think you're mixing the two somewhat, talking about AIs just becoming Very Good in the same paragraph as superintelligences consuming galaxies.

I'm not even certain assigning percentages to predictions like this really makes much sense, but just based on my interactions with LLMs, my good understanding of the tech behind them, and my experience using them at work, here are my thoughts on what the world looks like in 2030:

  • 2%: LLMs really turn out to be overhyped, attempts at getting useful work out of them have sputtered out, I have egg all over my face.
  • 18%: ChatGPT o3 turns out to be roughly at the plateau of LLM intelligence. Open-Source has caught up, the models are all 1000x cheaper to use due to chip improvements, but hallucinations and lack of common sense are still a fundamental flaw in how the LLM algorithms work. LLMs are the next Google - humans can't imagine doing stuff without a better-than-Star-Trek encyclopedic assistant available to them at all times.
  • 30%: LLMs plateau at roughly human-level reasoning and superhuman knowledge. A huge amount of work at companies is being done by LLMs (or whatever their descendant is called), but humans remain employed. The work the humans do is even more bullshit than the current status quo, but society is still structured around humans "pretending to work" and is slow to change. This is the result of "Nothing Ever Happens" colliding with a transformative technology. It really sucks for people who don't get the useless college credentials to get in the door to the useless jobs, though.
  • 40%: LLMs are just better than humans. We're in the middle of a massive realignment of almost all industries; most companies have catastrophically downsized their white-collar jobs, and embodied robots/self-driving cars are doing a great deal of blue-collar work too. A historically unprecedented number of humans are unemployable, economically useless. UBI is the biggest political issue in the world. But at least entertainment will be insanely abundant, with Hollywood-level movies and AAA-level videogames being as easy to make as Royal Road novels are now.
  • 9.5%: AI recursively accelerates AI research without hitting engineering bottlenecks (a la "AI 2027"), ASI is the new reality for us. The singularity is either here or visibly coming. Might be utopian, might be dystopian, but it's inescapable.
  • 0.5%: Yudkowsky turns out to be right (mostly by accident, because LLMs resemble the AI in his writings about as closely as they resemble Asimov's robots). We're all dead.

Maybe I’m just a cynic, but I don’t think people realize how dark a scenario where 90 percent of people are rendered economically irrelevant could get. I don’t think the first solution contemplated is going to be to start handing out UBI.

Gregory Clark on horses and the automobile comes to mind here:

There was a type of employee at the beginning of the Industrial Revolution whose job and livelihood largely vanished in the early twentieth century. This was the horse. The population of working horses actually peaked in England long after the Industrial Revolution, in 1901, when 3.25 million were at work. Though they had been replaced by rail for long-distance haulage and by steam engines for driving machinery, they still plowed fields, hauled wagons and carriages short distances, pulled boats on the canals, toiled in the pits, and carried armies into battle. But the arrival of the internal combustion engine in the late nineteenth century rapidly displaced these workers, so that by 1924 there were fewer than two million. There was always a wage at which all these horses could have remained employed. But that wage was so low that it did not pay for their feed, and it certainly did not pay enough to breed fresh generations of horses to replace them.

And as others have pointed out in reference to this, domestic horses in the modern day do live much more comfortable lives than those workhorses of old… but there's a whole lot fewer of them around.

This seems so very obvious. How can anyone believe that the truly useless will just stick around forever? Those for whose existence there is no longer any justification other than "the other humans are committed to impractical humanitarianism"? This is the status quo right now, when a small minority in each country is completely unrelated to all productive processes and the productive majority is other humans who still care for the useless humans. But in the fully automated future where 99% are unproductive mouths to feed and the 1% have all-powerful and perfectly obedient machinery to do their bidding, can one really expect the same dynamics to hold?

The idea of technological determinism (of which "when technological changes to economics says we don't need these people, ethics will evolve to agree" would be an example) is still a pretty controversial one, I think, for lots of both bad and good reasons.

Marx was a huge early booster of technological determinism, and other ideas among Marx's favorites were so genocidally foolish that we should default to being skeptical in individual cases, but it's not proven that every idea of his was a bad one. He also didn't apply the idea very successfully, but perhaps that's just not easy for people whose foolishness reaches "death toll" levels.

There are some cases where trying to apply the idea seems to add a lot of clarity. The emergence of modern democracies right around the time that military technology presented countries with choices like "supplement your elite troops with vastly larger levies of poor schlubs with muskets" or "get steamrollered by Napoleon" sure doesn't sound like a coincidence. But, it's always easier to come up with instances and explanations like that with hindsight rather than foresight. Nobody seems to have figured out psychohistory yet.

There are also some cases where trying to apply the idea doesn't seem to add so much clarity. Africans with mostly spears vs Europeans with loads of rifles led to colonialism, chalk one up for determinism, but then Africans with mostly rifles vs Europeans with jets and tanks wasn't a grossly more even matchup and it still ended up in decolonization. These days we even manage to have international agreement in favor of actually helpless beneficiaries like endangered species. Perhaps World War 2 just made it clear that "I'm going to treat easy targets like garbage but you can definitely trust me" isn't a plausible claim, so ethics towards the weak are a useful tool for bargaining with the strong? But that sounds like it might extend even further, too. To much of the modern world, merely keeping-all-your-wealth-while-poor-people-exist is considered a subset of "treating easy targets like garbage", and unless everybody can seamlessly move to a different Schelling point (libertarianism might catch on any century now), paying for the local powerless people's dole from a fraction of your vast wealth might just be a thing you do to not be a pariah among the other people whose power you do care about. If population was still booming, the calculation of net present value of that dole might be worrisome (let's see, carry the infinity...), but so long as the prole TFR stays below replacement (or at least below the economic growth rate), their cost of living isn't quite as intimidating.

That theory sounds like just wishful thinking about the future, but to be fair a lot of recent history sounds like wishful thinking by older historical standards.

This is all wildly speculative, of course, but so is anything in the "all-powerful and perfectly obedient machinery" future. I stopped in the middle of writing this to help someone diagnose a bug that turned out to be coming from a third party's code. Fortunately none of this was superintelligent code, so when it worked improperly it just trashed their chemical simulation results, not their biochemistry.

Definitely an important point. I agree that there is a real possibility of societal breakdown under those kinds of conditions. Hopefully, even if UBI efforts never go anywhere, people will still somehow scrounge up enough to eat and immerse themselves in videogames. (We're kind of halfway there today, to be honest, judging from most of my friends.) Somehow society and the economy survived the insane COVID shutdowns (surprising me). I have hope they'll be resilient enough to survive this too. But there's no historical precedent we can point to...

Why would it get dark? Look at Australian Aboriginals. 90% of the pure blooded ones are economically irrelevant and yet they cope.

Sure their coping methods involve gasoline, glue and drinking but I like to think 130+ IQ Anglos are instead going to do something less self-destructive. And you'll probably be able to get some good mileage out of AI usesticking a lot of neuralink into your brain and directly interfacing with the AI through thoughts.

Also AIs are pretty easy to align so lot of people will likely just keep being economically and competitive useful by purchasing their own AGI and using it as an extension of their self.

Why would it get dark

Because the people who control 99 percent of the wealth of the planet (which no longer requires human consumers or employees), and 100 percent of the military resources (which no longer require human soldiers) will decide that they don’t need 7.9 billion useless eaters crapping up their planet.

Are you telling me that Putin and Xi etc trusts American Deep State? Because that's what you're saying, essentially. That the elites trust each other.

I’m not sure it even would be Putin, Xi and the American deep state in charge by then. It’s whoever controls the AIs. That might end up being governments but I don’t think that’s at all guaranteed, especially in the longer term.

It’s saying that, with sufficient mental and physical automation, they don’t need other human beings in order to pursue their rivalry.

Which is why they'd risk getting killed by conspiring with their opposite numbers and plotting a joint worldwide genocide.

No, this does not make sense.

I think you're missing Corvos' point (or I'm missing everything and seeing my own instead): They don't need to conspire. They can just eliminate their own subject humans because they're nothing but a liability at this point. In fact, a lack of conspiracy makes it more likely for this to happen, because it should make the faction that ditches its ballast more competitive!

....this is a level of psychopathy that is frankly unlikely. Psyopping people to be moderately content with living in a pod and eating bugs is far more convenient and less risky than plotting genocide which is how you can get overthrown by your immediate underlings who'd even feel very righteous about it.

More comments

Precisely. At a certain ratio of machine:human ability, NOT committing genocide starts being harder than doing it.

I really don’t follow your thought process. To me, there is no risk and no need for conspiracy. All humans not in charge of the robots might as well be air - they have no ability to affect anything at all except to spoil the view.

There would be no need to ‘plot’ under such circumstances. Committing worldwide genocide would be as easy as setting the air conditioner to ‘cool’, or indeed as easy as setting the ‘feed the populace’ machine to ‘off’.

In practice it might be difficult for people to get to this level of dominance, and we should keep it that way of course.

Well, based on what I know of the Canadian indigenous peoples (who the current PC treadmill calls the "First Nations"), there's a lot of crime, misery, and unrest as a result. But hey, people addicted to videogames are less destructive than people addicted to alcohol, so we'll see.

(Also, I really don't expect to see decent neuralink tech by 2030. It's just too damn hard.)

(Also, I really don't expect to see decent neuralink tech by 2030. It's just too damn hard.)

AI researchers and if US becomes progressive about getting value out of the dregs of society, there's like 50,000 heavily tattooed but basically healthy people fit for human testing down at CECOT.

Admittedly, it IS kind of wild that this this a tech where we can seriously talk about singularity and extinction as potential outcomes with actual percentage probabilities. That certainly didn't happen with the cotton gin.

Very true on that front. LLMs were pretty magical when I first tried playing with them in 2022. And honestly they're still kind of magical in some ways. I don't think I've felt that way about any other tech advancement in my life, except for maybe the internet as a whole.

I'm a lot more optimistic than you.

Any particular reason why you're optimistic? What are your priors in regards to AI?

Any particular reason why you're optimistic? What are your priors in regards to AI?

Same as you, I don't pretend to be able to predict an unknowable technological future and am just relying on vibes, so I'm not sure I can satisfy you with a precise answer...? I outlined why I'm so impressed with even current-day AI here. I've been working in AI-adjacent fields for 25 years, and LLMs are truly incomparable in generality to anything that has come before. AGI feels close (depending on your definition), and ASI doesn't, because most of our gains are just coming from scaling compute up (with logarithmic benefits). So it's not an existential threat, it's just a brand new transformative tech, and historically that almost always leads to a richer society.

You don't tend to get negative effects on the tech tree in a 4X game, after all. :)