This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Roughly 3 years ago, specifically in December 2023, I've reiterated my longstanding prediction that «the culture war's most important front will be about AI», and specifically AI accelerationism vs anti-AI/AI safety movement:
And the following political compass:
Seems like this was too much complexity for big tent politics in the US, at least so far; and the Trumpian State sees no use for the EA network, and instead (pretty rationally) perceives it as an uncontrollable alternative center of power. Instead we have, essentially, anti-AI left + pro-AI right (with some notable exceptions – eg. Steve Bannon, apparently, has been anti-China specifically because of his concern about Chinese AI progress, and now joins hands with Bernie Sanders). I've also failed to predict the salience and extent of Chinese open source dominance, as well as the bizarre datacenter water use backlash (charitably, can be shoehorned as Luddism?). Nevertheless, we've «succeeded» at the core task of making this a culture war front. The degree of (unnecessary) politicization of the issue is incredible. Trump is doing the yeoman's work, lumping it in with random Blue-coded ideas he disapproves of “Global “Warming,” where everyone was going to be dead by now, RUSSIA, RUSSIA, RUSSIA, UKRAINE, UKRAINE, UKRAINE, or Impeachment Hoax #1, or Impeachment Hoax #2.”. Jensen Huang enjoys his role as the kingmaker who has the King's ear. Sam Altman is opportunistic as usual but, after recent incidents, is genuinely spooked about AI risks (I have it on good authority that OpenAI really intends to slow down some internal projects specifically to bolster their oversight). The quasi-Masonic part is shaping up nicely, too – Dario Amodei, who's become the poster boy of Woke Left AI, promotes Embedded Evaluators with the central example being METR, very deeply connected to Anthropic and the broader EA scene. The EA itself is more explicitly Left-aligned now, despite efforts of some to paint them as TESCREAL eugenicists (the woke cancellation of Bostrom in Jan 2023 was the canary in the coal mine, Yudkowsky laments the missed opportunity of bipartisanship). There are clearly politically coded reports on prominent doomers.
This is all a bit meandering. What I want to know: how do you see this going further? We aren't anywhere close to the wall of capabilities; there are no walls in sight. Anthropic and OpenAI are holding back already, but their products will keep getting better, and fast. Google will make a comeback at some point (maybe in a couple weeks), xAI and Meta may catch up too. In my book, we (well, they) have practical superintelligence that's sufficient for both unprecedented productivity acceleration and really devastating, nation-crippling cyberattacks, which I guess will be discussed with Xi soon. At this rate, in a few months the level of capability Fable 5.1 or GPT 6 Astra will be mostly commodified and uploaded to HuggingFace (owned by Nvidia now). And those are relatively weak systems compared to internal models, which can build models that are vastly stronger still, without even any R&D breakthroughs, just by virtue of synthesizing stronger data trajectories and designing better RL environments. Superintelligence, in other words, is baked in. By Q1 2027 we'll see a jump from Astra that's at least as big as Sol => Astra. I fail to understand how that won't steamroll companies trying to build their moats on products downstream of frontier AI, labs need every bit of revenue to cover their costs, which will only increase due to growing self-imposed safety requirements; employers outside the AI sphere will also be increasingly feeling the heat. There's a whole ugly dimension of circular financing, too.
Americans as a whole are pretty pessimistic about AI even at these mediocre levels of diffusion (I am skeptical of this data that purports to demonstrate much lower adoption than in China, Americans are probably lying more due to widespread negativity on AI, but in any case AI isn't currently doing most of their jobs). Astra+ level models with very low error rate and 300 tok/s output totally can replace most knowledge workers. On the other hand, it seems that so far AI has not caused anything like mass unemployment, and perhaps economists have a point about comparative advantage, so that'll reduce the intensity of class dynamics.
Democrats are likely to sweep both chambers of Congress, which I guess is what Dario is hoping for and why he feels emboldened to antagonize Trump&Hegseth. Nevertheless, the needs of national security and GDP-maxxing (as well as the Executive's will) should prevent any nontrivial exogenous industry slowdown. So by default we'll see further crystallization of Red Accelerationism vs Blue Decelism, and as AI becomes more undeniably scary, that may begin to eat into the Red political base. It'll be interesting to watch, but I'm really uncertain as to how it'll go.
Your model seems generally sound, as far as it goes. You ask how people see it going further. I think the part you aren't clearly accounting for is, for lack of a better term, systemic risk of serious collapse throughout the current socio-economic system. You're framing this in terms of the preferences of large-type movers and shakers because that's generally been proven to be a reasonable thing to do when you're talking about geopolitics. I think for a couple of reasons that you're underpricing the risk of intractably adversarial conditions for large-type movers and shakers as a general class.
If I understand it correctly, your model seems to assume that there are no small-scale overhangs on the road to the really big ones, where AI kicks off a new industrial revolution and previous modes of organizing society obviously stop making any kind of sense as the economic graphs go vertical. I think it's at least plausible that this sort of timeline is correct, that white-knuckling just the next one or two or five years might get us to the sort of runaway tech breakthrough sufficient to reshuffle our present concerns until the socio-political landscape is completely unrecognizable. The problem firstly is that such a shuffling does not seem to be a necessary consequence of an AI revolution, and less necessary the slower the revolution goes, and secondly that it's possible to identify existing overhangs of sufficient magnitude to at least potentially demolish society as we know it. It's probably a lot harder to develop AI in Bostrom's "easy nukes" vulnerable world. The world we are in is probably not that vulnerable, but it seems to me that it's very probably a couple orders of magnitude more vulnerable than you or, I think, most others are accounting for, and maybe vulnerable enough that large, complex organizations couldn't keep the lights on.
Worse, it's hard for me to see that vulnerability as strictly negative. I do not want to live in the world where the Authorities pay me in Starbucks coupons and slot-machine tokens while banning ownership of good steel and machine tools forever. I do not want to live in such a world badly enough that I would rather the world I do live in be vulnerable, and indeed be made additionally vulnerable, if it means significantly lowering the probability of such an outcome.
I think a lot of people arrive at a similar conclusion, if by different paths. People do not want to be cattle. Politics runs on hope, and hope is harnessed by plausible paths forward toward a better tomorrow. Those, it seems to me, are growing increasingly thin on the ground regardless of one's tribal affiliation. Extremism appears to me to be flourishing, and the modern Western world is still best modelled as a massive, distributed search for ways to hurt the outgroup as badly as possible without getting in too much trouble. If an overhang drops, that's the path I expect the energy to be channeled down, and given the overhangs I'm observing, I don't expect that to be a path the current system can easily survive.
TL;DR - current US AI dominance doesn't survive if the labs all burn down before they finish immanentizing the eschaton.
I guess I'm just optimistic about the US. People call me a China shill, but the US has too many capable people in too many well-resourced companies (not clear how they'll cope with Frontier Labs trying to eat them, but the state may prefer them non-eaten and that could be decisive). Even the government is not entirely inept (at least it's more capable than EU governments, Russia, LatAm, most of the rest of the world). In particular, the US is excellent at kicking the can down the road (ballooning debt is a case in point). AI productivity gains might turbocharge this capability, and indeed this seems to be explicitly the plan – Bessent proposes just growing out of the debt, and growth is all about AI now. So, the system will receive generous injections of surplus energy to deal with overhangs, as you call it.
Whether this will be enough to compensate for new sources of entropy, and whether that compensation will take the form of a broadly tolerable American way of life plus AI upgrades or a Palantircore dystopia with UBI, this I admit I don't know. But I wouldn't bet on some societal collapse that exceeds the costs of another culture war conflagration. Betting against the US is, historically, a fool's errand.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link