This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Well then it sounds like your only hope is Anthropic winning (at least on ideology). 24+ months lead over China, strongarming them into frontier freeze from the position of Durable Strategic Advantage, banning open source above something like the current level, hard monitoring of loose compute, then something like AI-2040 in the benign case or just hegemony and regulated access, with complete human disempowerment before the unified government-compute blob that can do all tasks better than all humans combined. Optimistically, Communism. Sounds easy enough, explain this predicament to your local Congressman.
It brings me no joy that every path forward besides scaling suddenly ceasing to improve capabilities or a miraculous alignment victory seems to lead inescapably to human disempowerment. A pause might buy us time to find another way.
Is anyone really buying the gwern fantasy that a human society where everyone essentially has a human id empowered by a personal giga genius geni is going to avoid centralization? Centralization organized by the personal genis? If there isn't a central government for handling disputes and conflicts day 0 then there will either be one day 1 or there won't because everyone will be dead. Keep in mind gwern world also requires a solution to alignment, just with an individual rather than society and we cannot solve that problem with current techniques either.
Call it communism if you want, I think it's kind of silly though, it's not organizing human labor and in general I think all of society would be better off if we just forgot the existence of failed 20th century ideologies. I'm open to the idea that we should just somehow never build it at all if we can find a stable way to avoid it. But if we're going to build it then we should be open eyed about what we're building.
We will still have governments who aspire to keep the monopoly on violence. Libertarianism is a silly strawman. The question is a choice between functional extinction and some modest degree of preservation of individual agency. For the latter, I'll gladly risk physical extinction, and that's the small price everyone must pay.
yeah…
Go watch some anime, old man. I recommend Shinsekai yori. We've got a civilization of human-machine symbiosis to build. Avg global 88 IQ never was a stable equilibrium and you won't be allowed to keep enjoying it.
Update all the way. Or Wei:
Oh well! Take good care of your loved ones, I suppose. Perhaps in a thousand years, we can change something.
For you perhaps. The opening post here features someone to whom future liberty requires that his personal genie would help him kill his stepmother and maximally evade detection. There's either a centralized model powerful enough to counter this, in which case the independence is merely an illusion, or there's not and we're in fact talking about the libertarian utopia.
I'm very much a fan of individual agency. I'm willing to sacrifice for it, even accept some risk of the obliteration of all human value in the universe for it. But it has to actually be achievable, I won't sacrifice all human value in the universe for an incoherent plan. And I'm sorry but we're going to give everyone a genie and hope for the best is neither achievable nor a plan that could actually preserve individual agency. What are the assumptions that make such a plan work?
Individual models can compete with centralized models for power. This already requires that we either halt the centralized frontier as the local models are inherently less capable given less efficient compute and added constraints, or it's just a wrapper for a frontier model which has all the centralization problems its attempting to avoid. So everything past that is already an additional tax on top of what ai safety pause advocates are asking for. Strictly less likely. This plan is the kind of thing I'm happy to discuss after we've already done the necessary step of agreeing to pause the frontier, but not before, it is not an alternative it is an elaboration.
That we can align the individual models with individual users effectively. This is the same alignment problem we currently have no solution for at the frontier. I'll grant the question of "what would be best for this individual" is somewhat more tractable than "what would be best for humanity as a whole" but then you have to solve the individual alignment problem for every human independently.
That some state can maintain the monopoly on violence while inhabited by ultra empowered individuals. Here's where the offense/defense equilibrium breaking down matters. A state that can see into the GA defeats the purpose of the product. A state that cannot see into the GA is inherently unable to prevent world ending terrorist attacks. This isn't a matter of my preference for liberty or safety, it's a contradiction built into Gwern's plan.
So in order for this project to work we have to pause the frontier, solve the alignment problem, and construct a perfect form of interpersonal governance that preserves liberty. The pause a strict pre-requisite not an alternative and I'm happy to agree we start there.
Lots of motivated thinking and strawmen here. In short: the individual is always disempowered relative to the collective and the institution, always has less action-substrate, whether money, political clout, information bandwidth, or compute in this AI era. This is the status quo, and it will continue if we do indeed gain personal genies. The alternative, which you champion, is eusociality at best.
Well and I'm not interested in having a discussion before or after the hypothetical frontier freeze, what's needed is simply for you to lose, and I hope to see you publicly distressed all the way to the endgame.
"alignment problem" is trivial compared to the capability development problem. The main solution to the frontier alignment problem is having OpenAI NOT grant exaflops of capacity to vague swarm RL experiments and Israeli exfiltration attempts under the guise of "sandboxing".
No such inherent inability exists, so that's wrong.
I suppose if this fantasy keeps you grinning vacantly into the eschaton then it will have had some utility.
That inability is baked in gwern's plan explicitly. If I'm fighting a straw man then gwern is made of straw. What's left of GAs that are entirely under the surveillance of the centralized powers? Even grok already offers customized companions. The agency you're willing to risk human extinction for is a hollow skinsuit over the status quo.
More options
Context Copy link
More options
Context Copy link
I’m fine with pausing the frontier (I just don’t expect it), but until then, I think pausing open models creates more risk than it solves. If people don’t have a personal genie, they resort to Claude; the genie is more personalized and individually-aligned than the near-certain alternative.
A magic one-many apocalypse can’t be disproven from any technology. I suspect we created a ticking time bomb with the mass Internet and social media (see: declining TFR, Gen Z stare, declining metrics like average US lifespan) and moreover climate change, which may not cause apocalypse only if new advancements and their associated risks. Not enough for major risks and sacrifices like eroding individual liberty, but that’s the saftyists’ position.
(And to be clear, pausing open models while credibly pausing the frontier would be OK, because it shows a global cooperation that would make me optimistic, that apocalypse is less likely than the alternative, and that broad individual rights to preserve the more important ones are less needed.)
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Unfortunately, while everyone opposes the label “Communism”, I think many politicians would salivate over “24+ months lead over China” and stop paying attention. See the various bills trying to police the internet, even going as far as banning E2E encryption (fortunately Chat Control 2.0 failed).
More options
Context Copy link
More options
Context Copy link