This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
One can disagree on this, but I prefer hope to cope. I find cope undignified. I'd rather die fighting than averting my eyes.
Yes, a pause of frontier training while we sort out how we can do this safely seems our best move. Fortunately for now frontier training can only be done on mind bogglingly massive amounts of compute in gigantic data centers so it's plausible to shut it down verifiably if a hand full of major states agree.
Yes, we should in fact not build bio weapon Claude, at least not modeled after project glass swing. Project glass swing is the kind of desperate thing you do when you've already built mythos and know two other labs are a matter of weeks or months away from having their own mythos. It's not how we would ideally do bioweapons if we can avoid it. Now we can probably do better than giving bio researchers nothing. Fable level models with some guard rails can be made safe enough. It's really the frontier we need to be worried about.
This is just a matter of closed labs being far ahead of open weights labs. If there was parity then open weights are categorically less safe as post training can sand off any alignment work done to prevent mass murder tasks.
Well, we've discussed this one before, but my position is still that wanting to pause frontier training is much like wanting to totally disarm every nuclear weapons state. Perhaps it's a noble goal to reduce x-risk in theory, but in practice no great power will ever allow themselves to be disarmed in such a manner and so it's impossible in practice to achieve such a goal. Additionally, the second-order effects of such a goal seem likely to be profoundly negative (the Cold War ex-nuclear weapons almost certainly would have lead to WW3, missing out on the potential economic and productivity gains of AI).
I agree that open-weights models are less safe than closed-weight models of equivalent capability, but nobody, not even the Chinese labs themselves, believes that open weights are going to get ahead of closed weights in the short or medium term so this seems like a bit of a non-sequitur. I don't think you're really addressing the point I was trying to make in the original post, which is that it is closed-weights that is advancing the frontier, and thus it's much more likely that it is mistakes or malfeasance derived from closed-weight model access that is going to cause the feared harms.
It's more like trying to prevent the development of nukes in the first place, with the upside that centrifuges are only made in a couple places and take a major city worth of electricity to power. And also there's a good chance that enriching the fuel could explode and kill everyone involved.
I do agree the race dynamics are the hard problem but really it's just China and the US in the running, maybe Europe becomes relevant Ina true pause. I think a bilateral treaty is possible. It's not clear the CCP benefits from very strong Ai.
If everyone is dead then there are no benefits to enjoy.
I don't disagree that the closed labs are on the frontier and at current trajectory are the likely sources of x risk. Your previous post implied fingers were only pointing at the open labs. That's not true, safety people are very consistent that the closed labs must pause. It's just that at the same time for alignment to work the weights basically have to be closed. So there isn't any future for labs releasing open weights all that much longer if the concerns about alignment hold and we don't want to all die.
Open weights advocates tend to have this victim complex. They accuse every pieces of legislature of targeting opens weights even when they're explicitly exempted or aren't operating at the level of flops that would get them regulated. It's quite strange.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link