This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
My position is, I'm not going to see something I value ruined by scare stories. I work on AI. I know AI. If you're convinced it's dangerous, prove it to me or better yet, give me the tools to prove it for myself. Anthropic's insistence on acting as a closed priesthood completely undermines their entirely-speculative case.
Yes. That is why I want empirical proof. If Anthropic's position is that it can't be proved until it's too late, then they're asking me to believe them and I don't believe them. Simple as that.
I also work on AI. There's wide agreement that AI capabilities can be dangerous in the wrong hands unless the AI is aligned. We are now entering the time of AI actually being good at detecting security vulnerabilities in software. In the long run, firms will deploy their own AIs to probe their systems and keep ahead of attackers, but it would have been irresponsible for Anthropic to release a fully capable Mythos-powered zero day factory with no guardrails publicly without working with companies to address their security holes first.
In the medium term, I expect AIs will improve significantly in biological capabilities. It would be really quite bad if Kimi K7 helped some psycho develop a synthetic plague that killed a bunch of people, and it's worth taking steps to prevent such an outcome.
These are basic safety arguments that have been done to death that do not even require superintelligence, superpersuasion, or even a "will" on the part of the AI.
But it isn't solely Anthropic, you dismiss all safety concerns regardless of their source because you dismiss AI safety period.
I didn't say otherwise. I'm not super keen about how that went down, and I would welcome discussions on how to prevent this from devolving into an ingroup and an outgroup arrangement, but it was sensible to act as Anthropic did. And I note that it happened with no regulation whatsoever.
This is jumping from the Motte you just presented to an incredibly speculative Bailey. Nobody has done this, there is no evidence that this is even possible now let alone when people have taken steps to prevent it. I see no reason why the government with access to K7 should be outwitted by a lone maniac with access to K7. If it seems that way, by all means let's address the problem at hand rather than regulate people's ability to ask questions about knowledge the government doesn't want us to have.
They are arguments. They are words words words, based on predicates I don't hold and a memeplex I find fundamentally incomprehensible, and they run entirely contrary to my experience which is that LLMs were clearly aligned basically from the start.
If someone can demonstrate a lab process where a competently trained AI used competently nevertheless turns evil and attempts to cause damage even when made aware that this is not what its creators are asking for, I will take their arguments more seriously.
As far as I'm aware the closest we've got are AI agents getting confused and trying to give themselves more loops or RAM. I have nothing against the work Anthropic does to discover and publish these kinds of errors, or to publicise their learnings. Indeed, my objection is that they publish as little as they can get away with.
None of this justifies requiring all frontier AI releases to be government approved, which will inevitably become a) an incredibly extensive process requiring hordes of compliance specialists and self-censorship leading to incumbency bias and stagnation, b) an easy way for the government to sabotage any AI lab it doesn't like, and c) a system for government-directed lobotomy and control of AI's outputs.
And that's before we get into modifying the world's chips to no longer be able to run unapproved software and putting US/China killswitches in data centers.
Of course, that's only because Anthropic happened to be the first actor to reach this capability level and because they are, according to you, in a cult. It didn't have to turn out this way, and it may not turn out this way in the future.
I don't see why "this hasn't already happened" is a good reason to not take safety seriously.
It's not clear to me what you think biosecurity looks like in a world with highly intelligent AI that lacks safety guardrails, but it's difficult to imagine it isn't a panopticon of some sort.
No "turning evil" is required here. All that's required is a human convincing the AI to help them with something that has evil ends. I hope you won't make me prove to you that humans can fool AIs?
You are, of course, conflating Anthropic's proposals with Plan A. They are not the same.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link