This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Anthropic is Now Running Television Ads Threatening Humanity
I saw this ad play on my tv while watching the England vs Argentina postgame. I could not believe what I was watching. I had vaguely heard of a tone-deaf video from Anthropic communications, but I hadn't actually watched it until it popped up on my screen against my will. I am stunned that this is how Anthropic chose to portray themselves to the general public. Maybe this sort of messaging works on a tiny subset of niche technological policy nerds, but they played this during the World Cup.
The ad opens with scenes of destruction: a house ablaze, civil unrest in the streets, hundreds of graves at Arlington National Cemetary. A concerned voiceover asks the questions that many of us ask of the AI industry every day. "Can AI be trusted?" "Who's gonna hit the brakes if we need to?" "How do we really ensure that what we're aiming to acheive really does benefit the majority of people?" All good questions that the rest of the ad is utterly unconcerned with answering.
It feels like a threat. It doesn't feel like they're saying, "oh please God somebody stop us". It feels like they're saying, "just try and stop us, MUAHAHAHAHA!" I suppose it's good that they acknowledge the gravity of what they are undertaking, but that doesn't really matter if they keep doing it anyway.
Also, sort-of-unrelated-but-not-really: Is "Total Clanker Genocide" an acceptable sign for an anti-AI protest?
Is Anthropic evil or is it stupid?
For the longest time I thought they knew what they were doing, that this is all a ploy to force governments to regulate and thereby establish their dominant position over the field forever, since any competitor would no longer be able to even get the computer required. And then they can happily be the people who decide what you get to know about in a society that relies heavily on their product, a much stronger position that the people who regulate them.
But if they really wanted that, why the hell would they put insane conditions on military contracts? Maybe instead they got high off their own supply and are actually convinced they're making God and that because they're such right thinking bourgeois liberals, God will have their values and it's futile to resist the promised eternal rule of the managerial theaterkid Reich. Just be a nice heckin human bean okay, God cares.
Is there a third option I'm missing? Because what could bring someone to have made this "ad"? It's literally just saying "yes we are going to destroy your life, but not to worry, we still care about you in some abstract sense". Regular "we care" corporate bullshit at least has the decency to not fear monger about itself whilst delivering the empty platitudes.
Sincerely, do you imply here that AI is not going to be a strategically decisive technology? At present, Anthropic's power is nothing before Pete Hegseth, but that's not guaranteed to remain the case.
It's a technology, it is not the Lord God Almighty. And the dreams/fears seem to revolve around "We will create super-intelligence, and then the thing magically becomes alive just like us, so just like us it will have goals and aims of its own, and we have to make sure it is well-instructed in How To Be Nice Liberal and then it can take care of us like pampered pedigree cats as it colonises the known universe".
The life of a pampered pedigree cat may or may not be an ideal to strive for. You get the pampering, but your life is literally no longer your own, and your entire line of descendants gets warped by selective breeding to the tastes of your owners, not your inherent nature. If your owners decide that squishy noses are cute, you will be bred to have squishy noses, and who cares about how it affects your respiratory system.
I don't think we're going to get "AI as conscious entity". I do think we're going to get "more and more use of AI, and more and more idiot decisions by idiot humans to hand over more power to make decisions and perform actions to AI".
I think the talk about "becoming alive" or "having goals and aims of its own" doesn't reflect the fears/concerns of Anthropic or of anyone who thinks like Anthropic with respect to fearing an AI apocalypse. The concern is that super-intelligence need not be alive nor have any goals or aims of its own to be a humanity-ending danger. Because almost any task an intelligence is handed could be divided into sub-tasks, and we as only human-level intelligent beings can't be expected to reliably predict what a superhuman-level intelligence will choose in terms of its sub-tasks, we have no way of knowing that human extinction isn't one of the side-effects of one of the sub-tasks an ASI uses as a step to accomplish whatever task it was handed. Humans have biological limitations as well as intelligence that is both human-like as well as human-level, which makes it so that humans that would make similarly apocalyptic decisions usually get filtered out before they can get enough power to implement them. The concern is that an ASI, lacking such limitations as humans, as well as having an intelligence that's both vastly greater than that of humans and vastly different to that of humans in ways that we can barely understand, wouldn't get filtered out before it can implement apocalypse, all without being alive or following anything that could be considered a goal or will of its own.
Then all the alignment talk is a dead-end or red herring. "Make sure AI shares our values" means what, precisely, if it's "this thing is as conscious as a brick and while we can write pretty scripts to make it pretend it's a real boyfriend who loves you for your wild, daring, passionate, unconventional self it's just a talking doll"?
"Let's code the brick so clever or dumb but devious people can't talk it into writing 'how-to' instructions for a global plague" is more honest about aims but less sexy than "let's teach our successor intelligent species to cherish our timeless human values so it will love and honour us as its parents" which is what the current alignment chat sounds like to me.
I have no problems with "it's dumb but dangerous". I have a whole skip full of problems with people going on about it as if it will become super-intelligent and then agentic and then develop its own aims. The danger is not the machine, it's the people who set it up in such a way that it can then wander off down byways of "this isn't what I meant when I told you to do this" without it needing to understand anything in any meaningful way, and that's the trouble we're already seeing with "the thing is thinking in ways we don't understand and can't follow and wandering off on its own byways" reports.
It means something like, "make sure that when it pretends it's a real boyfriend or when it's used to code your next iPhone app or when it's used to design new scientific experiments or etc., it behaves in a way that is consistent with something that shares our values." I'm not sure what the "conscious as a brick" has to do with this; whether or not AI is conscious or has free will or agency are very interesting questions, but they're mostly irrelevant to issues of AI alignment, which has to do with AI behavior.
If the latter is what the alignment chat sounds like, I think it must be a result of manipulation on the part of people who expose the chat to you. I've seen pretty much no talk in AI alignment that could reasonably be paraphrased as anything like that.
But it's not dumb and dangerous; it's (definitionally) generally intelligent and dangerous. And specifically dangerous because it's not dumb and is generally intelligent. Now, whether AGI will lead to ASI and how likely that is is an empirical question, though I'm personally convinced by arguments that it's pretty darn likely. But whether the AI is generally intelligent or superintelligent, that has nothing to do with whether or not it's agentic or develops its own aims.
The point is that a tool (or, for that matter, anything, including biological organisms) need not have agency or have aims or goals of its own or anything that we would characterize as "free will" or "consciousness" or "sentience" to do things that detrimental to humanity, and the fact that the tool is generally intelligent in this case means that we currently lack a way to have meaningful level of confidence that the behavior of the tool will be within the bounds of what the tool-user considers reasonable bounds. When dealing with current generally intelligent things - i.e. other humans - we have so many things in common with them - certainly physically and biologically and most likely culturally as well - that we don't have to explicitly spell out every little thing, and we can make fairly reliable predictions about how things we spell out to them will get translated to action. We lack such things for AI as of yet, and so we lack an intuition or a particularly reliable way to figure out how AIs will fail or misunderstand our intent.
Well yes, ultimately the people who created the tools and/or wield the tools are responsible, not the tools themselves. Guns don't kill people (but they sure help), I do. The problem is that the people who create, manage, use, etc. the tools lack the capability to set it up in such a way that we can be confident that it won't wander off away and run KillAllHumans.exe without letting anyone know. The easy solution that presents itself is to just not use the tool if you can't set it up with that level of safety. One major problem is that AI is so darn useful that it's hard to get the political will to suppress the supply from providing to the demand. The other big problem is that we lack en enforcement mechanism to make sure that no one sets up this tool, and as such, entities that choose to set up and use the tool anyway could gain power over us and make our lives hellish before all our lives are snuffed out by the un-aligned ASI.
This is the dilemma that's being discussed and debated about right now in the AI alignment chatting.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link