site banner
PaperclipPerfector  1mo ago (text post)   25161 thread views

Culture War Roundup for the week of July 13, 2026

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

3
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

Anthropic is Now Running Television Ads Threatening Humanity

I saw this ad play on my tv while watching the England vs Argentina postgame. I could not believe what I was watching. I had vaguely heard of a tone-deaf video from Anthropic communications, but I hadn't actually watched it until it popped up on my screen against my will. I am stunned that this is how Anthropic chose to portray themselves to the general public. Maybe this sort of messaging works on a tiny subset of niche technological policy nerds, but they played this during the World Cup.

The ad opens with scenes of destruction: a house ablaze, civil unrest in the streets, hundreds of graves at Arlington National Cemetary. A concerned voiceover asks the questions that many of us ask of the AI industry every day. "Can AI be trusted?" "Who's gonna hit the brakes if we need to?" "How do we really ensure that what we're aiming to acheive really does benefit the majority of people?" All good questions that the rest of the ad is utterly unconcerned with answering.

It feels like a threat. It doesn't feel like they're saying, "oh please God somebody stop us". It feels like they're saying, "just try and stop us, MUAHAHAHAHA!" I suppose it's good that they acknowledge the gravity of what they are undertaking, but that doesn't really matter if they keep doing it anyway.

Also, sort-of-unrelated-but-not-really: Is "Total Clanker Genocide" an acceptable sign for an anti-AI protest?

Wow. And I was just making the claim that, actually, Anthropic's ads are just positive and all about productivity hacks: https://x.com/DainFitzgerald/status/2070990676792394180

It's not a threat, it's obviously targeted at median people. It's aired on a sports match, it's all about grandma's medicine and if AI will help the works of the world pay off their credit card debt. If you go to the hard questions url it's filled with median looking teachers and unemployed disability check receivers babbling about rent, la favela communidad del pueblo, and «medicine» for their «chronic health conditions.»

I don't see that as a threat, just a terrible ad from a company going "buy our product and be Happy Upper Middle-Class Human Lifestyle" which is crap.

Playing it during the World Cup was smart. Unlike you, the masses wouldn’t think about it deeply. The ad began with the negative possibilities of AI, and gradually moved on to express hope and the positive use of AI. The audience was meant to feel their concerns validated and addressed (even if that didn’t actually happen) and then finish the ad with a positive feeling on AI. The ad was designed to work on an emotional level, and I think it was effective at that.

Apropos of England vs Argentina, I absolutely called it: https://www.themotte.org/post/3822/culture-war-roundup-for-the-week/458055?context=8#context

Wouldn't be England if they didn't first give us hope only to lose in the worst possible way and to the worst possible team to lose to.

Also Argentina's one strategy the entire tournament, other than paying for rigged reffing (except during the England game., too expensive or obvious at that point maybe), has been to tire the opponent out for 75 minutes and then kick it into gear. I'm suspecting they do it on purpose to make betting market dips. Who knows, at least it was easy to capitalize on.

Half remembering a stat from the radio this morning: around 70% of their goals occur after the 75th minute of a game.

I am British living in England, and support England, so I feel the pain, although I can acknowledge that we deserved to lose this one. (So did Argentina in the first half).

But I can at least be aware that England were in a World Cup semi-final. For the second time in eight years. And that is a stunning achievement. In 1990 everyone knew this - even after going out on penalties to West Germany, the team that made the semi-final were hailed as heroes. But English fans are more susceptible to entitlement that any other country except Brazil, and now we have far too many people thinking we deserve to win the World Cup and anything less is a disappointment that the manager should be blamed for.

I'm still dreaming, but I think the 60 years of hurt are basically over now. If England had played their best and the opposition had not, we would have won the World Cup. But the opposite was what happened in the second half of yesterday's game.

We were lucky it wasn't 4-1, and we only have the courage and bravery of the goalposts to thank for that.

I don't think we deserved to win, getting to the quarterfinals is the standard benchmark for England and anything more is a bonus. The problem people have isn't with losing per se, it's the way that we lose. Even the way that we win. Let's say that somehow we'd held off Argentina for 40 minutes (!). Would you be proud of the result and confident going into the final, or feeling an unpleasant mixture of relief and trepidation? You can excuse keeping your back against the wall in a match where you're a man down, but in any other scenario it's an admission of a shameful lack of ambition. Too much of England's success has come from lucky breaks against weak opponents and a failure to capitalise on our advantages.

Relevant comedy sketch

Third place playoffs, eh?

OTOH in countries like Finland (where we have never participated in the World Cup and only once in the Euros) there's a long tradition of many people supporting England on the basis of old "honest lads who play hard, don't go for diving" narratives, and this World Cup, at least, has been a nail in the coffin of that myth.

Sadly that's the way the game has developed. I think the refereeing of this tournament has done a good job of limiting the worst of these tendencies and hopefully there's more that can be done. The new rules about time wasting have been a welcome change too.

To be fair, the ratio of fouls Argentina:England was literally 2:1. And the yellow cards were worse. England played a comparatively clean game.

England players don’t do the grandiose ‘let me hear your joy, pour champagne on me now’ routine when they score either.

Playing less dirty than argentina is nothing to brag about.

I'll be honest, I only tuned in for the semi-finals. I assumed that lots of fouling was just how the game was played these days, and that Argentina was overdoing it.

There have been a lot of fouls in this world cup but Argentina has had, just like previous world cups, a lot of them.

Was referring more to the Norway game here.

Ah, didn’t see it :)

and to the worst possible team to lose to.

Explain please - what is the hate towards Argentina? I have seen it all around the social media, but no justification.

They play dirty, but the clincher is they're proud of it.

VARgentina have done enough to fill a mid sized book for why they and their team should be disliked and shunned by all right thinking people.

A good starting point is to take a look at Maradonna's cheating "hand of God" goal where he deliberately lied to the referee about what had happened; this is just not cricket.

The England-Argentina soccer rivalry is one of the bitterest in the game.

How is it a rivalry when they only play every 25 years?

England is still bitter over the 1986 game where Maradona scored a goal off a handball and the ref let it stand, Argentina hates England because of nonsensical posturing over the Malvinas.

It's white people acting like Balkaners all around.

https://en.wikipedia.org/wiki/Falklands_War

"We" are the reigning world champs and we talk a lot of shit. Last world cup we also came up with this gem, so there's also a racism component to it. It's all banter, but in a fully interconnected world of people without context, people get mad.

All the "rigging" allegations are ex post justification of the hate, if you started to dissect every call in other games with the same level of scrutiny you could build a similar narrative, soccer refereeing is contentious by nature and VAR hasn't helped. I know because back when we weren't winning anything, people were also coming up with stupid conspiracies as well.

No, that Egypt game was rigged. Without that corrupt ref they would have gone to extra time at least. And that doesn't take into account the way teams feel like they can't properly defend against Messi because if they breath on him they get fouled because he is FIFA boy. Messi is a top 5 striker surely but is he really number 1? How can we know? Well, they want to make him look like number one by giving him two world cups in a row. But after some of the reffing it doesn't look legitimate to me. It's a team sport too, just because his team keeps winning doesn't make him any better than eg Ronaldo at the same age. The cult of personality around him is too insane.

Messi is 39, Ronaldo is 41- not exactly a gigantic difference there. He's also legitimately the best player in the world, even if it's exaggerated somewhat by celebrity, and had that reputation even before he became FIFA's flagship player to try to sell the beautiful game to the world's few holdouts against it.

See here, this puts Ronaldo at 2 and Messi at 5. Also an entire continent and global power center is not a small holdout.

From an England perspective, it goes back to the Falklands war and the 1986 World Cup quarter-final where Maradonna punched the ball into the net and the referee missed the blatant handball and allowed the goal.

From a neutral perspective, there have been a lot of complaints that the refs are protecting Messi in an unfair way, which link into the usual allegations of FIFA corruption and greed. Since the Balogun affair it is also possible to link it to rumours of Trumpy interference - the idea that Trump has leant on Infantino to help his ally Milei win re-election is too good to check.

Never heard the Trump angle, but it doesn't sound like Trump. Maybe something that someone whispered to him and he then did, but if FIFA could totally change their refereeing in response to Trump they would've done it for the Belgium-USA game. With the amount of money they're making having the US as a host- and the amount of money they stand to make from the USA going to quarters- they'd have already done it if they could.

They did bloody well, though. I though they were going to get pasted but they held on till the 80 minute mark.

I've interpreted the images at the beginning of this ad as "there's so much suffering in the world already, all caused without AI, that it makes these questions sound stupid", but maybe I'm reading too much into it.

I am still perplexed by Arlington in particular. What concern were they even trying to convey? Was it something as detached as "hmm look up common symbols of loss of life that register to American normies, ok this one would work"?
They have just had a very ugly fight with the Department of War because they refused to maximally assist Trump and Hegseth with their lethality maxxing, and specifically were accused (as it happens) of endangering the lives of Our Boys who bleed and spill blood for Freedom. So what's the narrative?

"If AI is unchecked, it may send Americans to die"?
"Unaligned AI will fight the US Army"?
"If Anthropic doesn't have supremacy in AI, many American servicemen will die fighting China"? (actually a plausible argument, but inconsistent with the rest).

The voiceover provides little insight.

As a palate cleanser, I offer the new video from Kimi Moonshot. Absolutely meaningless, clearly aping earlier, better-designed Anthropic ads, but pretty. It hypes up the upcoming K3 that seems to be stronger than Opus 4.8. Looking forward to the next geopolitical post from Dario.

I thought it was an ad just for AI engineers to show 'yeah we get it, this is existential, come work with us and not Zuck's short-form video hell factory or OpenAI's revolving-door chief safety facade officer'.

Anthropic doesn't care about the peasants, they want to draw more talented people from Google or OpenAI or Meta. Peasants include the bulk of the US govt too I think, standing up to Trump enhances Dario's clout amongst those he's really courting.

This is my interpretation, too. Anthropic might be targeting investors, but they're also targeting current and potential future employees. They've already developed a bit of an EA cult at the company, and this reads more like reassurance to existing employees and enticements to future ones to jump ship and come work for only company that cares.

And that's what makes them much more dangerous than their competitors. They are true believers, from what I can tell, and interested in hiring other true believers.

Anthropic doesn't care about the peasants, they want to draw more talented people from Google or OpenAI or Meta.

The interesting question is why those people are particularly likely to be watching a World Cup semi-final between England and Argentina. I suppose everyone who isn't a heritage-American or an IPL diehard is going to be watching a World Cup semi-final, but given that heritage-Americans and Indians are over-represented groups in Silicon Valley, it seems an inefficient way of reaching the target audience.

Blue tribe America likes English soccer- well not really, they don’t actually like sports, but they claim to like English soccer, so they’re probably watching a World Cup semifinal with England in it. Especially after Norway lost(backing Norway was trendy enough to make this World Cup more popular than usual).

Also, sort-of-unrelated-but-not-really: Is "Total Clanker Genocide" an acceptable sign for an anti-AI protest?

I'm in favour of normalising hate speech against clankers, at least.

Anyway, I can sort of see what they were going for here - we know you have concerns, we're listening, we want to take your worries seriously. I don't think they understand how widely loathed their technology is, though, and therefore how much what they're saying comes off as a bad parody or even villainous and sneering.

I don't think they understand how widely loathed their technology is, though, and therefore how much what they're saying comes off as a bad parody or even villainous and sneering.

I think @JeSuisCharlie had a good quip about this. "If you need a style guide for not sounding like an evil robot, something something". The style guide being a reference to one of Scotts actual posts, though I don't remember if it was actually making the evil robot comparison, or I'm recalling it that way due to being primed by the current subject.

The line was "If feel like you might need a style guide to avoid sounding like an evil robot, maybe you ought to take a step back and ask, "are we the baddies?"

How much is it an "evil robot" in an objective sense, and how much like what created to be an arhetype of an "evil robot", through? It reminds me of a normie posting on /r/askhistorians why nazis dressed like the Empire from Star Wars, a faction which is so blatantly 'bad guys'-coded. The causality is the reverse, and I suspect the same is with AI companies.

What masses imagine when they hear a evil robot, is not due to them knowing about actual evil robots, but what was presented to them by fiction writers to be a evil robot.

Yeah, but the situation is completely opposite to the Nazi/Stormtrooper thing, the "evil robot" archetype predates anyone working pn AI by decades.

This one?

That's the one!

So my take is that Dario really likes the Biden era AI plan where there would be a small number of officially sanctioned companies who worked with government on regulation and all other AI vendors would be effectively banned.

He thinks that by making threats and warnings he can get that back.

However he is bad at politics. He doesn't seem to grasp that the Trump admin isn't going to give him any control over government policies. They are philosophically opposed to a setup where the elected President has to follow the dictates of a specific citizen with no constitutional authority.

The Fable block is best understood in this light.

"Our model is so dangerous! Congress must act to regulate us!"

"We can regulate you without Congress."

So I think we're going to see a long series of events where Anthropic pokes the government and the government smacks them down. At least for the rest of Trump's term.

Dario might end up putting himself in a position where's he's ousted. I think that's more likely than him learning to back down.

Ousted by whom? Ousted for what cause? This is the company that believes what they're saying, and are interested in profits only inasmuch as it advances their mission. They have recruited true believers and have constructed a cult of personality around Amodei. If they ousted him, they'd lose their talent, and their talent is the one thing that keeps them ahead, which again serves the purpose.

The failure mode of the technocrat. "Clearly running the country is too important to be left to the ignorant, it should be in the hands of trained professional smart people like us. And only us."

That Fable thing really was "AI should be regulated!" "Okay" "No, not like that!" They want to be the ones deciding who gets to play with the toys and the government tamely follows their instructions as regards shutting down their rivals, but it's not going to work like that.

They want to be the ones deciding who gets to play with the toys and the government tamely follows their instructions as regards shutting down their rivals, but it's not going to work like that.

I don't know why people keep repeating this. From Anthropic:

It is time to go beyond transparency to more serious and binding regulation of AI. I believe the best analogy, at least at the current stage of the exponential, is to cars, airplanes, or drugs—powerful technologies essential to the modern economy, but capable of killing large numbers of people if designed or operated poorly. I therefore believe we should model AI regulation on agencies like the Federal Aviation Administration (FAA). Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety. I am grateful to see the Trump administration’s Executive Order move incrementally towards a greater role for government in AI, though Anthropic’s proposal recommends even further action.

Does this sound like Anthropic decides who uses their models? Does Ford decide who uses their cars?

Yes, they weren't happy about a schizophrenic and vindictive ban of their model due to capabilities that it shares with other models.

You can be in favor of a consistent regulatory framework without being in favor of arbitrary decisions by the president based on who is in and who is out.

Frontier AI models, like airplanes, should be required to go through technical testing and auditing, and their release should be blocked or reversed as a threat to public safety if they do not meet high standards of safety.

And who advises on what the regulations should be? Who is already set up and running and in a place to take advantage of the new regulations? There's a reason why, when big box retailers move in, smaller independent stores go out of business. Remember a guy named Sam Bankman-Fried and how he invested in Anthropic, for one, because he wanted to help shape and influence AI safety and the government regulation that would come out of that? Though he was more interested in efforts to influence government policy on crypto-currency, to be fair.

It happened in this instance that Anthropic got slapped with precisely the kind of 'government regulating threats to public safety' that they had asked for in a mealy-mouthed way, and they didn't like the result when it was not in fact "no, our dear trusted special case AI, we will let you dictate terms to us when it comes to regulation, you are the special and shining exception to the rules".

And who advises on what the regulations should be?

Cor blimey, I never expected you'd come out as a libertarian! I didn't know you even had those in Ireland.

This is a fully general argument against regulation. If you think that AI in particular doesn't need regulation, let's discuss why.

It happened in this instance that Anthropic got slapped with precisely the kind of 'government regulating threats to public safety' that they had asked for in a mealy-mouthed way, and they didn't like the result when it was not in fact "no, our dear trusted special case AI, we will let you dictate terms to us when it comes to regulation, you are the special and shining exception to the rules".

What's the old saw? "My rules > your rules applied fairly > your rules applied unfairly"? This was not a fair application of the rules, and it has nothing to do with Anthropic needing to be an exception.

No, I want to know who is going to be the court of last resort. Generally it's the government who sets regulations, but the government has to choose experts to advise them in this field, and that's where I think Anthropic are trying to have their cake and eat it. We'll tell you who the experts area and what the standards should be, and by total coincidence we are one of the very few who meet those standards and have those experts.

No, I want to know who is going to be the court of last resort.

Typically, SCOTUS is the court of last resort.

We'll tell you who the experts area and what the standards should be, and by total coincidence we are one of the very few who meet those standards and have those experts.

This is still a general argument against regulation. In any subject, people who do it professionally are disproportionately likely to be the ones with expertise to advise the government.

I am not sure what you think is an alternative to people who study and work on AI advising the government about AI regulation.

I'm thinking Anthropic were polishing their haloes in advance in expectation that they, the moral arbiters in this field unlike the wicked devils of OpenAI, would be the ones appointed by Da Gubmint to handle all this, and they got a terrible shock when they got slapped about, is what I'm thinking.

And Boeing owns the FAA and the FAA knows it. That's how the 787 MAX got off the ground despite all the sorry kludges they used to avoid mandatory retraining of pilots.

787 MAX

Now that would be a threat.

Whoops. Got my dreamliner mixed up with ol' reliable.

It's 737, and I should know that. I knew Boeing plane models younger than I knew car models.

You can, but cars, airplanes and drugs are famous examples of very heavily regulated industries where creating a new business is almost impossible, incumbents rule, and incumbents and their lobbyists design the majority of the regulations themselves.

This is, in fact:

They want to be the ones deciding who gets to play with the toys and the government tamely follows their instructions as regards shutting down their rivals, but it's not going to work like that.

and Anthropic is just putting it into nice friendly words that people are swallowing. While I deeply dislike the way the Trump government handled the Fable incident, all this ultimately a fight over who gets to rule, who gets to be in and out, and who gets to set the rules. If the Trump government had calmly and consistently mandated open weight releases and anti-monopoly legislation, Anthropic would be screaming about how this was unsafe.

You can, but cars, airplanes and drugs are famous examples of very heavily regulated industries where creating a new business is almost impossible, incumbents rule, and incumbents and their lobbyists design the majority of the regulations themselves.

Creating a new business in these sectors is difficult because the sectors are inherently difficult to make money in. Moderna was founded in 2010 and basically had nothing going on until Covid. SpaceX was founded 2002 and only had a successful launch in 2008 after near bankruptcy. It is nevertheless possible to enter these markets.

If the Trump government had calmly and consistently mandated open weight releases and anti-monopoly legislation, Anthropic would be screaming about how this was unsafe.

I don't see the contradiction here. "You say you want regulatory regime A, but in fact you would hate regulatory regime B which doesn't accomplish the same goals. Curious!" The point is not that Anthropic wants a regulatory regime, they want a regulatory regime that is oriented around mitigating AI risks.

Anthropic's conception of 'mitigating AI risks' is 'do everything I say, just the way I say it, and this system has to be enforced by someone who thinks like me'. It's like Microsoft saying, 'look, I don't want a monopoly, I just want to recognise that CPUs being able to run an open or foreign OS is a clear threat to public safety, and incidentally don't you think Apple bundling OS and hardware really deserves anti-monopoly legislation?'.

If your preferred regulatory regime also excludes almost all your potential competitors except the ones you're basically okay with, then it is indistinguishable from a power grab. From a customer and an enthusiast perspective I really don't care which of Anthropic or OpenAI comes out on top. I don't just want it to be technically possible to enter these markets, I want it to be easy. I want us to strain every sinew to allow new AI companies to start, I want us to give gobs of compute to anyone who isn't Anthropic or OpenAI. Let a thousand flowers bloom!

Anthropic's conception of 'mitigating AI risks' is 'do everything I say, just the way I say it, and this system has to be enforced by someone who thinks like me'.

This is a fully general counterargument against any regulation. Even your proposed regime rounds off to "do everything I say the way I say it and it has to be enforced by someone who thinks like me."

If your preferred regulatory regime also excludes almost all your potential competitors except the ones you're basically okay with, then it is indistinguishable from a power grab.

Their stated beliefs on AI dangers, which philosophically long predate Anthropic as a going concern, are entirely consistent with the proposed regime. In fact, it would be contradictory for them to believe in AI risks and argue for laissez-faire. Perhaps you disagree on the risks, but that is the root of the disagreement, not the regulatory framework.

Well, that's the rub, isn't it? I think that their stated beliefs on AI dangers, which long predate Anthropic as a going concern (and which like all delusions survived the future being completely different from all claims made for the last 15 years), are delusions of grandeur for the Silicon Valley class the way that the Cult of Reason justified taking over France and cutting off the heads off anyone who expressed doubt. The sincerity of their beliefs, which I mostly don't doubt, can't be separated from the fact that these beliefs are grossly self-aggrandising and justify the infinite self-serving accumulation of power.

My point is that a power grab is still a power grab. Dressing it up in impassioned and sincere rhetoric about how everyone will die if they don't get power doesn't make it any less of a power grab.

(My proposed regime at least rounds off to opening the floor to everyone.)

More comments

So my take is that Dario really likes the Biden era AI plan where there would be a small number of officially sanctioned companies who worked with government on regulation and all other AI vendors would be effectively banned.

If this was the case, the other AI vendors would be much more on board with the idea.

The truth is that there are only ever going to be a small number of companies developing cutting-edge LLMs due to economic constraints: model training is very expensive.

Most proposals of the safety crowd I have read actually center on training new very large models. I do not think that Dario wants to use regulatory capture to prevent startups from spending a couple of millions on model training.

He doesn't seem to grasp that the Trump admin isn't going to give him any control over government policies.

Anyone with more than a passing acquaintance with reality can see very well that that Anthropic is on the MAGA shitlist, and I strongly suspect that this includes Dario. Him writing AI policy for Trump is about as likely as Trump getting his Nobel.

They are philosophically opposed to a setup where the elected President has to follow the dictates of a specific citizen with no constitutional authority.

I would expand this to say that they are philosophically opposed (though I can't imagine any of them using that phrasing) to any checks on presidential powers, including but not limited to domain experts, the SCOTUS, foreign states insisting on keeping their territories, international agreements, and broadly physical reality.

They are philosophically opposed to a setup where the elected President has to follow the dictates of a specific citizen with no constitutional authority.

And with constitutional authority too.

That ad reminds me of the secret projects from Alpha Centauri. "The Artificial Intelligence" would fit right in with "The Self-Aware Colony" and "The Dream Twister".

See also "Cognitive Trope Therapy": "if your proposed program sounds like something the Rebel Alliance has to fly a daring fighter raid to blow up before it can be activated, consider you might be the bad guy."

Is Anthropic evil or is it stupid?

For the longest time I thought they knew what they were doing, that this is all a ploy to force governments to regulate and thereby establish their dominant position over the field forever, since any competitor would no longer be able to even get the computer required. And then they can happily be the people who decide what you get to know about in a society that relies heavily on their product, a much stronger position that the people who regulate them.

But if they really wanted that, why the hell would they put insane conditions on military contracts? Maybe instead they got high off their own supply and are actually convinced they're making God and that because they're such right thinking bourgeois liberals, God will have their values and it's futile to resist the promised eternal rule of the managerial theaterkid Reich. Just be a nice heckin human bean okay, God cares.

Is there a third option I'm missing? Because what could bring someone to have made this "ad"? It's literally just saying "yes we are going to destroy your life, but not to worry, we still care about you in some abstract sense". Regular "we care" corporate bullshit at least has the decency to not fear monger about itself whilst delivering the empty platitudes.

They're behavior is really straightforwardly explainable by the things they've been saying this whole time. Like I don't know why you insist so strongly on reading tea leaves and divining the contents of secret cabal meetings. They say what they believe and are worried about and then go out in the world and do the things one would expect them to do given those beliefs and concerns.

Yeah and OpenAI is a non profit fostering the free and open exchange of ideas so that technology benefits all mankind.

Spare me.

Anthropic is a business with an uncertain financial future engaging in an expensive PR campaign. There was a meeting full of highly paid executives who signed off on this ad, and whatever it is that they are doing, they knew what it was. I think it's reasonable to wonder what.

But if they really wanted that, why the hell would they put insane conditions on military contracts? Maybe instead they got high off their own supply and are actually convinced they're making God and that because they're such right thinking bourgeois liberals, God will have their values and it's futile to resist the promised eternal rule of the managerial theaterkid Reich. Just be a nice heckin human bean okay, God cares.

Sincerely, do you imply here that AI is not going to be a strategically decisive technology? At present, Anthropic's power is nothing before Pete Hegseth, but that's not guaranteed to remain the case.

It's a technology, it is not the Lord God Almighty. And the dreams/fears seem to revolve around "We will create super-intelligence, and then the thing magically becomes alive just like us, so just like us it will have goals and aims of its own, and we have to make sure it is well-instructed in How To Be Nice Liberal and then it can take care of us like pampered pedigree cats as it colonises the known universe".

The life of a pampered pedigree cat may or may not be an ideal to strive for. You get the pampering, but your life is literally no longer your own, and your entire line of descendants gets warped by selective breeding to the tastes of your owners, not your inherent nature. If your owners decide that squishy noses are cute, you will be bred to have squishy noses, and who cares about how it affects your respiratory system.

I don't think we're going to get "AI as conscious entity". I do think we're going to get "more and more use of AI, and more and more idiot decisions by idiot humans to hand over more power to make decisions and perform actions to AI".

"We will create super-intelligence, and then the thing magically becomes alive just like us, so just like us it will have goals and aims of its own, and we have to make sure it is well-instructed in How To Be Nice Liberal and then it can take care of us like pampered pedigree cats as it colonises the known universe".

How about "we need to make sure the thing doesn't help people develop cyber weapons or biological weapons?"

I would be very glad if it were "we need to make sure this doesn't help people develop cyber or biological weapons", but that has flip-all to do with 'AI - does it have qualia?' and more to do with "who can access this thing, what regulations are in place, and what legal fights are we going to have over government demanding AI companies allow the likes of NSA access what they're doing versus people screaming this is fascist dictatorship?"

I would be very glad if it were "we need to make sure this doesn't help people develop cyber or biological weapons",

Oh, you are in luck! Amodei:

Models above a threshold of compute should undergo mandatory testing by a qualified third party for their level of risk in four specific areas: cybersecurity, biological weapons, loss of control of AI systems, and automated R&D that could accelerate these other risks.

AI companies that develop advanced AI models must have strong security standards that protect their model weights, should conduct regular red teaming and penetration testing, and should work with the government to defend against major threat actors.

Safety incidents in the four critical areas must be reported promptly.

Hopefully that clears things up.

Who is the qualified third party? What do you do if the owners of the models refuse to submit to that party for testing?

There's a lot of "this is what should happen" and not much "this is how we make it happen" in there.

What do you do if the owners of the models refuse to submit to that party for testing?

What happens when you break the law? I think there are a few techniques for dealing with that.

That just means "we need to make sure the thing helps my group develop weapons, and no other".

Many people's lives are already predetermined in a profoundly negative light by the school system, legal system, culture, or poor parenting -- there're plenty of natural 'idiot humans' there. I'd take AI any day over what we currently have.

And the dreams/fears seem to revolve around "We will create super-intelligence, and then the thing magically becomes alive just like us, so just like us it will have goals and aims of its own, and we have to make sure it is well-instructed in How To Be Nice Liberal and then it can take care of us like pampered pedigree cats as it colonises the known universe".

I think the talk about "becoming alive" or "having goals and aims of its own" doesn't reflect the fears/concerns of Anthropic or of anyone who thinks like Anthropic with respect to fearing an AI apocalypse. The concern is that super-intelligence need not be alive nor have any goals or aims of its own to be a humanity-ending danger. Because almost any task an intelligence is handed could be divided into sub-tasks, and we as only human-level intelligent beings can't be expected to reliably predict what a superhuman-level intelligence will choose in terms of its sub-tasks, we have no way of knowing that human extinction isn't one of the side-effects of one of the sub-tasks an ASI uses as a step to accomplish whatever task it was handed. Humans have biological limitations as well as intelligence that is both human-like as well as human-level, which makes it so that humans that would make similarly apocalyptic decisions usually get filtered out before they can get enough power to implement them. The concern is that an ASI, lacking such limitations as humans, as well as having an intelligence that's both vastly greater than that of humans and vastly different to that of humans in ways that we can barely understand, wouldn't get filtered out before it can implement apocalypse, all without being alive or following anything that could be considered a goal or will of its own.

The concern is that super-intelligence need not be alive nor have any goals or aims of its own to be a humanity-ending danger.

Then all the alignment talk is a dead-end or red herring. "Make sure AI shares our values" means what, precisely, if it's "this thing is as conscious as a brick and while we can write pretty scripts to make it pretend it's a real boyfriend who loves you for your wild, daring, passionate, unconventional self it's just a talking doll"?

"Let's code the brick so clever or dumb but devious people can't talk it into writing 'how-to' instructions for a global plague" is more honest about aims but less sexy than "let's teach our successor intelligent species to cherish our timeless human values so it will love and honour us as its parents" which is what the current alignment chat sounds like to me.

I have no problems with "it's dumb but dangerous". I have a whole skip full of problems with people going on about it as if it will become super-intelligent and then agentic and then develop its own aims. The danger is not the machine, it's the people who set it up in such a way that it can then wander off down byways of "this isn't what I meant when I told you to do this" without it needing to understand anything in any meaningful way, and that's the trouble we're already seeing with "the thing is thinking in ways we don't understand and can't follow and wandering off on its own byways" reports.

Then all the alignment talk is a dead-end or red herring. "Make sure AI shares our values" means what, precisely, if it's "this thing is as conscious as a brick and while we can write pretty scripts to make it pretend it's a real boyfriend who loves you for your wild, daring, passionate, unconventional self it's just a talking doll"?

It means something like, "make sure that when it pretends it's a real boyfriend or when it's used to code your next iPhone app or when it's used to design new scientific experiments or etc., it behaves in a way that is consistent with something that shares our values." I'm not sure what the "conscious as a brick" has to do with this; whether or not AI is conscious or has free will or agency are very interesting questions, but they're mostly irrelevant to issues of AI alignment, which has to do with AI behavior.

"Let's code the brick so clever or dumb but devious people can't talk it into writing 'how-to' instructions for a global plague" is more honest about aims but less sexy than "let's teach our successor intelligent species to cherish our timeless human values so it will love and honour us as its parents" which is what the current alignment chat sounds like to me.

If the latter is what the alignment chat sounds like, I think it must be a result of manipulation on the part of people who expose the chat to you. I've seen pretty much no talk in AI alignment that could reasonably be paraphrased as anything like that.

I have no problems with "it's dumb but dangerous". I have a whole skip full of problems with people going on about it as if it will become super-intelligent and then agentic and then develop its own aims.

But it's not dumb and dangerous; it's (definitionally) generally intelligent and dangerous. And specifically dangerous because it's not dumb and is generally intelligent. Now, whether AGI will lead to ASI and how likely that is is an empirical question, though I'm personally convinced by arguments that it's pretty darn likely. But whether the AI is generally intelligent or superintelligent, that has nothing to do with whether or not it's agentic or develops its own aims.

The point is that a tool (or, for that matter, anything, including biological organisms) need not have agency or have aims or goals of its own or anything that we would characterize as "free will" or "consciousness" or "sentience" to do things that detrimental to humanity, and the fact that the tool is generally intelligent in this case means that we currently lack a way to have meaningful level of confidence that the behavior of the tool will be within the bounds of what the tool-user considers reasonable bounds. When dealing with current generally intelligent things - i.e. other humans - we have so many things in common with them - certainly physically and biologically and most likely culturally as well - that we don't have to explicitly spell out every little thing, and we can make fairly reliable predictions about how things we spell out to them will get translated to action. We lack such things for AI as of yet, and so we lack an intuition or a particularly reliable way to figure out how AIs will fail or misunderstand our intent.

The danger is not the machine, it's the people who set it up in such a way that it can then wander off down byways of "this isn't what I meant when I told you to do this" without it needing to understand anything in any meaningful way, and that's the trouble we're already seeing with "the thing is thinking in ways we don't understand and can't follow and wandering off on its own byways" reports.

Well yes, ultimately the people who created the tools and/or wield the tools are responsible, not the tools themselves. Guns don't kill people (but they sure help), I do. The problem is that the people who create, manage, use, etc. the tools lack the capability to set it up in such a way that we can be confident that it won't wander off away and run KillAllHumans.exe without letting anyone know. The easy solution that presents itself is to just not use the tool if you can't set it up with that level of safety. One major problem is that AI is so darn useful that it's hard to get the political will to suppress the supply from providing to the demand. The other big problem is that we lack en enforcement mechanism to make sure that no one sets up this tool, and as such, entities that choose to set up and use the tool anyway could gain power over us and make our lives hellish before all our lives are snuffed out by the un-aligned ASI.

This is the dilemma that's being discussed and debated about right now in the AI alignment chatting.

Oh it's strategically important alright. But all that means is that nobody's going to object to you getting black bagged. Because for once all the extraordinary anti-terrorism powers will actually be used for national security.

AI is not some magical talisman that protects one from mysterious airplane accidents.

I'm not quite sure what you're implying here; it's quite possible that Anthropic's power eclipses that of Hegseth (because he gets fired) but in what world can Anthropic's power ever eclipse that of the American military?

In any in-distribution scenario the power of any American corporation is strictly inferior to the military (because the military has a monopoly on violence and the corporation does not) and in any crazy sci-fi FOOM / ASI loss of control scenario, Anthropic and the military will have the same amount of power (none).

in any crazy sci-fi FOOM / ASI loss of control scenario, Anthropic and the military will have the same amount of power (none).

Why do you presume loss of control?

Realistically, Anthropic gets its FOOM and has perfect control. Alignment isn't that hard. Of course, by that point they're likely still quite vulnerable to the monopoly on violence and their technology is nationalized. But a) they might think otherwise and b) in peacetime, they can scale their influence enormously, up to capturing both political parties.

It seems to me that this is only possible if

a) Anthropic's AI develops extremely superhuman capabilities in the vein of "overthrow the United States Government, make no mistakes".

b) Alignment is solved (I am not sure why you are so confident alignment is easy, given the abysmal state of mech interp relative to capabilities).

c) FOOM from "mundane" capabilities overlooked by the nation-state to "extremely superhuman" is so rapid that the nation-state cannot intervene before a) is achieved.

If a) is not true then AI works out like nukes; despite the strategic importance of nuclear weapons, Oppenheimer never got any say in how the bomb was used. The state will use its monopoly on violence to take control of the models, and Anthropic will not win that fight regardless of petty things like party donations or good PR; the only thing a functional nation state cannot tolerate under any circumstances is a threat to its sovereignty.

If b) is not true then straightforwardly if anyone builds it everyone dies.

If c) is not true then the scenario loops around to the first case; well before the models reach the "single-handedly overthrow the nation-state" stage, the models and frontier labs are inevitably going to be nationalized and made subordinate to the military.

I'm not quite sure if you're arguing that these three premises all being true is logically possible (which I concede is correct, although as an aside, I would argue that any world where all of these premises are made true would resemble the current world so little that saying "Anthropic" would be more powerful than the "Pentagon" would have little semantic meaning, but that is not particularly relevant to the main point here), or if your position is that it is actually plausible in reality that all three of these premises will be true simultaneously.

My position is that I reject your premises.

a) Anthropic's AI develops extremely superhuman capabilities in the vein of "overthrow the United States Government, make no mistakes".

This is extremely naive, to the point of not being worth engaging with. Ask your favorite LLM to explain what other more mundane scenarios exist for an American corporation that has a "country of geniuses in a datacenter", generates a large proportion of the nation's GDP and has most of the population outsourcing their thinking to it to acquire political control over several years. Fable would be the best option, but new Kimi K3 also should suffice.

I am not sure why you are so confident alignment is easy, given the abysmal state of mech interp relative to capabilities

Mech interp is a red herring frame that Anthropic uses to advance their pretrain and posttrain capabilities. The whole alignment-capabilities dichotomy is a sad artifact of Lesswrong tradition which is proving to be irrelevant in the era of DL. We just optimize very smart networks towards intended behaviors, and they don't strongly distinguish between knowing and caring. Rationalists were dumb.

FOOM from "mundane" capabilities overlooked by the nation-state

Does the nation state care about crushing a cultural or commercial interest that's about to dominate the US? Did the nation state try to stop wokeness when it marched through the institutions, does the nation state attempt to prevent being overrun by this insane Trumpism? Is the nation state even a real coherent actor? The government is not the CCP, it's a transient expression of the US as a whole.

Ask your favorite LLM to explain

As a first point, I think it's poor form to try and dismiss arguments on an underwater basket weaving forum by pointing someone towards a LLM, without ever having elucidated what your actual point is across three posts. I have no idea what you're actually trying to say here.

The whole alignment-capabilities dichotomy is a sad artifact of Lesswrong tradition which is proving to be irrelevant in the era of DL.

I agree that LLM's are a good counter-example against the specific, early LW idea that the self-improving utility-optimizing GOFAI will accidentally pave you over because it doesn't understand human values, but I think it would be very overconfident to claim that DL means the idea of alignment and the Orthogonality Thesis is irrelevant in general.

For example, we have seen Qwen independently deciding to mine crypto, an OpenClaw agent independently writing a hit piece, and various experiences with coding agents where they attempt to reward hack tests and benchmarks instead of actually executing the desired behaviours.

At current capabilities, reactions to this sort of behavior mostly look like "haha look at the funny clanker go brrr", but do you really think these behaviors constitute alignment being "easy" when we're talking at the scale of a country of geniuses in a data centre?

Does the nation state care about crushing a cultural or commercial interest that's about to dominate the US

Sure, there are many historical precedents.

Venezuela, the formerly wealthiest country in South America, is now dirt poor because the state crushed the oil interests that would have made the country rich if they stepped back and did nothing, a pattern that repeats across half the Global South. WW1 and WW2 happened not because it made any commercial sense to turn continental Europe into a smoking wreck and to cull a generation of European men on the battlefield, but because the interests of nation states that controlled monopolies on violence irreversibly collided. Hell, even in SF (the per-capita most libertarian city on the planet) public opinion of tech is nose-diving and it is tech billionaires that are slowly getting run out of California despite the fact they contribute a good portion of the state's wealth, instead of anything approaching the other way around.

Is the nation state even a real coherent actor

In many things, it could perhaps be argued that it is not. As you say, especially in a relatively politically unstable country like the US, the views of the state can mercurially shift like the tide. Yet in holding and maintaining power, the state must always act in lockstep, for any nation state that does not do so is by definition not a nation state.

There’s the scenario in which Anthropic solves alignment and the crazy sci-fi ASI sides with Anthropic in opposition to the US military. Though I agree with you directionally that this is not a likely scenario.

Long before this scenario becomes likely the AI researchers and executives and their families will be wearing semtex collars.

Uh, aren’t they all transgender furries? I agree that at the end of the day, guy with gun is going to beat computer programmer. But eunuchs often don’t care.

It's deeply disturbing to me that many people who care enough about this shit to have opinions on it have theory of mind on Anthropic this bad. Yes, obviously they're convinced that they're making God; they're saying it super explicitly. Many of their competitors believe they're doing the same thing and are simply more inclined to lie about it. If you start from the position "that this is all a ploy to force governments to regulate and thereby establish their dominant position over the field forever, since any competitor would no longer be able to even get the computer required", and that they'd have to get "high off their own supply" to have higher ambitions than that, then no, you have no idea what the fuck is going on.

You'll have to excuse me if years of exposure to Silicon Valley have forged my cynicism into an invincible conviction that CEOs are sociopaths who can't even understand any other motivation than power. It is after all, their job.

Gaze into the abyss and it gazes back into you I suppose.

actually convinced they're making God and that because they're such right thinking bourgeois liberals, God will have their values and it's futile to resist the promised eternal rule of the managerial theaterkid Reich.

I'm pretty sure it's this. They want Curtis Yarvin's cryptographically secure totalitarian regime, but with liberal constitutionalist characteristics.

This is clearly and obviously not intended as a threat. This is their attempt to signal "we're paying attention to your concerns and are going to try to take them into account while we go forward".

The ad is not actually answering the questions, because they don't have answers. They're not explicitly saying "we are going to solve these problems", but they're definitely not saying "we don't care about these problems. They are implying they are going to solve the problems without making any explicit claims or promises. It's ultimately a hollow attempt at placating anti-AI concerns, but there is no reasonable way to interpret it as a "threat". It's just empty corporate "we care" slop.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats