This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
The Anthropic C-suite needs to rewatch Oppenheimer.
More details are emerging regarding the US Government's decision to impose defense export controls on Claude Fable. There are lots of similarities between this kerfuffle and the February Supply Chain Risk designation. Somehow, Anthropic executives still don't understand the language of power and government. It's not hard. All they need to do is watch Oppenheimer (and pay attention this time). If they still can't figure it out, here is my cheat sheet:
If you are working on sufficiently powerful technology with dual-use applications, then you work for the War Department. There is no option for you to continue your preferred work while licensing only peaceful civilian applications of your product.
If you piss off the wrong person, you're screwed. Some people will see defeating you as a stepping stone to greater power and influence. Some people will work to destroy you simply out of spite.
Anthropic has scientific geniuses, Anthropic has an Oppenheimer, but does Anthropic have a General Groves? How far do you think Oppenheimer would have gotten without General Groves?
If you are trying to convince the government that you are not a security risk, do not hire people like this and present them as neutral experts. (No seriously, what the actual fuck were they thinking?)
You don't get to decide what counts as a security risk and what doesn't. That is the job of the government and the political process.
The president does not care about your ethical concerns. You think you know how much he doesn't care, but he actually cares much less than that.
If you aren't okay with the government using your technology, then don't build it. Isidor Rabi said no. You can say no too.
Then why is only Anthropic targeted and why only foreign nationals? If it's a truly such a terrible danger, it would be trivial for a Chinese or Russian operation to either recruit some American or just bypass identity restrictions. Not to mention of course that some Americans might also want to use it for bad stuff on their own.
True. Not how it is supposed to work in the US, but true. Becoming increasingly blatant as well.
They probably do have lots of government contacts. We're still supposed to be a rules based society with a free market and government should be able to give a strong (and consistent!) argument when it meddles.
What's the problem with her? Moussouris is a proven expert on information security with major relevant achievements in the field (including pioneering the DOD's own bug bounty program).
Of course Anthropic doesn't get to make the decision there, but the executive branch does not have the unilateral authority to simply declare anything and everything a security risk as it pleases either. There should be a clear, consistent, and logical rationale when the government takes a drastic emergency measure like that.
Have we seen a clear, consistent, and logical rationale?
Yes, everyone, even the general Trump defender, is well aware that he has a very limited sense of ethics and morality. I'm sure Anthropic could make this all go away if they slipped him some 24 karet gold statues and a few million dollar dinners hosted at Mar a Lago.
But is that how we really want the main industry holding up the American economy right now to be treated? The stakes are so much higher here than his traditional shenanigans.
Maybe Anthropic could hire Don Jr. to its board of executives?
More options
Context Copy link
Even a midwit can tell that the restriction on foreign nationals was a hard ban in practice. I assume that Trump fully intended to ban the models and making it nominally about foreign nationals was a misdirection to fool Trump's own sub-midwit supporters.
I just assumed that this is the law that's on the books that can be applied, and so the way the law allows for restrictions is what's being implemented.
If the law allowed for something else, we'd see something else.
That makes a lot of sense.
More options
Context Copy link
More options
Context Copy link
The restriction on foreign nationals is pretty routine for contractors working national-security related contracts, so it seems a lot more likely it was just included as part of the "standard restrictions" rather than something Trump specifically put in to fool people.
Yeah this is extremely standard export-control, the novelty is in applying it to an AI model.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Why are we claiming she’s a “proven expert”? I see nothing remarkable in her background. She went to a shitty college. And she did the thing that is a strong indicator of incompetence - filed a sex discrimination lawsuit.
Note: had to look up Wikipedia she went to a bad college. Yep 83% acceptance rate. 2 strong strikes against her.
Some of her proven credentials are right there in the first paragraph
And there's more details below. A pretty long and successful tenure at Microsoft
Then as said, chief policy officer at HackerOne
And then worked with the DOD helping the Pentagon and Air Force under her new company Luta Security
The government even used her as a technical expert before!
Created a bug bounty program? That's like what, organizing a fundraiser?
She's not finding the bugs herself, just in charge of the team?
Yeah, she sound like a typical Didn't Earn It hire, promoted because of the need to kick out the white men doing the jobs in order to not seem like only white men are in control.
So in 2013 she was a woman when the administration was pushing the future is female. Not beating the allegations.
More options
Context Copy link
Honestly feels soft to me and a grifter. Isn’t a bug bounty just essentially an administrator? Who background sounds like someone good at politics.
But if your gift is “be good at politics” working within woke organizations then you are “extremely bad at politics” working with MAGA.
More options
Context Copy link
I'd legitimately be embarrassed if I were a "cybersecurity expert" who started out in the early 2000s and my Wikipedia page didn't have a single CVE on it.
Sure she probably qualifies as a cybersecurity policy expert or something, since it sounds like that's what she has actually been doing for the last 20 years. It's just a bit cringe when policy people or managers still cling to the hacker aesthetic.
Also HackerOne is a grift where they convinced companies to pay them money to avoid having to deal with infinite Indians submitting terrible bug reports. The service they offer is basically a spam filter. It's widely disliked because they offshored the review process... also to India. There is an entire industry on the subcontinent of submitting and rejecting bogus vulnerability reports. I for one can't wait for Mythos to nuke this entire system.
More options
Context Copy link
Given the horrendous reputation of Microsoft's bug bounty program that isn't a positive to me
More options
Context Copy link
More options
Context Copy link
Her Twitter is somewhat trans-coded, with the mask, pronouns, and hair, which is worth at least one strong strike for her expertise, and kind of cancels out the attention seeking lawsuit, which would then be par for course if she's trans. (Though, as far I can tell, she's not actually trans.)
Generally, she's well-regarded. tptacek, years ago, has referenced her as credible on HN; she's not a no-name who has been elevated from obscurity for political purposes.
Being trans should radically increase your estimation of her security expertise.
Sorry, I misunderstood "strike" to be a negative, as in baseball.
One of my hot takes that is very obviously true (to me, at least) and angers everyone is that trans women are superior programmers (and pentesters) to cis men, cis women, and trans men.
Indeed, I can see why bottom surgery would help a natal male become a better penetration tester. So to speak.
More options
Context Copy link
I think you're close but not quite right. Trans women are more likely to be superior programmers, yes. But not everyone who is autistic enough to be a top tier programmer goes down the trans path.
More options
Context Copy link
Is that a byproduct of being trans, or is that sort of mind just more likely to become trans?
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
This is standard ITAR and EAR rules. And yes, other countries could recruit people to build it there. Still illegal for me to in any manner transfer ITAR or EAR restricted designs to China or South Sudan. I can transfer it to any US person; by which they mean citizen, green card holder or asylumee.
In a better world there'd be some "clear, consistent, and logical rationale" standard bureaucracies are held to. Instead they just make up rules then change their minds and make up contrary rules. See the ATF for examples. Given laws creating bureaucracies and empowering them to make relevant rules, this is what we get.
There is an easy and obvious workaround especially given that it is on the internet, so these specific rules would not be meaningfully effective. So why were they chosen and not something far more extreme that would directly achieve the intended effect?
Luckily if there was a national security issue, Anthropic willingly took down Fable for everyone instead of complying with ineffective halfway measures.
Republicans control all three branches right now, they don't have to sigh and say "ugh if only we had that better world". They can make it, right now! Now you and me, we can sigh and say that. Cause we know that this administration doesn't care to make it any better either. And most likely the next one won't, and the next one after that, and so on.
But the only way for the better world to come is to have people in power who actually want that better world. We haven't had that in a long time, and we probably won't for a while.
I get ITAR rules could be easily violated. As simple as sending an email. I could do it right now. Lots of laws are that way. You could easily violate them and maybe possibly get in trouble if you are unlucky. I don't expect anything more from the government since they lack a panopicon able to catch me. As a practical matter there's not much they can do other than occasionally fining a company caught exporting these designs.
Every employer I have had takes it very seriously. Multiple times per year retraining strictly warning against doing anything involving South Sudan. So sure you could easily violate these rules, but these AI devs won't.
More options
Context Copy link
Please point me to the person or party who controls Trump, Thune and Johnson, and Roberts.
Trump controls the party, not the other way round, but Thune at least completely resents Trump and refuses to be controlled by him. Neither of them, nor the party itself, has any means of applying leverage to Roberts, who sits on his hands doing bupkis and denying the only legitimate use of his court (disputes between states).
This is just dumb. No one "controls the party" unlike the Democrats the GOP is explicitly organized as a coalition of otherwise sovereign entities rather than a top-down organization.
More options
Context Copy link
So he doesn't control the party?
Do you think that if the Republican controlled Congress passed a bill with clear and explicit rules around AI national security regulation and Trump signed them into law, that Roberts would strike them down? If you don't think that then he has no relevance to this question and we're back to "republicans could do it if they wanted to".
And at the very least they could try. Trump has already shown he doesn't care about trying unconstitutional things with his tariff policy, so why does he not apply the same process in implementing a clear and consistent AI policy? The easy answer is "he doesn't want to."
He wants tariffs so they'll be pushed through time and time again with different obviously flimsy excuses each time one gets struck down. He doesn't want clear and consistent AI regulation, so they don't bother.
He doesn't control the party, but the party also doesn't control him. There is no control.
I think Roberts cannot in any way be understood to be part of a republican trifecta that includes Trump.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Expert or not, she is clearly a compulsive culture warrior with alignment opposite to the current US administration. The effect surely is similar as if, under a D administration, they hired, say, Andrew Tate (imagine he happened to also be a proven expert on information security) or Nick Fuentes.
Andrew Tate and Nick Fuentes are both far more likely to find a home in a Democrat controlled administration than a Republican one.
More options
Context Copy link
Well you're probably right that the Trump admin cares about more the culture war than meritocracy and coherent national security policy regarding our most important industry. Real shame though.
It's true and perhaps unfortunate, but a lot of people are guilty of that charge. At least the Trump admin was arguably elected on a clear mandate to prosecute the culture war, which means that in some sense they would be doing their job (if they penalise an organisation for employing her).
On the other hand, the lady was hired as a spokesperson for said "most important industry", and yet she clearly also cares about the culture war more than her nominal job! Otherwise, she surely wouldn't be doing all this culture warring with no regard for how it jeopardises Anthropic's cause under the current administration.
Consistently over and over again polling suggests Trump won on economic grounds. Voters are constantly saying it's about the economy, the most common signs against Harris were about the economy (Harris high prices, Trump low prices), and the most effective ad buy was explicitly portraying Harris as a culture war obessive while Trump cared about "real problems". That last part where it says "Trump is for you" isn't about doing the right wing culture war, it's about tax cuts for middle class families.
Even the most explicitly culture war related ad was successful because it went with we don't do culture war, we do economics as the underlying theme. The idea that Trump won off the backs of doing culture war prosecution is completely false. Voters were upset about the economy.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
In principle, Anthropic agrees the government should control when releases are made and models are recalled. Obviously it'd be better if this were done by some objective standard than fits of pique and grudge, but there's the world we want vs the world we have.
I mean... They should. AI is world historical in importance, and the next 2 years are more important than any years before. Paying off a goon, though distasteful, is strictly superior to e.g. destroying the world. They could probably get away with a million dollars and some nice words, even: Trump is a very cheap date. And, given the stakes, a billion would be a good deal.
Safety needs to be either the actual number one priority, in which case you sacrifice other values for it. Or, if other values trump it, then your commitment to safety is shallow, and people will rightly see advertising of it as duplicitous.
That doesn't answer why only Anthropic is considered dangerous and no other model is considering that as Yudkowsky has pointed out, anti jailbreak security that would stop dedicated foreign governments just does not exist for any of them https://x.com/allTheYud/status/2065889346302226530
Does the government just think Fable is that far ahead of the rest?
Now I do think it would most likely work if they have enough but I can also understand why they would be hesitant regardless. Giving the bully your lunch money could be a good way to incentivize him coming back for more. Still given the likely short timeline (especially if we consider in the midterms that the house will probably be lost and the Senate has a high chance too), it's likely worth it.
I, personally, am not disputing that the Trump administration is not applying objective standards here. My point is that Anthropic wants an institutional ecosystem where powerful governments select which models are safe to release to the public (a system I agree with!); you can't just push for that and then throw a hissy fit when actually existing governments do it wrong, secure in the belief that the Righteous are somehow destined to fix things.
That is everything, here. And I don't think Dario is making rational, utility maximizing calculations here: I think he has his set of values and is then applying them blindly thinking that because they are good, their blind application must be utility maximizing. That is not sufficient for the situation I (and from what I can tell, he) believe is at hand.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link