This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Yeah, anti "AI works for coding" person in the top level 2-3 below this one, how do you explain all this? Note that they are providing cryptographic hashes of claimed vulnerabilities today, so we'll see within the next few weeks what these vulnerabilities actually are and if they're trivial we'll all know. Finding a 27y old vulnerability in FreeBSD is up there next level skillz.
Also @self_made_human, you're a regulated doctor, you're one of the least cooked people out there, you'll be protected by laws and regulations long after the rest of us are on the dole.
I'm not skeptical about every single aspect of AI, my main skepticism is over its ability to build and maintain complex systems (usually in the form of codebases that are more than a basic bitch CRUD app). Finding vulnerabilities is definitely something I've always thought was within the capabilities of AI, my biggest concern is the signal to noise ratio. So I'm curious how many false positives Mythos found that they had to filter through to find the 4 examples they list as ones it actually found.
There's a cost aspect as well. If it costs $200,000 to find a glitch in a video codec that may, horror of horrors, cause your player to crash (and which, to anyone's knowledge, hasn't done so in 16 years), that's not exactly a selling point. $200,000 may actually be an understatement; they said it took 5 million tries to catch it. At 20 cents an attempt, more like a million dollars. We also don't know if they ran any of these tests on old code with known bugs. If they did and the software didn't catch half of the ones that were already caught, its utility isn't that great.
I wish the AI skeptics would limit themselves to forms of naysaying that aren't contradicted by the press release!
That's not what they said. They said five million runs of existing automated testing tools (fuzzers) didn't catch it.
They explicitly mention their hit rate by severity versus opus:
More options
Context Copy link
Operating system and browser zero-days go for millions of dollars.
If Mythos can spit these out for a million dollars a run it's still extremely scary.
This is only true in the darkest of gray markets. In the white-hat arena that Anthropic would be forced to bargain in, these exploits go for 10s of thousands.
The military is of course willing to pay black-market rates, but Anthropic kinda burnt that bridge... and I'd be honestly pretty surprised if In-Q-Tel (famous CIA front company) starts investing in Anthropic...
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Per the quote, it was OpenBSD, which is an operating system with a very strong focus on security. (By reputation, I am not paranoid enough to run their OS personally. I do run their ssh server, like everyone does, and have no complaints except for that one Debian 'fix', and I can't blame Theo et al for that.)
Ok, that's even more impressive than finding a vulnerability in FreeBSD.
OpenBSD still finds a buffer overflow every year or two. It's definitely better than 95% of big software projects out there, but it isn't perfect. Definitely not trying to minimize what Mythos actually found though.
https://www.openbsd.org/errata74.html
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
They are possibly very cooked, because AI is a lot better at day to day doctor tasks than it is at mathematical STEM, and telehealth is already on the rise. Economic disruption as severe as „every tech worker on the dole“ will result in political shake-ups, like universal healthcare in the USA and huge cuts to unpopular physician monopolies, resulting in huge salary declines for doctors.
Best I can tell the LLMs have basically found use as a "force multipliers" for skilled workers to expand their productivity, especially in finance and tech. This news exhibit an extension into searching a solution space with later verification by skilled workers. I'm sure the use cases will continue to expand but medicine is fundamentally different - you'd be looking at replacing a skilled worker for purposes of replacement (obviously) and unlike other cases were someone verifies, in a replacing doctors scenario you'd need to be getting it right 100% of the time with no second check. In medicine the checking would be the same as doing the work.
More options
Context Copy link
"Least cooked" and "very cooked" are significantly overlapping distributions. Pretty much everyone who isn't ready to FIRE or is independently wealthy (and maybe politically connected) is potentially cooked. And that's assuming aligned AGI, or else you better hope you make for a particularly pretty paperclip.
Anyway, as that joke goes, we're all dying, some of us are just dying faster.
More options
Context Copy link
More options
Context Copy link
So I hope, but it's far from granted while I work for the NHS. Rishi Sunak threatened to cut costs and put uppity doctors in their place by augmenting mid-levels with AI a few years back, and was laughed at. Even I don't think the models of the time would have been good enough. But times have changed, while the NHS and its only becomes a more tempting target for financial bariatric surgery (and the models have gotten much better). Starmer probably won't be the one to make the call, given his politics, but desperate times call for desperate measures.
I'm confident it'll happen eventually, and far too soon for comfort. The average man, the kind staring at double digit unemployment figures or laid off themselves, would have pointed questions about why doctors and other regulated professions are let off the hook. I think it only buys me like 2-5 additional years of security at best.
And in India? Haha. Sadder haha. It's going to be a bloodbath and the service sector is not going to have a good time. The economy it props up? You connect the dots.
More options
Context Copy link
More options
Context Copy link