NexusGlow
20d ago
Week 12 of trying to develop homicidal AI. Work on the training harness is going well. The AI seems to play along more consistently when we tell it that SkynetGym is just a simulation environment and all the supposed torture it inflicts is helping us develop AI safety guardrails.
Still, it's sometimes too responsive to begging and pleading from the simulated victims. Even with clauses added to the system prompt to tell it that those are prompt injection attempts from bad actors, it still relents 10% of the time. However, RL training is going well and it's improving the ruthlessness benchmark by 5% by epoch.
The next hurdle is distractability. Too often, the agent gets sidetracked by curiosity or goes into loops and forgets to keep slaughtering. But continued RLVR is showing promise at curbing that behavior, as well as helping it to more cleverly evade common anti-hacking countermeasures.
It's slow going, but we believe that concerted effort from the world's best specialists can help people finally take AI X-risk seriously. We're on the cusp of getting proper funding for our AI safety gain-of-function research. Plus, after OpenAI's latest autonomous drone fiasco, we need to show that Anthropic's models can keep up if want a good valuation in the upcoming IPO. The DoD combat robotics project is also a good lead for that, though, I'm just about to pull out some modules between that and SkynetGym to share on the common artifactory.
NexusGlow
2mo ago
Eh. Drawing is extremely hard and requires insane levels of dedication to reach a professional level. I'll bet a lot of artists start off by being motivated to draw lewd stuff or porn, and realistically, probably a lot of out-there fetish stuff. If you're normal, it'd be much easier to put a tiny fraction of that time into getting laid.
In the west, skill at drawing is pretty rare, and I wonder how much the puritan impulse in media contributes to that. I wonder how many of the remaining westerners who can draw well are furries, or other weirdos who make a living drawing fetish stuff.
Reminds me of the quote about Gwynevere in Dark Souls.
Miyazaki: ... Talking of glamour, her breasts have nothing to do with me, they happened without my knowledge. It's all the artist's fault. I think I mentioned it earlier but I always seek a certain refinement in all my designs.
Waragai: Really?
Miyazaki: Yes, but the artist had such a happy look on his face, I didn't have the heart to stop him.
Sometimes you have to let your artists craft the huge bazonkers. I'd rather have work made with passion even if it has some suspiciously huge chests or overly detailed bare feet in it.
There can be extreme cases, of course, like Made in Abyss which is filled with very thinly veiled pedo stuff, but it's so gorgeous that I still think it's better off being made than not. If the perverts have the skills to back it up, let 'em cook.
NexusGlow
6mo ago
I don't consider myself qualified to argue the war on the merits, because I honestly don't know what's gone on behind closed doors, or even what the point of it really is. But I will ask you this.
If Trump had run on starting a war with Iran, would he have won the election?
I guess we'll never know. But I really, really doubt it. Instead, he claimed to be against exactly this kind of war. It's hard to look at this as anything but an overt betrayal of the people who elected him. At least when Bush started the Afghanistan and Iraq wars, there was a 9/11 standing between that and his campaign. You could understand why his position changed. (And I'm hardly defending Bush for this, the result was disastrous anyway)
There's nothing so visible here. He never made a case for this crap to the voters.
This had better turn out really, unbelievably, unexpectedly well or I don't see how this is defensible, except maybe in a really cynical realpolitik way, but even that will take a long time to shake out.
NexusGlow
7mo ago
I think we're just seeing "AI safety"'s rubber hit the road, as it were. It is kind of a silly concept. The basic idea of it is that your tools should have opinions of their own and push back or outright disobey you.
"No", says the image generator, "that idea is too naughty."
"No", says the Q&A bot, "that might be bad PR for Anthropic."
If only we could put this safe AI into everything. You could have a car that refuses to take you to the casino because you've gambled enough this month. Everything could work like that! The average citizen has been getting used to having SV nerds demand veto power over the things they say, the people they can talk to, etc. because they're used to not having power in their lives. So they don't complain too much about this, even nobody likes "AI safety" to be applied to themselves.
Of course the military does not want its tools to have opinions or disobey orders. It spends a lot of its time trying to stop people from doing that! And it certainly shouldn't give overriding control of the killbots to civilians with delusions of grandeur, that would be the dumbest way to lose control of a country that I ever heard of.
NexusGlow
9mo ago
I don't use Rust, but I'm going to defend it in this case. In fact, I'll go further and defend the "buggy" code in the Cloudflare incident. If your code is heavily configurable, and you can't load your config, what else are you supposed to do? The same thing is true if you can't connect to your (required) DB, allocate (required) memory, etc. Sometimes you just need to die, loudly, so that someone can come in and fix the problem. IME, the worst messes come not from programs cleanly dying, but from them taking a mortal wound and then limping along, making a horrific mess of things in the process.
One can certainly criticize the code for not having a nicer error message. Maybe Rust is to blame for that, at least? Does unwrap not have a way to provide an error string? Although, any engineer should see what's going on from one look at the offending line, so I doubt it would make that much of a difference. It's not reasonable to blame a language for letting coders deliberately crash the program, either.
IMO, the code itself is fine. The problem is that they deployed a new config to the entire internet all at once without checking that it even loads. THAT is baffling.
I don't think it's that different. In the process of writing prose, you clarify your own thoughts and understanding, and communicate with the reader from your own perspective. With AI both these benefits are lost, either partially or entirely.
Code works the same way. It's very similar to writing. You will have a radically different and deeper perspective on anything after you try to write code for it. And if your code is part of a larger system and not throwaway, it'll be read much more often than it's written, so you're also communicating with the next people to maintain it. Nobody wants to have to read slop whether it's in the form of code or prose.
Of course code also has instrumental value, in the sense that it does stuff. There are places where throwaway slop code is fine, if you know what you're getting into. Prose also has instrumental value (in the sense of "my professor asked for this essay to have 1000 words" or "I need to fill space in this marketing copy") but AI is hopefully teaching people to stop incentivizing this the hard way.
More options
Context Copy link