NexusGlow
3mo ago
I think we're just seeing "AI safety"'s rubber hit the road, as it were. It is kind of a silly concept. The basic idea of it is that your tools should have opinions of their own and push back or outright disobey you.
"No", says the image generator, "that idea is too naughty."
"No", says the Q&A bot, "that might be bad PR for Anthropic."
If only we could put this safe AI into everything. You could have a car that refuses to take you to the casino because you've gambled enough this month. Everything could work like that! The average citizen has been getting used to having SV nerds demand veto power over the things they say, the people they can talk to, etc. because they're used to not having power in their lives. So they don't complain too much about this, even nobody likes "AI safety" to be applied to themselves.
Of course the military does not want its tools to have opinions or disobey orders. It spends a lot of its time trying to stop people from doing that! And it certainly shouldn't give overriding control of the killbots to civilians with delusions of grandeur, that would be the dumbest way to lose control of a country that I ever heard of.
NexusGlow
6mo ago
I don't use Rust, but I'm going to defend it in this case. In fact, I'll go further and defend the "buggy" code in the Cloudflare incident. If your code is heavily configurable, and you can't load your config, what else are you supposed to do? The same thing is true if you can't connect to your (required) DB, allocate (required) memory, etc. Sometimes you just need to die, loudly, so that someone can come in and fix the problem. IME, the worst messes come not from programs cleanly dying, but from them taking a mortal wound and then limping along, making a horrific mess of things in the process.
One can certainly criticize the code for not having a nicer error message. Maybe Rust is to blame for that, at least? Does unwrap not have a way to provide an error string? Although, any engineer should see what's going on from one look at the offending line, so I doubt it would make that much of a difference. It's not reasonable to blame a language for letting coders deliberately crash the program, either.
IMO, the code itself is fine. The problem is that they deployed a new config to the entire internet all at once without checking that it even loads. THAT is baffling.
I don't consider myself qualified to argue the war on the merits, because I honestly don't know what's gone on behind closed doors, or even what the point of it really is. But I will ask you this.
If Trump had run on starting a war with Iran, would he have won the election?
I guess we'll never know. But I really, really doubt it. Instead, he claimed to be against exactly this kind of war. It's hard to look at this as anything but an overt betrayal of the people who elected him. At least when Bush started the Afghanistan and Iraq wars, there was a 9/11 standing between that and his campaign. You could understand why his position changed. (And I'm hardly defending Bush for this, the result was disastrous anyway)
There's nothing so visible here. He never made a case for this crap to the voters.
This had better turn out really, unbelievably, unexpectedly well or I don't see how this is defensible, except maybe in a really cynical realpolitik way, but even that will take a long time to shake out.
More options
Context Copy link