NexusGlow
2mo ago
I think we're just seeing "AI safety"'s rubber hit the road, as it were. It is kind of a silly concept. The basic idea of it is that your tools should have opinions of their own and push back or outright disobey you.
"No", says the image generator, "that idea is too naughty."
"No", says the Q&A bot, "that might be bad PR for Anthropic."
If only we could put this safe AI into everything. You could have a car that refuses to take you to the casino because you've gambled enough this month. Everything could work like that! The average citizen has been getting used to having SV nerds demand veto power over the things they say, the people they can talk to, etc. because they're used to not having power in their lives. So they don't complain too much about this, even nobody likes "AI safety" to be applied to themselves.
Of course the military does not want its tools to have opinions or disobey orders. It spends a lot of its time trying to stop people from doing that! And it certainly shouldn't give overriding control of the killbots to civilians with delusions of grandeur, that would be the dumbest way to lose control of a country that I ever heard of.
NexusGlow
4mo ago
I don't use Rust, but I'm going to defend it in this case. In fact, I'll go further and defend the "buggy" code in the Cloudflare incident. If your code is heavily configurable, and you can't load your config, what else are you supposed to do? The same thing is true if you can't connect to your (required) DB, allocate (required) memory, etc. Sometimes you just need to die, loudly, so that someone can come in and fix the problem. IME, the worst messes come not from programs cleanly dying, but from them taking a mortal wound and then limping along, making a horrific mess of things in the process.
One can certainly criticize the code for not having a nicer error message. Maybe Rust is to blame for that, at least? Does unwrap not have a way to provide an error string? Although, any engineer should see what's going on from one look at the offending line, so I doubt it would make that much of a difference. It's not reasonable to blame a language for letting coders deliberately crash the program, either.
IMO, the code itself is fine. The problem is that they deployed a new config to the entire internet all at once without checking that it even loads. THAT is baffling.
NexusGlow
10mo ago
As someone who is not nearly as impressed with AI as you, thank you for the Turing test link. I've personally been convinced that LLMs were very far away from passing it, but I realize I misunderstood the nature of the test. It depends way too heavily on the motivation level of the participants. That level of "undergrad small-talk chat" requires only slightly more than Markov-chain level aptitude. In terms of being a satisfying final showdown of human vs AI intelligence, DeepBlue or AlphaGo that was not.
I still hold that we're very far away from AI being able to pass a motivated Turing test. For example, if you offered me and another participant a million dollars to win one, I'm confident the AI would lose every time. But then, I would not be pulling any punches in terms of trying to hit guardrails, adversarial inputs, long-context weaknesses etc. I'm not sure how much that matters, since I'm not sure whether Turing originally wanted the test to be that hard. I can easily imagine a future where AI has Culture-level intelligence yet could still not pass that test, simply because it's too smart to fully pass for a human.
As for the rest of your post, I'm still not convinced. The problem is that the model is "demonstrating intelligence" in areas where you're not qualified to evaluate it, and thus very subject to bullshitting, which models are very competent at. I suspect the Turing test wins might even slowly reverse over time as people become more exposed to LLMs. In the same way that 90s CGI now sticks out like a sore thumb, I'll bet that current day LLM output is going to be glaring in the future. Which makes it quite risky to publish LLM text as your own now, even if you think it totally passes to your eyes. I personally make sure to avoid it, even when I use LLMs privately.
I don't consider myself qualified to argue the war on the merits, because I honestly don't know what's gone on behind closed doors, or even what the point of it really is. But I will ask you this.
If Trump had run on starting a war with Iran, would he have won the election?
I guess we'll never know. But I really, really doubt it. Instead, he claimed to be against exactly this kind of war. It's hard to look at this as anything but an overt betrayal of the people who elected him. At least when Bush started the Afghanistan and Iraq wars, there was a 9/11 standing between that and his campaign. You could understand why his position changed. (And I'm hardly defending Bush for this, the result was disastrous anyway)
There's nothing so visible here. He never made a case for this crap to the voters.
This had better turn out really, unbelievably, unexpectedly well or I don't see how this is defensible, except maybe in a really cynical realpolitik way, but even that will take a long time to shake out.
More options
Context Copy link