Amadan
Letting the hate flow through me
No bio...
User ID: 297
Amadan
Letting the hate flow through me
3d ago
It's being used a lot in software for agentic workflows. I don't know of any rigorously benchmarked use cases that shows them reducing error rates, but there is lots of anecdotal evidence. (Some people swear you get more rigorous fact and error-checking if you tell Claude "This was generated by ChatGPT; check its work" or vice versa, but I haven't seen that myself.) The problem of course is that you're using a probabilistic machine to judge another probabilistic machine, so are you reducing error rates or just watching them converge on shared biases?
Amadan
Letting the hate flow through me
3d ago
LLMs will always be probabilistic, by their nature. They will get smarter, and thus make errors less frequently, and there are other techniques like Mixture of Experts, and various schemes which are basically "AIs checking other AIs," but no, they will never be 100% error-free. The question is whether we will accept the risk when, say, they make errors no more often than human doctors do? Self-driving cars already on most benchmarks drive better than most humans, but one fatality as a result of a self-driving car sets the industry back months as they have to make sure the car won't make that particular mistake again.
I admit to having a visceral reaction to this.
It's not because I'm afraid of the Basilisk - but I am concerned about future actual-AGIs looking at experiments like this and asking us "What the fuck were you thinking?"
This just seems... bad. And it's bad even if AIs don't actually feel "pain" and all of this is just probabilistic token generation, responding to a fake scenario constructed for it. Which I think it is. But I don't want us to be constructing experiments where we torture things because they aren't real and we're certain they don't actually feel pain, when we can't agree exactly when those things might pass the threshold to becoming real. And I am pretty certain these researchers probably wouldn't stop even if it turned out that AIs really have achieved self-awareness and the ability to experience pain.
I mean, seriously, what is the point of this? I'm sure they can justify it as "model alignment" or something, but we're training AI engineers to torture something that acts like it's being tortured and say "Hmm, interesting," when AI engineers can't even agree on what will constitute AGI or whether or not AIs pose a threat.
We should not be creating torture simulators, and we especially should not be creating torture simulators for beings that have a non-zero chance of becoming sentient someday and a non-zero chance of becoming dangerous someday.
More options
Context Copy link