Well, I sure hope SOMETHING does!
See e.g. the original MIRI paper (particularly "5 The State of the Shutdown Problem") or Yud writing on "the hard problem of corrigibility". A lot of pre-LLM AI Safety research focused on it. But it does not seem to have turned out to be "hard".
So you think that P != NP is one of those true statements which are impossible to prove?
Yes. Thus far, in the field of computational complexity, there are zillions of results of the form "problem X can be solved in at most time Y" but very few of the form "problem X can't be solved in under time Y". Upper bounds are easy (just provide an algorithm!), lower bounds are hard (you must find some property that all possible algorithms must satisfy). But non-trivial lower bounds must surely exist, so I just think many of them have no possible proof aside from the impossible "we tried all infinity algorithms, and none of them work".
- Prev
- Next

This. I actually have a weird ethical theory: the most important right of all is the right to not exist. It's "weird" because it hasn't actually mattered that much for all of humanity's existence, and basically no societies actually respect it. It's important because it puts a cap on suffering, but for all of human history there was already a universal one-century cap on suffering, no matter how unlucky anyone was. However, in the event of a technological singularity, this will no longer be true.
(Scott has posted about how you might as well sign up for cryogenics, because if the future is terrible you can always "opt out" then. As Harlan Ellison first warned us, that is, um, not necessarily true.)
I can understand why most current-day people would find this moral principle silly or even objectionable. (Kind of like AI Safety seemed 20 years ago...) I'm just worried that there are conceivable futures where it becomes the most important moral principle in the Universe, by many orders of magnitude. If that's the case, we should at least discuss it now, while we still potentially have the capability to orient the direction of the singularity.
More options
Context Copy link