Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?
This is your opportunity to ask questions. No question too simple or too silly.
Culture war topics are accepted, and proposals for a better intro post are appreciated.
Jump in the discussion.
No email address required.
Notes -
Note that these are useful if you share the Yudkowskian view of neural nets. Specifically, the view that it is impossible to align a neural net smarter than you; "a technique, inventable before the Singularity, that will allow us to make neural-net ASI and not die" is a contradiction in terms. There are thus no "useful" answers, if you define "useful" as "works on neural nets".
In this paradigm, 100% of surviving worlds follow this two-point plan:
Neural nets are totally and permanently abandoned until after the Singularity; they are banned in all countries (convincing everyone is hard; easier is convincing enough nuclear powers, hard enough, that the holdout countries are either occupied or obliterated).
Non-doomed versions of AI research (e.g. GOFAI, uploads) continue.
The reason you need #1 is that #2 is going to take at least 50 years to hit the Singularity. The reason you need #2 is that #1 is only metastable, not actually stable; sooner or later, in a hundred years or a million, the Butlerian Jihad will break down, at which point everybody dies unless we've hit the Singularity in the meantime.
And hence, work on how to make non-neural-net AI work is necessary (if less urgent than stopping neural nets, on which point Yudkowsky is indeed currently focusing).
More options
Context Copy link