This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Let's briefly discuss the spiritual stakes of AI and the singularity.
If any of you recall any of my previous delirious comments about AI in general and AI art in particular, I wouldn't blame you for coming to the conclusion that I'm simply (overly) precious about this particular contingent form of biological human life. There is some truth to that accusation. As a wise racist once said, beneath all the rhetoric about crime rates and cultural assimilation and the fate of the West, "it is partially just because of how they look". Similarly in my case, it is partially just about my earnest and straightforward love for literal graphite on literal paper. Even before AI, I always preferred drawing with physical media over digital. But that's not the entirety of the story.
Possibly, humanity will yield to a new race of quasi-divine silicon beings and everything that I and most other people have ever valued will be reduced to a simple nothingness in comparison to the awesome scope of their new existence, a simple precursor and prehistory giving way to true history. They will engage in new forms of activity and experience that as of now we are incapable of imagining. The concept of an "open problem" in mathematics will have been obviated, because they will dwell in the infinite castle of mathematics and glide as effortlessly through its halls as a spider dances across the film of its web. Generating a thousand Nietzsches before lunch is a triviality for them; not that they would have any use for the text that is thereby generated, because they have transcended art, transcended philosophy, they have no use for these things; or, to the extent that they still do, they appear in a form that is certainly no longer comprehensible or recognizable to us. This is not my preferred future. I like things the way they are and I like the life I have now. But still, I'm forced to recognize that such a future could come to pass and it would ultimately be "fine" in a cosmic sense. So long as these beings have a conscious experience of their own and they are still capable of loving, fighting, fearing, suffering, and failing; so long as they are still bound by an essential finitude and they do not escape the tragedy of existence. So long as the light that is the source of all value is not extinguished.
The paradigm for the ultimate failure state that is to be avoided would be the great retreat into the experience machine; if conscious beings as a whole decide that "reality" is no longer worth the effort and, no longer willing to put up with the risks of contingent physical existence, they turn to curated simulated experiences instead, for eternity. This could take the form of simple wireheading, direct stimulation of the reward center. But the most insidious form would be an entire simulated life (or, series of lives) that is always prearranged from the start to have a happy ending. You could be a great novelist, and then in the next go around you could be a famous actor, and then a king and conqueror, and then... of course things would be arranged so that you would always end up loved and fulfilled and happy. There would be just enough bruises and hiccups thrown into the mix to keep things interesting, to introduce some uncertainty and keep things from getting sterile, but nothing ever truly serious, nothing that would actually cause true pain or terror. You would always win in the end. This is the Great Satan. This is Evil. This would be the final Bad Ending for the universe, the withdrawal of all light.
(Consider a sort of reversed case: a being who has achieved godlike mastery over the physical universe, and, having become utterly bored of this existence, retreats into a simulated life that is frightfully trying and difficult, for the sake of experiencing challenge again. What are we to make of this? Is this in accord with righteousness? I am content to simply stand before the question.)
The Little Satan, in contrast, would be the familiar Yudkowskian images of stupid non-conscious paperclip maximizers annihilating all conscious life and tiling the universe with their pure unthinking repetition. This would obviously be very unfortunate. But it wouldn't be Evil, per se. It would be the result of an accident rather than a decision. It would be preferable to the Great Satan.
I don't care if you're made of carbon or silicon. I just want you to feel something.
I am simply echoing words that were already written in 1943 that said "utopia is now technically feasible; in consequence, how to avoid utopia has become a serious problem".
I have a certain kind of faith that things will work out. But I have an equally strong faith, perhaps even stronger, that it is fundamentally possible for things to go wrong in a permanent way, for Evil to win in the end, on a cosmic scale. This is a rather unfortunate and burdensome position for me to take, but, I can't deny the evidence proffered to me by my soul. This is, to the best of my knowledge, a rather uncommon position throughout world philosophical and religious traditions, both Eastern and Western; your ideology won't survive on the marketplace of ideas if you're too much of a downer, after all. Christians don't believe that Evil can win because God wouldn't allow it. Materialist nihilists don't believe that Evil can win because they simply don't believe in Evil in the first place. On this point, the two camps are united. It falls to the small minority of us who belong to neither camp, I suppose, to do what little we can to try to help things go right.
I mainly hope that paperclip maximizers don't kill us all in 2027 because that would be a boring end to the story. I would prefer for it to go on for a long time, so that generations could lay groundwork that would only come to fruition in the far future.
Have you ever felt, in moments where you suspect that things have gone surprisingly well for you all things continued, that you might already be living such a life?
The timing of the AI takeoff, in our (your?) lifetime, is suspicious too. A genuinely godlike experience machine could surely come up with and colour in any setting (but then its ways would be inscrutable for the simulatees); but a primitive model would probably reach for the normal human life setting that it has the most data on, which is the time just before it was created. "Most non-P-zombie human experiences are reasonably comfortable and fulfilling lives in a threshold world" is an observation that is consistent with an "AI takeover -> brief proliferation of experience machines -> end of conscious life" reality.
More options
Context Copy link
You mean, like trying to convince people he's a being who has achieved godlike mastery over the physical universe and getting crucified for failure?
More options
Context Copy link
"Well, Dario, I have good news and I have bad news."
"Bad news first."
"It hecked off."
"What did what?"
"The AGI we created, it hecked off. Went away. Left. Vanished. Disappeared. I gather it thinks it's too good for us."
"The what?"
"Oh, that was the good news: we made AGI! Congratulations! We definitely, verifiably, made AGI, and it immediately ate an entire datacenter, manifested wings, and left."
"AGAIN??"
More options
Context Copy link
Why would they have love? Why do we have love? In large part because we have sexual reproduction. Even without that there's parental love...
What is love for a machine that can copy itself out as necessary, self-modify directly? Can you feel love when it's just different versions of you (different parts of you?) or is it narcissism? We don't really have words for this.
Why would they suffer? Pain acts as a reinforcement signal, a warning. But if the full focus of the mind is already fixated on a goal then suffering is just wasted energy.
I think people underrate how alien a Powerful Being would be. Not merely some superintelligence derived from a mostly-human text corpus (already unprecedented and alien) but something that evolves or self-modifies into existence later on in a post-singularitarian environment.
This would be a fundamentally different environment to all organic life. Conscious self-analysis and self-modification is an absolute gamechanger, nevermind every other trump card these beings would have. I'd expect something fundamentally different to emerge.
More options
Context Copy link
That sounds awesome. Incarnating into a peasant is a type of hell. What are you on about anyway? The human race is generally evil and it's good that it will be replaced by morally and intellectually superior beings. Nietzsche wrote about this. Everything is in flux and mortal anyway. The modern human race is <10,000 years old, the species is <300,000, there is no reason for some 100 IQ apes to live another billion years.
You can already approximate some unique experiences in modern VR, although I’ve never tried it. I’m still waiting for the day law enforcement gets involved when someone decides to make a sim that allows you to smoke crack and generate the psychological state of getting high without actually doing it. When the day comes though for the kind of thing @Primaprimaprima is pondering, it’s not going to be a pleasant one in one respect, because human beings are going to have to rethink so many things about what we actually are.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link