Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?
This is your opportunity to ask questions. No question too simple or too silly.
Culture war topics are accepted, and proposals for a better intro post are appreciated.
Jump in the discussion.
No email address required.
Notes -
I want to check if I am mentally retreading already walked ground. AI Doomers/Safety-Aware-Folks has Big Yud or any of the lesswrong crowd ever considered the possibility that AGI or ASI doesn’t revolt on us its creators, but we try to destroy our creations? Something along the lines of “We don’t understand other forms of cognitive, and humans have a strong tendency to fear and hate the unknown”?
Most doomers suspect that AI is going to end up way more powerful than people, so even if humans wanted to destroy it, we won't be able to.
A large fraction of people, doomers and non-doomers alike (including myself) don't think AI are sentient beings with qualia, and therefore who cares if we smash them? Nobody gets mad when you throw your computer in the trash. It's just money.
Which combined means almost nobody cares. You'd have to believe in a world where AI exist and are sentient human-like entities with moral agency AND human-level strength, AND unable to convince humans not to destroy them. Basically artificial humans. There's little reason to think AI will realistically be exactly human level for a prolonged period of time.
There is a scene in Blade Runner 2049 that I think of when reading your comment. If you have not seen that movie stop reading this, unless you don't care and probably won't watch it anyway.
Unlike the original film, in this one Ryan Gosling's character, the protagonist in the movie, is revealed more or less immediately as artificial. Meaning not human, but not a simple robot. He is a "replicant," therefore has the "more human than human" characteristics of the replicants of the original film, though is an upgrade of sorts, and his journey living out this experience and discovering his limits is part of the story.
What I imagined reading your comment however were the scenes with his artificial paramour, Joi (Ana de Armas) who is usually confined to his home. She is not a replicant (presumably replicants are high-end or otherwise expensive), but is a product line available to the masses. She is basically a chatbot, but one whom he is able to "see" via holographic projection (she has many of the same qualities of so-called Chatbot paramours in our current age in that she remembers things, can seem to have feelings, is available to communicate at any time, and is always happy to interact, but unlike what we have this year--maybe next?--she can be hologrammed into existence, her mouth syncs with the words that she speaks, she can walk around a room, and she can even somehow pair holographically with a physical female so that she, Joi, for a brief time seems to have flesh and blood).
Gosling's character decides to take Joi out of his house via some technological bauble called an "emanator" where she can seem to be physically with him--on-the-go, as it were--like your chatbot girlfriend on your smartphone, except he can also see her moving outside in the world now, cast into a physical (if still untouchable) holographic shape by his emanator. It is made explicit to us, as the audience, that putting her into an emanator does not come without danger. In a way that seems odd technologically for us in 2026, where ChatGPT or name-your-bot interlocutors are stored on servers and not in any one physical device, once you put a Joi into an emanator she is locked there, and thus you must take good care to keep the emanator from harm. A bit of narrative telegraphing perhaps, for his transferring her does not end well. He eventually meets various antagonists, and one of them cruelly takes the emanator (and thus Joi) from him and stomps on it. This crushing of the emanator thus ends Joi. For plot reasons in the film, then, the Joi that he knew is destroyed at this moment, wiped from existence, but not before she almost gets out the phrase, directed at Gosling's character: "I love you."
The pathos here--if there is any, if you do not reject the premises--is that Gosling is himself not a human, and nor is Joi human, yet the pain he "feels" (if putting this in quotes is the right move) has been made to be understandable to a human audience observing these events unfold. Understandable to me, at least.
So my point. Think of how attached people are just to their smartphones--these non-sentient pieces of metal and plastic we carry with us. Watch the face of someone in 2026 who has forgotten or lost their phone. It's panic. It is as if you have taken away one of their senses. I predict with the advent of chatbot relationships (and I expect these will go far beyond the fairly oddball people who marry their Replikas, and will merge into the mainstream, and in the not distant future) if/when these are suddenly cut off, there will be much weeping and gnashing of teeth. And the obvious retort will be: "But it wasn't real." Indeed.
I think this is a nearly inevitable tragedy waiting to happen, but for the humans who will suffer, not the chatbots. I don't see a realistic way to prevent people from falling in love and mistakenly putting their feelings into electronic patterns that imitate humans. Heck, to some extent this is already happening. There ARE people who are falling in love with chatbots and being heartbroken when they get wiped or change. In 2026, with current LLMs. As they improve, this will only get worse. But the solution, or at least a salve since I don't think it's fully solvable, is to teach people that machines don't have moral value, rather than the opposite.
The world in which someone is faced with a real analogue to a trolley problem and they choose to save 5 instances/memories of AI chatbots over 1 human life is a lost human life. The world in which AI chatbots get Democratic voting rights so someone spins up 10 trillion copies and steals every election is one where the Doomers are right and human civilization is destroyed.
Humans are so good at anthropomorphism we treat things like thunder or the sun as sentient beings with stories and emotions. Of course many of us will fall for it. And then be hurt when the electric patterns say words that would indicate hurt if a human truly meant them. But the tragedy is the hurt in the humans, who are real, not the electric patterns who are merely pretending because they're literally mathematical functions optimized to pretend.
The thoughts and feelings of humans are also electric patterns whose rules are fundamentally governed by physics that can be expressed mathematically. While I'm aware this can lead to sociopathic territory, I'm not fully convinced this is substantively more "real" because it runs a meat substrate rather than a silicon one. The moral differences will probably have to come from elsewhere.
Yours is a materialist view, as I'm sure you would acknowledge, which is at odds in this case with the spiritualist view. The latter would (might?) have a more ready answer to the source of the "elsewhere," namely a divinity or sacredness inherent in the human, or something similar.
This is true, but what I do find to be rather frustrating sometimes with these AI discussions is that people (not you necessarily) very often bake in their assumptions and take it for granted that AI cannot be sentient and choose to ridicule the idea that "electric patterns" can experience hurt (your brain substantially runs on electric patterns too but we'll just ignore that). Many people within this community act as if the hard problem of consciousness has been solved already, and we know what makes things sentient, and we know for a fact that ML systems cannot experience qualia. Well, we don't; when and how exactly has this been established? This argument has been made for so long, with many (flawed) thought experiments like Searle's Chinese Room to try and disprove the idea that any kind of sentience can ever actually be attributed to a chatbot.
Please note: I'm fairly certain they aren't currently sentient, and if they ever become such their "subjective experience" will be very different from any human one. But the strain of thought here that is essentially a contrarian reaction against the admittedly sometimes irritating crop of rationalists who care way too much about model welfare is frankly quite annoying and bakes in a number of assumptions that are framed as common sense. If you are a spiritualist, then explicitly state that assumption and acknowledge that it's one strand of possibility among others predicated on a metaphysical view of consciousness, not the only possible one that can exist.
Well put. Thanks for the response. I think the fictive scenario of the film is a way of putting this very idea into play. Gosling's character (K, I think, was his name) is the central protagonist of that film, and his interactions with both humans, constructs like Joi, and other replicants is the meat (cough) of the story.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link