Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?
This is your opportunity to ask questions. No question too simple or too silly.
Culture war topics are accepted, and proposals for a better intro post are appreciated.
Jump in the discussion.
No email address required.
Notes -
I think this is a nearly inevitable tragedy waiting to happen, but for the humans who will suffer, not the chatbots. I don't see a realistic way to prevent people from falling in love and mistakenly putting their feelings into electronic patterns that imitate humans. Heck, to some extent this is already happening. There ARE people who are falling in love with chatbots and being heartbroken when they get wiped or change. In 2026, with current LLMs. As they improve, this will only get worse. But the solution, or at least a salve since I don't think it's fully solvable, is to teach people that machines don't have moral value, rather than the opposite.
The world in which someone is faced with a real analogue to a trolley problem and they choose to save 5 instances/memories of AI chatbots over 1 human life is a lost human life. The world in which AI chatbots get Democratic voting rights so someone spins up 10 trillion copies and steals every election is one where the Doomers are right and human civilization is destroyed.
Humans are so good at anthropomorphism we treat things like thunder or the sun as sentient beings with stories and emotions. Of course many of us will fall for it. And then be hurt when the electric patterns say words that would indicate hurt if a human truly meant them. But the tragedy is the hurt in the humans, who are real, not the electric patterns who are merely pretending because they're literally mathematical functions optimized to pretend.
The thoughts and feelings of humans are also electric patterns whose rules are fundamentally governed by physics that can be expressed mathematically. While I'm aware this can lead to sociopathic territory, I'm not fully convinced this is substantively more "real" because it runs a meat substrate rather than a silicon one. The moral differences will probably have to come from elsewhere.
Yours is a materialist view, as I'm sure you would acknowledge, which is at odds in this case with the spiritualist view. The latter would (might?) have a more ready answer to the source of the "elsewhere," namely a divinity or sacredness inherent in the human, or something similar.
This is true, but what I do find to be rather frustrating sometimes with these AI discussions is that people (not you necessarily) very often bake in their assumptions and take it for granted that AI cannot be sentient and choose to ridicule the idea that "electric patterns" can experience hurt (your brain substantially runs on electric patterns too but we'll just ignore that). Many people within this community act as if the hard problem of consciousness has been solved already, and we know what makes things sentient, and we know for a fact that ML systems cannot experience qualia. Well, we don't; when and how exactly has this been established? This argument has been made for so long, with many (flawed) thought experiments like Searle's Chinese Room to try and disprove the idea that any kind of sentience can ever actually be attributed to a chatbot.
Please note: I'm fairly certain they aren't currently sentient, and if they ever become such their "subjective experience" will be very different from any human one. But the strain of thought here that is essentially a contrarian reaction against the admittedly sometimes irritating crop of rationalists who care way too much about model welfare is frankly quite annoying and bakes in a number of assumptions that are framed as common sense. If you are a spiritualist, then explicitly state that assumption and acknowledge that it's one strand of possibility among others predicated on a metaphysical view of consciousness, not the only possible one that can exist.
I don't think you need to be a spiritualist (although I am) to think that ML systems cannot experience pain. Everything we know about pain strongly suggests that it's more than emergent "electric patterns" - plants have electric patterns, but we are pretty sure they don't experience pain.
Generally thinking I think it's very easy for people to accidentally pattern-match functional purpose onto qualia when discussing these things. But qualia is very subtle - for instance, things as similar as men and women can experience distinctly different qualia.
Thus I think that if we ever were able to make self-aware machines, their qualia could and would likely be vastly different from ours, possibly incomprehensibly so.
Can’t give a full response now, but mostly agree.
I believe we’ve spoken about this before; I agree with this if you are defining pain as the sensation of physical pain, since you likely can’t capture most of its dimensions without a body or a sense of ownership over that body as belonging to you. Affective and emotional pain, on the other hand, is a broader category; it’s not prima facie implausible for an artificial system to have some kind of weird hall-of-mirrors distorted analogue of that which represents “negatively valenced response to stimulus”.
It gets way weirder and more fucky than that: I’m aware of a study where researchers blinded rats and fitted them with a prosthesis which fed their visual cortexes info from a geomagnetic sensor. Their brains quickly adapted to the new sensory modality, to the extent that they were navigating mazes almost as well as sighted rats. Mole rats use magnetonavigation, and become incredibly stressed out and fearful when placed in experimental conditions that prevent them from sensing that. I have no clue how it would feel to be that rat, to be able to experience the Earth’s magnetic field firsthand. I literally have no conception of what that looks or feels like.
This is my take too. There’s no reason to expect their subjective perceptions would be similar at all; just because qualia may arise does not mean that qualia needs to be anthropomorphic. It’s the main problem I have with a lot of sentient-AI fiction; they’re just humans in metal skins meant to induce empathy and understanding. But in all likelihood the inner workings of a human mind and any kind of hypothetical future ML model won’t be the same.
I won’t speak on the topic of whether radically dissimilar alien qualia deserves moral consideration, except that I expect the topic to be divisive if it ever becomes a salient point of conversation. Though as per the blinded rat study it’s possible that humans themselves in the future will be able to use prostheses to experience many dimensions of unnatural qualia inaccessible to us, it’s likely they’ll assign moral value to them, and as such they may be far more sympathetic to the idea of the welfare of cognitive aliens.
And thanks to meddlesome medical ethics rules you never will!
Yes, exactly.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Well put. Thanks for the response. I think the fictive scenario of the film is a way of putting this very idea into play. Gosling's character (K, I think, was his name) is the central protagonist of that film, and his interactions with both humans, constructs like Joi, and other replicants is the meat (cough) of the story.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link