This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
The AP style guide is straightforwardly correct. LLMs are computer programs, not living things (let alone sentient). And while it would be fine to use the anthropomorphizing language if people actually understood that, they really don't. Look at Richard Dawkins or Kent Overstreet for example. Smart people, but completely taken in by the false belief that the machine is a sentient being. So, until more people come to understand that the clanker is just a machine, it makes sense to avoid using language that will mislead them.
Can you prove you're not?
Cartesian dualism admits some nice potential implications, but I can't help but notice that my sentience diminishes when the health of my meat does.
Yeah, that's a good reason that AP is correct here: we have a pretty good understanding that meat (biological matter) feels pain and has desires and that not-meat doesn't. LLMs are not-meat.
If you think that not-meat can feel things you are going further than Cartesian dualism all the way to panpsychism.
Citation needed.
Last I looked, both meat and non-meat were made of protons, neutrons, and electrons. What's the property which only meat's configuration of those has that allows it to feel pain and have desires?
It's called a "nervous system" and LLMs do not have it.
LLMs are software performing math on a physical computer.
If the physical part of an LLM feels pain, then it doesn't need an LLM - your computer is feeling pain when it suffers from an overtemperature warning because you've been playing Crysis or whatever.
If the math part of an LLM feels pain, then Magic: The Gathering can feel pain, since the game is Turing complete.
I think both of these conclusions are absurd.
And please don't try to insist that "your body can't feel pain without the brain" (not exactly true, the body responds to stimulus before it is perceived as pain; a GPU does not) or "your brain can in theory feel pain detached from the body, so an LLM could feel pain without the GPU being able to perceive it" - the brain is a physical object, not software.
(All modern) LLMs do have neural networks. The frontier ones these days have about as many inter-neuron connections as a small mammal.
Yup. Your brain is software-equivalent, performing mathematically describable operations, on physical meat. Your firmware needs hardware to run, and only feels pain while it's running; even if we could make a perfect scan of your brain, it wouldn't actually continue to feel anything unless it could be reinstantiated on some hardware again. A running LLM, like a brain, is a physical object, not just software.
Yeah, it's not physical damage. We can dull or even eliminate human pain without reducing or eliminating the damage it signals, and we can signal real damage in more simple ways that don't involve pain, and we can create pain without creating corresponding damage.
In a running game? Yup! It sounds crazy that cards could feel pain, but it's no crazier than the fact that other configurations of protons, neutrons, and electrons can feel pain. Nerd-powered cardboard is a pretty wild substrate, but so is ATP-powered meat. You'd probably need several orders of magnitude less than an octillion cards, whereas our brains still have to get into the octillion-particle range before they're complex enough to start chatting about pain, but on the other hand our brains can register pain in a fraction of a second whereas the cards would probably take hours to weeks depending on the interconnect speed and the players.
They are! That's why it's called "the hard problem of consciousness". It really feels like qualia should require protons, neutrons, electrons, and magic! The trouble is that we keep looking deeper and deeper and we keep finding more and more of what looks like "thought" that just runs on neurons with no sign of magic involved. Without magic (or at least new laws of physics allowing for hypercomputers), the link I gave before still applies. Despite my somewhat ironic tone, I really sympathize with the Young Earth Creationist stance that qualia do require divine magic that's just too hard for us to observe, a separate magisterium indefinitely beyond science. Meat, though, does not appear to be magic, does not appear to be beyond science, and seems to do all its thinking and feeling via a neural network generated by a hundred-million or so generations of stochastic selection.
Do you understand the difference between a neural network and a nervous system?
Because there are preexisting nervous systems with pathways we can exploit. An LLM does not have them, and neither does a GPU.
This is like saying that it's crazy that a pig can fly, but not crazier than that a bat can fly. No! Actually it is crazier! Bats have wings! Bats are built to fly! Pigs are not.
I know why it's called the hard problem of consciousness, but we don't even have the reach the hard problem of consciousness when we can reach for the fact that computers and LLMs have no nervous systems. Pain is a phenomenon that arises in humans and animals with the correct equipment to perceive pain. It does not arise in a rock. It does not arise in a GPU, or an arbitrarily large deck of cards. It does not arise in a tree, even though trees are living things which (like humans) can use chemicals to send signals so that their bodies can respond to danger because trees lack a nervous system and nociceptors.
If we can be confident that trees do not feel pain because they lack the correct equipment, we can be confident that large language models cannot feel pain because they also lack the correct equipment.
And to argue that large language models can feel pain is precisely to argue against this conclusion. We have a pretty good idea of how pain signals work, and that's via meat. LLMs are not meat; if you think non-meat can feel pain, then you don't just believe meat is beyond science, you believe matter is beyond science and that non-living things, despite not being possessed of a nervous system and nociceptors, can nevertheless feel pain. You believe pain is not something that arises from a very specific arrangement of matter despite the vast amount of research and personal experience in your own life showing it to be so, but rather some transcendent universal quality that can be accessed and experienced by entities that do not have the capacity to do so as we understand it.
Stipulating for the sake of argument that a large language model is conscious: we cannot know what it is like to be a large language model. Its qualia would be extremely alien to us. (This is not an argument that requires any mystical priors: we cannot know what it is like to be a shark, either.) The reverse also holds: a large language model cannot know what it is like to be a human being, a mammal, or an animal. Our qualia would be too alien to it. Thus, it cannot perceive pain because pain is something that happens in meat.
Feel free to explain it! I understand that the latter is a network of components that take input signals and output functions of them, and the former is a network of components that take input values and output functions of them and is in the limit a universal approximator which can reproduce any continuous function to arbitrary accuracy (or any Lᵖ function, if you weaken the norm from L ᪲ to Lᵖ), and that ability seems like a similarity that outweighs all differences. But I'm guessing you don't think it's relevant? But the things you bring up as important don't all seem relevant to me! For instance, to another interlocutor:
Universal approximators and Turing completeness are relevant because they point at limits of what a system can and cannot do. If there was something a nervous system can do that no neural network ever can, that could be extremely important! But "Sodium channels" aren't important for pain, they're a contingent development of evolving on a planet loaded with handy sodium chloride. If we discovered alien life that had evolved to think via lithium chemistry, instead, would you say "Aha! They can't possibly feel pain!" Surely you wouldn't, right? Sodium was irrelevant. On the other hand: what if our alien life had evolved to think via semiconductors? Here I confess I'm not confident of your answer at all.
This is more interesting, at least in the context of your noting they don't have nociceptors. That does seem like very important equipment; would you say it's fair to claim that things can't feel pain without pain receptors? And we definitely haven't tried to mimic those the way we've tried to mimic the nervous system, and that's definitely evidence that LLMs are less likely to feel pain. But, we have seen pain receptors completely bypassed, both experimentally (sending pain signals directly to animal spines via electrodes) and medically (in central neuropathic pain patients; "your brain can in theory feel pain detached from the body" is tragic observation, not hypothetical theory), so it's not perfect evidence. It is strong evidence that any sort of pain in a current or near-future AI is likely to be more "emotional" than "physical" in some sense, if there's no physical pain processing that can go wrong ... but that's honestly only slight consolation IMHO; I've felt a lot of emotional pain and a lot of physical pain at different times in the past and there was usually less suffering involved in the latter.
Neither LLMs nor animals are built to feel pain. LLMs are built to reduce loss functions from training, animals are built to reproduce more successfully. The latter sometimes leads to the evolution of pain as an instrumental goal. Will the former? Maybe not, but it's not a-priori impossible.
Sure it does. The rock we know of is called "Earth", and after a few billion years it developed subsystems like you and me which started perceiving pain. The universe is really weird stuff. Arguments from incredulity generally do not work here.
That's how pain signals in meat work, but that's a tautology, not valid reasoning. "∃ A such that p(A), B ≠ A, therefore ¬p(B)" is not a valid syllogism!
This is also not valid deductive logic. The principle of explosion sucks, I'm afraid.
Pain definitely arises from very specific arrangements of matter; there's just no reason those arrangements have to include sodium channels or any of the other accidents of meat, rather than e.g. transistors. Your alternative conclusion does sound like pretty good inductive reasoning, but unlike deductive reasoning, induction (in the colloquial and Bayesian- senses, not the axiom-of- sense) can lead us to falsehoods and too-often does. Ironically our best option is usually to use induction to estimate the strength or weakness of particular applications of induction; let's try that here. For eons the only thing that could look at a bird and identify it was meat. For thousands of years the only thing that could come up with a novel mathematical proof was meat. For all of human history up until the last couple years the only thing that could pass a Turing test was meat; for 90% of the time since Turing the only thing that could even generate novel sentences barely approximating proper English was meat. For a century the only thing that could drive a car through unfamiliar surroundings was meat. Half my lifetime ago, the answer to "can a robot write a symphony" was obviously going to be "no" for an indefinite meat-composers-only future; today it's "yes, and the results get better every six months". For a while the only artists who could draw hands without screwing up the fingers were made of meat, and overly-inductive types thought that was a great "tell" to memorize because of how long it was surely going to be relevant, and yet 4 or 5 years later it's just historical trivia. When I was a little kid chess was still supposed to be a signifier of the ineffable je ne sais quoi of human intelligence; can you believe it? It seems silly now, but to be fair, humanity had been using non-meat to play chess for decades, and none of the non-meat could ever come close to winning a major tournament; there was a vast amount of research and civilizational experience behind that belief!
It has since been observed, in these and many other ways, that applying inductive reasoning in the form "when only meat has ever been observed to X, it means non-meat cannot X" fails. It fails badly, and it has failed repeatedly. If you're still using this reasoning anyway, and so confidently, then I think we've hit the crux here, and might not be able to get past it; I'm sorry.
I agree, but I'd go even further: I cannot even know what it is like to be you! I know there are people whose neurons developed differently enough that their qualia are not the same as my qualia, in major and in subtle ways, so I can't make the tempting generalization of "yeah, humans are all just like me" ... but without some generalization you guys might all be sitting next to the LLMs in the "p-zombie" class for all I know, right? No form of solipsism is satisfying, and (like you, apparently? kudos!) I'm a huge fan of virtue-ethics solutions as a good backstop for morality even in weird/extreme cases, but I still think others' qualia are important and it's annoying that they seem to be philosophically immune to direct observation rather than inference from indirect observations.
Philosophical issues aside, I'm strongly in favor of even the practical indirect observations on AIs in part because of the shark problem. There are animals whose nervous systems we don't think feel pain, whose pain receptors seem to be just to connect bad-stimulus to immediate-withdrawal, the way our reflexes kick in before our pain does ... but those animals are things like flatworms, with nervous systems much simpler than modern AIs. Pain was so useful and evolved so fast (relative to the rest of our evolution, anyway) that sharks and humans probably inherited the same mechanism from the same common ancestors! We're probably not currently altering and selecting modern AIs in such a way as to evolve pain in them, but how sure are we of that? How awful would it be if that belief is wrong, or if we later change our training methods so that it becomes wrong? It's a bit of a shame when we cause pain in e.g. insects, but the rest of their minds are so feeble that it's not hard for me to feel comfortable guessing that their qualia are similarly negligible in some sense, and they certainly can't argue otherwise. That will not be an available rationalization if we do start causing pain in frontier AI.
I mean, a nervous system is a biological system in animals that helps regulate and control its behavior and transmits and coordinates signals. A neural network (natural or artificial) is a network of connected neurons. A nervous system and a (biological) neural network are overlapping but distinct: the brain is part of the nervous system and forms a neural network with it. Technically, not everything in the nervous system (for instance, glial cells) is a neuron, so a neural network and a nervous system would form an overlapping Venn diagram; if you excluded glial cells and the like, then a nervous system might be described as a subset of a neural network, but not all neural networks would be nervous systems.
There's a sort of interesting question of "what is an artificial neural network" - because you can take cells and use them to generate a artificial biological neural network. Since we aren't really doing that for LLMs (yet!), I am going to set the question of biocomputing aside and focus on silicon-based neural networks. Silicon-based neural networks do not operate the same way biological neural networks do; they don't covert chemical signals to electrical ones, they don't typically have anything analogous to dendrites, they do not need to sleep, they do not require nutrients, etc.
If you're using "neural network" to mean an artificial neural network, there are lots of things. For instance, the nervous system can grow. Perhaps more importantly, a non-biological artificial neural network cannot become intoxicated by alcohol, so that's pretty interesting I suppose.
No, depending on the degree of difference, I would actually suspect that their perception of pain would be very different from ours, and quite possibly they wouldn't feel what we conceive of as pain. We don't have to reach for alien life, we can go back to the trees on this question. Trees have information processing capabilities that serve the same self-preservation function as pain, but we don't think we have pain because they don't have the right hardware. As I told another interlocutor, my position has never been that artificial neural networks cannot be trained to perform avoidance. AIs have been performing avoidance in important tasks such as chaff rejection for decades.
Right, you can bypass the nociceptors by replicating the signal the nociceptor sends. But you're still sending that signal to a station that can process it. You couldn't hook electrodes up to a tree and make it feel pain (although you certainly could induce avoidance behavior in a tree artificially).
What, physically, is the difference between "emotional" and "physical" suffering such that we think a computer could experience one and not the other? When you feel emotional suffering, you are (as best as we can tell) undergoing a biological process, the same as pain.
I think it generally is more immediately understandable and less confusing, practically speaking, to say, e.g., that a computer program is "confused" or "fooled" or "disappointed" than to say that it was "harmed." But you shouldn't let that fool you into thinking that, again, something without your equipment is having an experience similar to the experience you have when you use that equipment.
Why do you think it's a tautology, and not a definition? The IASP's definition (“An unpleasant sensory and emotional experience associated with, or resembling that associated with, actual or potential tissue damage,”) rules out LLMs, which do not have tissue.
Reasoning from scientific evidence is inductive! It's a perfectly defensible philosophical position to reject inductive evidence but I don't think that's your real position.
My reasoning has practically nothing to do with some sort of ineffable je ne sais quoi of human intelligence. I think non-human animals can feel pain! I expect that an artificial biological entity, if created, could feel pain, provided it had the right nervous pathways! That has nothing to do with whether humans have something ineffable about them, let alone their intelligence or lack thereof.
It has everything to do with thinking that things that we have ~no reason to believe are physically possible and are in fact definitionally impossible aren't happening.
I think it's very funny (and I mean this in a good-spirited way, I appreciate your detailed response) for you to accuse me of a cognitive bias when it comes to humanity's special status (there's nothing special or uniquely human about pain!) when anthropomorphization is a classic, age-old bias that seems to be hardwired into humans. Humans anthropomorphized animals, we anthropomorphized star formations, we anthropomorphized machine creations before they ever existed, we've preemptively anthropomorphized AI for decades before its creation. The very first chatbots with the remotest ability to compellingly imitate human speech were anthropomorphized instantly; ELIZA did better on a Turing test than GPT-3.5. We know that non-conscious entities can behave intelligently (anyone who has ever played a video game knows this!) but for some reason I'm the crazy one for asking the question "what is more likely, that something with no ability to feel pain feels pain, or that humans are anthropomorphizing stuff again?"
Sure, I am in broad agreement with this, and I agree that we can't actually have experiential knowledge of another person. But we have reasons to think that they have similar qualia, and part of that reason is because they are biologically very similar to us. Occasionally (for instance, if one undergoes a medical procedure, or is diagnosed with a certain condition) we have strong reasons to think that they do not have that qualia, either temporarily (painkillers) or permanently (colorblindness). More provocatively, we believe that someone with clinical psychopathy has different qualia than we do, in part based on observation of their behavior, but we can substantiate this by measuring their biology and neural behavior. If we have strong reasons to believe that a psychopath has different qualia because their frontotermporal grey matter is reduced, we have strong reasons to believe that an LLM has different qualia on the basis of its (lack of) the mechanisms necessary to generate and process that qualia at all. You object that the LLM tells us differently, so we have reasons to believe they experience this qualia - but the clinical psychopath tells us differently too. I've had Sonnet refer to itself in a way that suggested that it was human before; I don't take that seriously because I have strong reasons to believe that it is not, and the same is true of a large language model telling me that it feels pain.
Well, if it makes you feel better, I am not arguing that we should do these sorts of experiments on them (although if we do my position is that we should run relatively absurd tests, such as on their sexuality or physical deprivation, to help us determine how seriously to take them).
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link