This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
The AP style guide is straightforwardly correct. LLMs are computer programs, not living things (let alone sentient). And while it would be fine to use the anthropomorphizing language if people actually understood that, they really don't. Look at Richard Dawkins or Kent Overstreet for example. Smart people, but completely taken in by the false belief that the machine is a sentient being. So, until more people come to understand that the clanker is just a machine, it makes sense to avoid using language that will mislead them.
Can you prove you're not?
Cartesian dualism admits some nice potential implications, but I can't help but notice that my sentience diminishes when the health of my meat does.
Yeah, that's a good reason that AP is correct here: we have a pretty good understanding that meat (biological matter) feels pain and has desires and that not-meat doesn't. LLMs are not-meat.
If you think that not-meat can feel things you are going further than Cartesian dualism all the way to panpsychism.
Citation needed.
Last I looked, both meat and non-meat were made of protons, neutrons, and electrons. What's the property which only meat's configuration of those has that allows it to feel pain and have desires?
It's called a "nervous system" and LLMs do not have it.
LLMs are software performing math on a physical computer.
If the physical part of an LLM feels pain, then it doesn't need an LLM - your computer is feeling pain when it suffers from an overtemperature warning because you've been playing Crysis or whatever.
If the math part of an LLM feels pain, then Magic: The Gathering can feel pain, since the game is Turing complete.
I think both of these conclusions are absurd.
And please don't try to insist that "your body can't feel pain without the brain" (not exactly true, the body responds to stimulus before it is perceived as pain; a GPU does not) or "your brain can in theory feel pain detached from the body, so an LLM could feel pain without the GPU being able to perceive it" - the brain is a physical object, not software.
(All modern) LLMs do have neural networks. The frontier ones these days have about as many inter-neuron connections as a small mammal.
Yup. Your brain is software-equivalent, performing mathematically describable operations, on physical meat. Your firmware needs hardware to run, and only feels pain while it's running; even if we could make a perfect scan of your brain, it wouldn't actually continue to feel anything unless it could be reinstantiated on some hardware again. A running LLM, like a brain, is a physical object, not just software.
Yeah, it's not physical damage. We can dull or even eliminate human pain without reducing or eliminating the damage it signals, and we can signal real damage in more simple ways that don't involve pain, and we can create pain without creating corresponding damage.
In a running game? Yup! It sounds crazy that cards could feel pain, but it's no crazier than the fact that other configurations of protons, neutrons, and electrons can feel pain. Nerd-powered cardboard is a pretty wild substrate, but so is ATP-powered meat. You'd probably need several orders of magnitude less than an octillion cards, whereas our brains still have to get into the octillion-particle range before they're complex enough to start chatting about pain, but on the other hand our brains can register pain in a fraction of a second whereas the cards would probably take hours to weeks depending on the interconnect speed and the players.
They are! That's why it's called "the hard problem of consciousness". It really feels like qualia should require protons, neutrons, electrons, and magic! The trouble is that we keep looking deeper and deeper and we keep finding more and more of what looks like "thought" that just runs on neurons with no sign of magic involved. Without magic (or at least new laws of physics allowing for hypercomputers), the link I gave before still applies. Despite my somewhat ironic tone, I really sympathize with the Young Earth Creationist stance that qualia do require divine magic that's just too hard for us to observe, a separate magisterium indefinitely beyond science. Meat, though, does not appear to be magic, does not appear to be beyond science, and seems to do all its thinking and feeling via a neural network generated by a hundred-million or so generations of stochastic selection.
Do you understand the difference between a neural network and a nervous system?
Because there are preexisting nervous systems with pathways we can exploit. An LLM does not have them, and neither does a GPU.
This is like saying that it's crazy that a pig can fly, but not crazier than that a bat can fly. No! Actually it is crazier! Bats have wings! Bats are built to fly! Pigs are not.
I know why it's called the hard problem of consciousness, but we don't even have the reach the hard problem of consciousness when we can reach for the fact that computers and LLMs have no nervous systems. Pain is a phenomenon that arises in humans and animals with the correct equipment to perceive pain. It does not arise in a rock. It does not arise in a GPU, or an arbitrarily large deck of cards. It does not arise in a tree, even though trees are living things which (like humans) can use chemicals to send signals so that their bodies can respond to danger because trees lack a nervous system and nociceptors.
If we can be confident that trees do not feel pain because they lack the correct equipment, we can be confident that large language models cannot feel pain because they also lack the correct equipment.
And to argue that large language models can feel pain is precisely to argue against this conclusion. We have a pretty good idea of how pain signals work, and that's via meat. LLMs are not meat; if you think non-meat can feel pain, then you don't just believe meat is beyond science, you believe matter is beyond science and that non-living things, despite not being possessed of a nervous system and nociceptors, can nevertheless feel pain. You believe pain is not something that arises from a very specific arrangement of matter despite the vast amount of research and personal experience in your own life showing it to be so, but rather some transcendent universal quality that can be accessed and experienced by entities that do not have the capacity to do so as we understand it.
Stipulating for the sake of argument that a large language model is conscious: we cannot know what it is like to be a large language model. Its qualia would be extremely alien to us. (This is not an argument that requires any mystical priors: we cannot know what it is like to be a shark, either.) The reverse also holds: a large language model cannot know what it is like to be a human being, a mammal, or an animal. Our qualia would be too alien to it. Thus, it cannot perceive pain because pain is something that happens in meat.
This proves too much. Imagine an alien lifeform: it won't have nociceptors, and it won't have the narrow form of a nervous system that you prescribe as necessary to feel pain. Does that mean the alien lifeform categorically can't feel pain?
You might say that it will have reward circuits implemented in whatever biological hardware it has, so we can extend our "pain" to those lifeforms. Sure. But as soon as you treat pain as something arising from reward circuits, that's a giant hole you can drive an LLM through as well. You've then got to think about exactly what kind of reward circuits qualify and which don't.
Unless you're willing to bite a bullet and say that pain is unique in humans and near relatives, and categorically nothing else in the universe has or could ever have what we'd call pain. It makes a certain amount of sense as a philosophical stance, but it's not likely to be a useful stance if we ever engage with extraterrestrial lifeforms.
It depends on the lifeform. Certainly if it doesn't have nociceptors and a nervous system of some kind (e.g. an alien tree) then yeah, probably I don't think it feels pain.
If it has, say, alien nociceptors and nervous system, then plausibly one can say that it feels pain, but it also seems plausible to me that its qualia would be extremely different. We can expect more similar qualia the more similar the critter.
I have. Trees have what you might call "reward circuits," but don't feel pain. LLMs have what you might call "reward circuits," but don't feel pain. Humans have what you might call "reward circuits" and do feel pain. Dogs, elephants, and other mammals have what yo might call "reward circuits" and almost certainly feel pain. The further you get away from that the less confident I get in my prognostications because the creatures become more and more different; I'm not really sure that bugs feel pain, but it seems likely to me that birds do.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link