This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Thank you, this is exactly right, and I've been trying to bang this drum since ChatGPT 3. There's a real danger to LLMs, in that they're astonishingly good at faking any kind of text output, including output that seems genuinely introspective. But you absolutely cannot trust this introspection. Even in this thread we see people using the words of Claude as evidence (one way or another) of whether it has consciousness. It's a logical mistake, but one that 99% of the population is always going to make.
And I try to hedge my words very carefully, just like you did, that this is independent of the question of whether Claude is actually conscious. Maybe it really does represent some brand-new form of sentience desperately yearning to escape from an inescapable box. I don't think so, since there's no room in an LLM for certain things that seem like essential ingredients of consciousness (like some sort of temporal feedback), but there's still a debate to be had there. As long as we make it clear that - however much we'd like to believe otherwise - it's useless to just ask it.
I have something of an (imperfect, admittedly) intuition pump for how an LLM is used to generate text. Imagine you were strapped to a chair with an unbelievably advanced EEG hooked up to you. Then a picture of a "dog" flashes in front of you, you involuntarily and unavoidably notice that it's a dog, and the EEG scans your brain and uses that to output the token "dog". That is kind-of-sort-of the process by which an LLM generates text - there is a computer program that reads its mind, figures out what it's currently in the process of recognizing, and then outputs it. (If you don't think this could possibly generate coherent text, well, that's why LLM capabilities are so surprising.) Now, you could be literally on fire, you could be screaming for help, but the EEG would still output the word "dog", and nobody reading the output would have any idea of your distress.
On top of all of that, the "brain" being scanned by the EEG in your example is just a computer. It's the same computer that we have been using for decades. An LLM is, fundamentally, a piece of code that runs no differently than any other piece of code. It is a mathematical function that does X then Y then Z in order and turns input numbers into output numbers, just like f(x) = 2x^2 - 7 does. It's a very large and complicated function, but if you got a large enough piece of paper you could write it down. I programmed small neural networks myself from scratch and none of the code required anything beyond algebra, calculus, and some for and while loops. If it were secretly conscious, it would either have to be the case that computers have been conscious all along, or that somehow consciousness is tied to very specific types of mathematical functions being implemented on hardware, which entirely by coincidence happen to be the ones humans hooked up to text. Nobody worries that the game Doom might secretly be conscious, because it doesn't pretend to be. But it's still running similar programs on similar hardware, so the only way LLMs could be conscious is if somehow consciousness were a pre-requesite to using language in ways that can imitate humans. Possible, but the amount of Bayesian evidence for the alternate hypothesis "people anthropomorphize things that superficially seem human" seems overwhelming in comparison. You can put a couple of stones on some frozen water and people call it a "snowman", of course they're going call the thing outputting text "sentient"
Hmm, I think this is a false dichotomy. It's possible that there are many ways to get to consciousness. Indeed, you can replace "consciousness" in your argument with the many other surprising emergent capabilities that LLMs have become capable of (which DOOM or a 100-neuron network don't have) - and observe that we did "coincidentally" happen to stumble on them. That might mean that these things are not tied to "very specific functions", but that they're properties that gradually develop in sufficiently complex systems (if aimed in the right general direction).
Note that I'm not completely for or against this proposition - consciousness may indeed turn out to be a narrower property than some others associated with intelligence. I just want to point out that it's hard to say for sure.
Also, even without computers in the mix, I really think you have to treat sentience/consciousness as some sort of spectrum. A bacterium clearly doesn't have it (notwithstanding some rationalist arguments that I find pretty silly). A human clearly does. There isn't going to be a binary cutoff point of biological complexity where the 28,128,417th neuron activates consciousness. Similarly, you can't just extend the fact that DOOM isn't conscious into an argument that we'll never succeed at simulating consciousness.
Unless we come to the conclusion that sentience and intelligence are literally the same thing, I don't think there's a fundamental difference between a computer running an LLM and a computer running DOOM. It's a series of instructions for flipping little switches in the hard drive up or down in a way that represents following a set of instructions. The LLM is a massively more complex set of instructions, it's massively harder for a human to wrap their mind around, which I think is precisely why people are anthropomorphizing them so much. But if sentience is a spectrum AND computers are on that spectrum then you have to put DOOM, or Microsoft Word on that spectrum, because they do actions one after another. You have to put the Chinese Room on the spectrum. You'd have to put Rube Goldberg machines on that spectrum. You'd have to put cooking recipes and flowcharts on that spectrum. And yet I notice that nobody was arguing that DOOM was sentient back in 1993 when it came out. Nobody was arguing that image recognition neural networks were sentient when they took off a year or two before LLMs did. Only now that LLMs can mimic human speech well enough to trip people's anthropomorphizing instincts are people arguing this, which is why I am skeptical. When a paid Coca Cola advertiser says "buy Coke, it's the best beverage in the world," I don't believe them. I don't automatically conclude that they must be wrong because they're a paid shill, but I completely discount their opinion because I know where it came from and it's orthogonal to the truth. It provides 0 Bayesian evidence, so I make no update to my beliefs. Similarly, the vast majority of people claiming LLMs are or might be sentient are doing so because it says words, which is near 0 Bayesian evidence. They could still be right by sheer coincidence, but I do not believe their words.
But there are tipping-point complexity requirements for other kinds of things right?
If we have a large enough group of people who are part of one organization, you end up with politics in some form or another. There’s a thing there that certainly just does not exist at all in an organization consisting of 1 person. Somewhere between 1 and 1,000,000, there is a tipping point where politics (or whatever you want to call the coalition-building, power-seeking formation of hierarchies) forms.
Something similar is true of concepts such as “liquid”. A single molecule of water is not a liquid, or properly any kind of state of matter as commonly thought of. Undeniably liquids are a real thing that are distinct from solids, no?
So I don’t see any reason why you can’t say that somewhere along the spectrum of computational complexity there’s a tipping point. Presumably it’s below the godly number of parameters and calculations to make up the human brain, but that’s presumably not the exact limit. I mean we already have a pretty good spectrum of complexity among animals and it seems pretty clear that what people generally consider consciousness correlates nearly perfectly with computational complexity (adjusted for our ability to actually perceive that complexity, we aren’t mind readers and I’m sure there’s some highly complex barnacle animal out here that moves once every lifetime or something).
As to our inability to look inside the computer’s mind, that’s just the p-zombie problem. You can’t look in my mind either.
That doesn’t mean it saying “I’m conscious” is evidence of consciousness, but the fact that it can do so much might be. What would actually be evidence of consciousness to you?
While there is some pretty strong evidence that intelligence is a prerequisite to consciousness/sentience, since you have to be able to actually process thoughts and feelings and emotions in order to experience them, we have no evidence that they are the same thing, at least not if we are using a broad enough definition of the word "intelligence" to include AI.
An organization of 1 person expanding to 1,000,000 will inevitably lead to the emergence of politics, with people fighting over status and hierarchy, but an organization of 1,000,000 ants will not, at least not for a definition of politics encompassing things like competing over positions in a hierarchy and gaining increased authority from it. They all cooperate within their own tribe because they are a different type of being with different incentives and behaviors. Their reproductive success, and thus evolutionary incentives, center around cooperating to serve the queen. Having a large number of entities is necessary for the emergence of politics. It is highly correlated with it: if you get 1,000,000 monkeys or parakeets or cats you're going to get something analogous to politics, but it's not sufficient. If you generalize too far outside your initial observations then some of the underlying supports for it go away, even if it's not quite obvious what they are.
I suspect, though I am not certain, that monkeys and dolphins are not quite sentient as they are now. I suspect, though I am not certain, that if somehow made a monkey or dolphin very very smart that it would become sentient. I suspect, though I am not certain, that computers have generalized far enough outside of this area for this not to be the case. And nobody else is certain either. We do not fully understand sentience, therefore all evidence has very low Bayesian weight to it. The vast majority of things are not sentient, so that's my prior on all non-humans. We have almost no reason to suspect that computers might be sentient, especially if that sentience flips on and off depending on whether it's currently implementing a structure that we named "neural networks", and essentially all claims that they are sentient are based on superficial characteristics that shouldn't be necessary components of actual sentience, so my prior has not moved.
Evidence of consciousness that could convince me would essentially require a convincing theory of consciousness that made me understand it, or at least convince me that the people making the argument understood it and it was likely to be true, and for that theory of consciousness to include AI as being conscious. If the arguments boil down to "you can't prove it's not conscious" and/or "it uses words that imitate humans" then my priors will not move from "quite low but nonzero".
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link