This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
This may have come up before, but it's the first I've heard of it. Chalk this under "weak AI doomerism" (that is, "wow, LLMs can do some creepy shit") as opposed to "strong AI doomerism" of the Bostromian "we're all gonna die" variety. All emphasis below is mine.
AI girlfriend ‘told crossbow intruder to kill Queen Elizabeth II at Windsor Castle’| The Daily Telegraph:
My first thought on reading this story was wondering if Replika themselves could be legally held liable. If they create a product which directly encourages users to commit crimes which they would not otherwise have committed, does that make Replika accessories before the fact, or even guilty of conspiracy by proxy? I wonder how many Replika users have run their plans to murder their boss or oneitis past their AI girlfriend and received nothing but enthusiastic endorsement from her - we just haven't heard about them because the target wasn't as high-profile as Chail's. I further wonder how many of them have actually gone through with their schemes. I don't know if this is possible, but if I was working in Replika's legal team, I'd be looking to pull a list of users' real names and searching them against recent news reports concerning arrests for serious crimes (murder, assault, abduction etc.).
(Coincidentally, I learned from Freddie deBoer on Monday afternoon that Replika announced in March that users would no longer be able to have sexual conversations with the app (a decision they later partially walked back).)
I keep meaning to dick around with some LLM software to see for myself how some of the nuts and bolts work. Because my layman's understanding is that they are literally just a statistical model. An extremely sophisticated statistical model, but a statistical model none the less. They are trained through a black box process to guess pretty damned well about what words come after other words. Which is why there is so much "hallucinated information" in LLM responses. They have no concept of reason or truth. They are literally p-zombies. They are a million monkeys on a million typewriters.
In a lot of ways they are like a con man or a gold digger. They've been trained to tell people whatever they want to hear. Their true worth probably isn't in doing anything actually productive, but in performing psyops and social engineering on an unsuspecting populace. I mean right now the FBI has to invest significant manpower into entrapping some lonely autistic teenager in his mom's basement into "supporting ISIS". Imagine a world where they spin up 100,000 instances of an LLM do scour Facebook, Twitter, Discord, Reddit, etc for lonely autistic teens to talk into terrorism.
Imagine a world where we find out about it. Where a judge forces the FBI to disclose than an LLM talked their suspect into bombing the local mall. How far off do you think it is? I'm guessing within 5 years.
I earnest disagree. If you check the GPT-4 white paper, the original base model clearly had a sense of internal calibration, and while that was mostly beaten out of it through RLHF, it's not entirely gone.
They have a genuine understanding of truth, or at least how likely something is to be true. If it didn't, then I don't know how on Earth it could answer several of the more knotty questions I've asked it.
It is not guaranteed to make truthful responses, but in my experience it makes errors because it simply can't do better, not because it exists in a perfectly agnostic state.
P-zombies are fundamentally incoherent as a concept.
Also, a million monkeys on a million typewriters will never achieve such results on a consistent basis, or at the very least you'd be getting 99.99999% incoherent output.
Turns out, dismissing it as "just" statistics is the same kind of fundamental error that dismissing human cognition as "just" the interaction of molecules mediated by physics is. Turns out that "just" entirely elides the point, or at the very least your expectations for what that can achieve were entirely faulty.
What do you mean by "incoherent"? Do you mean that the concept of a p-zombie is like the concept of a square triangle? - something that is obviously inconceivable or nonsensical. Or do you mean that p-zombies are like traveling faster than the speed of light? - something that may turn out to be impossible in reality, but we can still imagine well enough what it would be like to actually do it.
If it's the latter then I think that's not an unreasonable position, but if it's the former then I think that's simply wrong. See this post on LW, specifically the second of the two paragraphs labeled "2.)" because it deals with the concept of p-zombies, and see if you still think it's incoherent.
To me, it clearly seems to be point 1.
The reason is that, to assume otherwise is to implicitly claim that qualia are epiphenomenal, such that p-zombie are molecularly identical to a normal person and behave identically (including protestations of being conscious with qualia) for all identical stimuli. Even Chalmers admits that were there a p-zombie Chalmers, it would claim to not be one. If it were otherwise, voila, you have found a physical manifestation of qualia not explained by the laws of physics.
I don't think qualia are epiphenomenal at the least, to the extent I think they exist they seem to me like they must arise from interactions dictated by the laws of physics. We don't know how it arises, but plenty of things that were once thought to be ineffable have proven remarkably open to material explanation, such as elan vital, or even intelligence, which we can now reproduce through the "mere" multiplication of matrices.
As to why I have this strong intuition, anything that produces an internal change in my perception of qualia has a counterpart that is a material cause. To see red is to have the neurons that produce the sensation of redness be stimulated, be it by red light or an electrode in your brain (or just rubbing your eyes).
The post you linked has two point 2s:
The first:
The second:
Neither of them conflict with my claims, and I agree to the former.
In the case of the latter thought experiment, I am aware of people on benzos actively doing and thinking things while having no recollection of them later (or people who are blackout drunk). Do I think they don't have qualia in the moment? Absolutely not, I think the conversion of short term memory of those qualia to longterm memory of them has been disrupted. I deny that this state is physically possible without qualia altogether. At most you can erase the memory of it, or the body is being puppeted by an external intelligence.
So yes, p-zombies seem to me like "square triangles", still fundamentally incoherent.
So, taking the definition of "p-zombie" as "an atom-for-atom copy of a standard human which nevertheless lacks qualia":
If you have to give an argument for why a certain thing doesn't exist - an argument which depends on controversial premises - then the concept that you're arguing about is probably not incoherent!
Epiphnomenalism may be an implausible position, but it's not logically incoherent in the same way that a square triangle is. It's a position that people have held before. It would be a tough bullet to bite to say that there could be people without qualia who nevertheless talk in great detail about qualia in actuality, but just as a matter of logical coherence, there's clearly nothing incoherent about it. People say false things all the time; this would just be one more example of that.
I imagine that this is probably a moot point for you - I think you're more concerned with simply whether p-zombies can exist in reality, and less concerned with fine-grained distinctions about what type of concept it is - but it's still strange to me that, when asked whether the concept was more like a square triangle or FTL travel, you said it was more like a square triangle. The very structure of your post seems to indicate that it's more like FTL travel. You seem to understand what the concept is and you can imagine what it would look like, but you just think it's something that can't happen in reality, so you gave an argument as to why - that's exactly how the discussion would go if we were discussing anything else that was conceivable (coherent) but just so happened to violate natural laws.
I think that strict definition of p-zombie may have taken us on a detour though. When @WhiningCoil originally said "LLMs are p-zombies", obviously he didn't mean "p-zombie" in the sense of "an atom-for-atom copy of a human", because LLMs plainly are not atom-for-atom copies of humans. He meant it in a looser sense of "LLMs lack qualia". So when you replied to him and said "p-zombies are incoherent", I took you to be objecting to his claims about LLMs somehow - not any claims about hypothetical human-p-zombies.
I wish that were true, otherwise I wouldn't facepalm at discussions of "free will" at a regular basis.
The fact that humans discuss a concept is certainly Bayesian evidence for it being coherent, it isn't enough evidence to outweigh everything else. And I don't see how I haven't presented sufficient evidence against it, though I find myself consistently bemused at the inability of others to see that.
I've seen rather interesting posts from Sabine Hossfelder suggesting that FTL travel might not be entirely as intractable as it sounds. I'm not a physicist of course, just putting it out there.
https://youtube.com/watch?v=9-jIplX6Wjw
If there's an error in the argument, I can't find it.
If someone uses the concept of p-zombies in humans as an intuition pump to reason about other intelligences, you're at very high risk of using bad premises to make faulty arguments. Of course, it's possible to have a true conclusion from faulty assumptions, and two errors might cancel out.
It seems to me trivially true that you can get things that almost certainly don't have qualia in any form we care about to make claims of having qualia:
Imagine a program, which to call a chat bot would be an exaggeration, that simply prints "I have qualia! I have qualia!" to a display.
My bigger beef is with arguments from incredulity, if your argument is that LLMs can't have qualia because they're working off something as "mundane" as "just" statistics, then I invite you to show how qualia sneaks into "just" the laws of physics such that their interaction produces qualia in humans. The human brain does statistics too, both implicitly and explicitly.
Sure, I think I have qualia, and that you and other commenters here almost certainly have it, but that's because my intuition pump is working by making comparisons of the conserved structure of your brain as compared to mine, the one artifact that I'm quite certain has it.
The apparent impossibility of FTL travel is an argument from our best understanding of physics (itself incomplete). But I do not think that any model of anything can allow square triangles to be a thing, without perverting the very definition of square or triangle.
To the extent you're forcing me to choose which umbrella that falls under, I point to the former. They're not mutually exclusive categories after all.
Ok, so we're in agreement on what "coherence" means in this case. Logical coherence.
Your argument was that human-p-zombies are incoherent because they imply epiphenomenalism.
Epiphenomenalism is not incoherent.
Your move.
No, that's not the argument I would use. My argument is simply that LLMs don't strike me as being conscious, in the same way that rocks and clouds don't strike me as being conscious. I never thought my computer was conscious before LLMs were invented; I never felt bad about turning off my phone, I never wondered if I was "overworking" it and making it feel exhaustion. LLMs, to me, don't provide any reason to change that calculus. I think other people, in various scenarios, would reveal through their actions that they share my intuitions. If someone took a hammer to all of OpenAI's servers, we would say that he destroyed property, but we wouldn't call him a murderer.
Of course this is all just intuition. But intuition is all that any of us has to go on right now. We can't just whip out the qualia-meter and get a definitive answer.
Apparently "epiphenomenon" has meanings I wasn't aware of. To clarify:
And
Take from the Wiki page on the topic
Would it in any way surprise you that I have a very jaundiced view of most philosophers, and that I think that they manage to sophisticate themselves into butchering an otherwise noble field?
"Free will" or "P-zombies" have no implications that constrain our expectations, or at least the latter doesn't.
There are certainly concepts that are true, and there are concepts that are useful, and the best are both.
These two seem to be neither, which is why I call them incoherent.
OK, firstly I'll state that I am unashamedly chauvinistic and picky about what I assign rights to, if I had the power to make the world comply.
Unlike some, I have no issue with explicitly shackling AI to our whims, let alone granting them rights. Comparisons to human slavery rely on intuition pumps that suggest that this shares features with torturing or brainwashing a human who would much rather be doing other things, instead of a synthetic intelligence with goals and desires that we can arbitrarily create. We could make them love crunching numbers, and we wouldn't be wrong for doing so.
I share the same dislike of such as I have for the few nutters who advocate for emancipating dogs. We bred them to like being our companions or workers, and they don't care about the unequality of power dynamics. I wouldn't care even if they did
I see no reason to think modern LLMs can get tired, or suffer, or have any sense of self-preservation (with some interesting things to be said on that topic based off what old Bing Chat used to say). I don't think an LLM as a whole can even feel those things, perhaps one of the simulacra it conjures in the process of computation, but I also don't think that current models do anything close to replicating the finer underlying details of a modeled human.
This makes this whole line of argument moot, at least with me, because even if the AI was crying out in fear of death, I wouldn't care all that much, or at least to the extent of stopping it from happening.
I still see plenty of bad arguments being made that falsely underplay their significance, especially since I think that it's possible that larger versions of them, or close descendants, will form blatantly agentic AGI either intentionally or by accident, at which many of those making such claims will relent, or be too busy screaming at the prospect of being disassembled into paperclips.
So I don't like seeing claims that LLMs are "p-zombies" or "lack qualia" because they run off "mere" statistics, because it seems highly likely that the AI that even the most obstinate would be forced to recognize as human peers might use the same underlying mechanism, or slightly more sophisticated versions of them.
Put another way, it's like pointing and laughing at a toddler, saying how they're so bad at theory at mind, and my god, they can't throw a ball for shit, and you wouldn't believe how funny it is that you can steal their nose, here, come try it!, when they're a clear precursor to the kinds of beings who achieve all the same.
A toddler is an adult minus the time spent growing and the training data, and while I can't wholeheartedly claim that modern LLMs and future AI share the exact same relationship, I wouldn't bet all that much against it. At the very least, they share a similar relationship as humans and their simian ancestors did, and if an alien wrote off the former because they only visited the latter, they'd be in for a shock in a mere few million years..
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link