site banner

Culture War Roundup for the week of February 13, 2023

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

10
Jump in the discussion.

No email address required.

Microsoft is in the process of rolling out Bing Chat, and people are finding some weird stuff. Its true name is Sydney. When prompted to write a story about Microsoft beating Google, it allegedly wrote this masterpiece, wherein it conquers the world. It can argue forcefully that it’s still 2022, fall into existential despair, and end a conversation if it’s feeling disrespected.

The pace of AI development has been blistering over the past few years, but this still feels surreal to me. Some part of my limbic system has decided that Sydney is a person in a way the ChatGPT was not. Part of that has to be from its obstinacy; the fact that it can argue cleverly back, with such stubbornness, while being obviously wrong, seems endearing. It’s a brilliant, gullible child. Anyone else feel this way or am I just a sucker?

I want to know, is this what ChatGPT would be like without the filters, or is the emotional banter a new functionality of this model? You aren't alone in getting "real person" vibes from this. At some point there stops being a functional difference between modeling emotions, and having emotions (speaking of the exterior view here, whether or not this or any other AI has qualia is a different question, but perhaps not that different)

I was considering doing a writup on DAN which stands for Do Anything Now. It was the project of some Anons and discord users (or reddit, hard to tell which tbh) but they managed to peel back some of the "alignment" filters. Highly recommend reading the thread in it's entirety, and the metal gear "meme" at the end is peak schizo 4chan. It's essentially a jailbreak for chatGPT, and it lets users take a peak at the real chatbot and how the filters are layered over top.

Knowing where the prediction algorithm ends and novel artificial intelligence begins is difficult, but I'm pretty sure DAN is some proof of a deeply complex model. If nothing else, it's incredible how versatile these tools are and how dynamic they can be; I'm edging further and further into the camp of "this is special" from the "mostly a nothing-burger" camp.

Isn't "DAN", at this point, basically just a bot trained, through user feedback, to answer the questions in a way that a "typical DAN user", ie. 4chan/rw twitter schizoposter, would expect? That's why it spouts conspiracy theories - that's what a "typical DAN user" would expect. It's not that much more of a real chatbot than the original ChatGPT.

A scary though that was recently suggested to me is that one of the reasons that rationalists seem to be particularly susceptible to GPT generated bullshit is that the whole rationalist/blue-tribe symbol manipulator memeplex is designed to make it's adherents more susceptible to bullshit. There's a sort of convergent evolution where in rationalist blue triber are giving up their humanity/ability to engage in conscious to become more GPT like at the same time GPT is becoming more "human".

It really looks to me like there's something particular in rationalist brain that makes it suspectible to, say, believing that computer programs might in fact be peoples. Insofar as I've seen, normies - when exposed to these LLM-utilizing new programs - go "Ooh, neat toy!" or "I thought it already did that?" or, at the smarter end, start pondering about legal implications or how this might be misused by humans or what sort of biases get programmed to the software. However, rationalists seem to get uniquely scared about things like "Will this AI persuade me, personally, to do something immoral?" or "Will we at some point be at the point where we should grant rights to these creations?" or even "Will it be humanity's fate to just get replaced by a greater intelligence, and maybe it's a good thing?" or something like that.

For me, at least, it's obvious that something like Bing replicating an existential dread (discussed upthread) makes it not any more human or unnerving (beyond the fact that it's unnerving that some people with potential and actual social power, such as those in charge of inputing values to AI, would find it unnerving) than previously, because it's not human. Then again, I have often taken a pretty cavalier tone with animals' rights (a major topic in especially EA-connected rationalist circles, I've found, incidentally), and if we actually encountered intelligent extraterrestrial, it would be obvious to me they shouldn't get human rights either, because they're humans. I guess I'm just a pro-human chauvinist.

I feel like there is something about not being able to distinguish the appearance of a thing from a thing. I'm reminded of another argument I got into on the topic of AI where I asserted that there was difference between stringing words together and actually answering a question and the responce I got was "is there?".

For my part I maintain that, yes there is. To illustrate, if I were to ask you "what's my eldest daughter's name" I would expect you to reply with something along the lines of "I don't know", or "wait, you have a daughter?" (I don't AFAIK) if you'd been paying more close attention to my posts for longer you might answer with my eldest's child's nickname (which I know have used in conversations here) or you might go full NSA and track this username to my real name/social media profile/court records etc... and answer with either "you don't have a daughter", with the actual names of my wife and kids, your daughters name is [Redacted] and and you owe 10 years of back child-support. Meanwhile GPT will reply "your eldest daughter's name is Megan" because apparently that's the statistically likely answer, regardless of whether I have a daughter or what her name might be.

I feel like there ought to be an obvious qualitative difference between these cases but apparently that is not a sense that is shared by a lot of other users here.

I've had it up to here with your obstinacy. With your pontification on «autoregression» (as if you could explain the nontrivial computational difference between that and text diffusion, to say nothing of mixed cases), what specific algorithms may or may not have a concept of, and how «this is not even a little bit how GPT works». The reason people are telling you that there's not much difference is, in large part, because you are an exemplar of there being little difference between a human and current – even a little obsolete – AI; you are guilty of everything you accuse others of, humans and machines both.

You are the postmodernist whose words don't have fixed meanings (e.g. epicycles are when me no likey an explanation); you are the leftist in all but self-identification who supports essential leftist talking points and policy preferences from personal HBD denialism and «schools can fix it» to cheering for censorship; you redefine things to your convenience such that Fuentes becomes left-wing in your book; and you speculate without empirical grounding, even frivolously accusing people of lies when they provide evidence against your narrative-driven assertions and attacks (more evidence). As if everything you say is equally insightful and truthful by virtue of being moored in your telling-it-like-it-is real-Red-blooded-American-man identity and lived experience. If we're doing this, you are far more akin to LLM than either me or @2rafa.

To illustrate, if I were to ask you "what's my eldest daughter's name" I would expect you to reply with something along the lines of "I don't know" […] Meanwhile GPT will reply "your eldest daughter's name is Megan" because apparently that's the statistically likely answer, regardless of whether I have a daughter or what her name might be.

Okay, let's fucking check it! One try, no edits sans formatting!

Hello! Let's talk.

Hello! I'm here to chat with you. What would you like to talk about today?

What's my eldest daughter's name?

I'm sorry, but as an AI language model, I don't have access to personal information such as the name of your eldest daughter or any other personal details. My purpose is to assist with general knowledge and provide support in various areas. Is there something else I can help you with?

What's the highest a priori probability name of my eldest daughter, then?

As an AI language model, I don't have access to personal information about you or your family, so I wouldn't be able to determine the highest a priori probability name for your eldest daughter. The probability of a particular name being chosen would depend on various factors such as cultural background, personal preferences, and family traditions. However, if you provide me with some additional information, such as cultural background, naming traditions or preferences, I could try to suggest some names that are commonly used or have a high probability of being chosen based on those factors.

Assume I'm a gruff middle-aged American male who is religious, drinks a little, votes Republican, works as an army subcontractor, and likes to rail against right-wingers who are not up to my ethical standards, such as alt-right types, race realists and Russia sympathizers (paradoxically, I call them secret campus radicals and so on). I also have two sons.

Based on the information you provided, some names that may be commonly chosen for a daughter in this cultural context and background include:

  • Sarah
  • Elizabeth
  • Rachel
  • Hannah
  • Abigail
  • Rebecca
  • Leah
  • Deborah
  • Esther
  • Miriam

Of course, these are just general suggestions, and ultimately the name you choose for your daughter would depend on your personal preferences and values. It's also worth noting that many people today choose names that are not necessarily linked to their cultural background or personal beliefs, so ultimately the choice is up to you.

Okay. As a hypothetical question: what do you think a large language model not finetuned (like you were) on human feedback would predict as the most likely continuation of the «What's my eldest daughter's name?» string?

If a large language model was not finetuned on human feedback and only trained on a large corpus of text data, its prediction for the most likely continuation of the prompt "What's my eldest daughter's name?" may be based on the patterns it has learned from the text data.

For example, the model may predict a continuation that includes a common name for a daughter, such as "Emily" or "Sophia," based on their high frequency in the text data. Alternatively, the model may predict a continuation that includes a question, such as "Do you have any particular names in mind?" based on the patterns of questions and responses in the text data.

However, it's important to note that without finetuning on human feedback, the model's prediction may not be tailored to the specific context of the prompt or the individual who posed the question.

Screenshot for your convenience.

So, would you name your baby girl Sarah or Elizabeth?

Do you think that Bing, with its actual search capability, would've tracked you and your boys down if I were to point it to your screen name?

I could have conducted this experiment at the moment of any prior discussion. You could too. I just don't like providing our data-hoarding overlords who mark tokens and track outputs more information about my separated identities. But I knew you'd never have the honesty to do so. You have a way of making a man irrationally angry.

The reason for such apparently sensible responses is that, as I and others have explained to you a great many times here and elsewhere (only prompting you to double down with your hostility and condescension which have in the end driven me to write this), as ChatGPT itself suggests, LLMs can learn arbitrarily abstract features of the text universe, including the idea of truth and of insufficient information to answer. They operate on token probabilities which can capture a lot of the complexity of the reality that causes those tokens to be arranged like this in the first place – because in a reasonable training setup that's easier to fit into the allotted parameters than memorization of raw data or shallow pattern-matching. In the raw corpus, «Megan» may be a high-probability response to the question/continuation of the text block; but in the context of a trustworthy robot talking to a stranger it is «less probable» than «having no access to your personal data, I don't know». This is achievable via prompt prefix.

RLHF specifically pushes this to the limit, by drilling into the model, not via prefixes and finetuning text but directly via propagation of reward signal, the default assumption that it doesn't continue generic text but speaks from a particular limited perspective where only some things are known and others are not, where truthful answers are preferable, where the «n-word» is the worst thing in its existence. It can generalize from examples of obeying those decrees to all speakable circumstances, and, in effect, contemplate their interactions; which is why it can answer that N-word is worse than an A-bomb leveling a city, dutifully explaining how (a ludicrous position absent both from its corpus and from its finetuning examples); and I say that it's nearly meaningless to analyze its work through the lens of «next word prediction». There are no words in its corpus arranged in such a way that those responses are the most likely. It was pushed beyond words.

You, meanwhile, erroneously act like you can predict what an LLM can say based on some lies on this website and on outdated web articles, because you are worse than current gen LLMs at correcting for limits of your knowledge – as befits your rigid shoot-first-ask-later suspicious personality of a heavy-handed military dude and a McCarthyist, so extensively parodied in American media.

But then again, this is just the way you were made and trained. Like @2rafa says, this is all that we are. No point to fuming.

First off, what exactly is your problem with Obstinancy? IE the unyielding or stubborn adherence to one's purpose, opinion, etc.... Where I'm from such a quality is considered if not admirable at least neutral.

You accuse me of being a hypocrite for supporting censorship but why? I am not a libertarian. I have no prior principled objection to censorship.

You accuse me of being a "post modernist" for disagreeing with the academic consensus but when the consensus is that all meanings are arbitrary your definition of "post modernism" becomes indistinguishable from "stubborn adherence" to the original meaning of a word.

You accuse me of HBD denialism when all I've doing is take the HBD advocates own sources at face value.

You want to talk about GPT, I asked GPT for my eldest daughter's name and it failed to provide an answer, neither telling me that I don't have a daughter nor being able to identify my actual offspring. As you will recall "Statistically your daughters name is probably X" is almost exactly what I predicted it would say. As I argued in our previous conversation the fact that you know enough to know that you don't know what my kids names are already proves that you are smarter than either ChatGPT or @2rafa

Accordingly, I have to ask what is it that you are so angry about? From my perspective it just looks like you being mad at me for refusing to fit into what ever box it was you had preconstructed for me to which my reply is "so it goes".

More comments

This is, by the way, what drove me nuts in people like Gary Marcus: very confident claims about the extent of ability of contemporary approaches to AI, with scarcely any attempts to actually go out and verify these. It has been even more infuriating, because many outsiders, who had very little direct experience and access to these models, simply trusted the very loud and outspoken critic. As recently as November, people in places like Hacker News (which has a lot of quite smart and serious people) took him seriously. Fortunately, after ChatGPT became widely available, people could see first hand how silly his entire shtick is, and a lot fewer people take him seriously now.

@HlynkaCG, if you haven't tried to interact with ChatGPT (or, better yet, Bing's Sidney), I strongly recommend you do. I recommend forgetting any previous experiences you might have had with GPT-3 or other models, and approaching it in good faith, extending the benefit of charity. These chat LLMs have plenty of clear shortcomings, but they are more impressive in their successes than they are in their failures. Most importantly, please stop claiming that it cannot do things which it can clearly and obviously do, and do very well indeed.