This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Last week, Anthropic released a new version of their Claude model. Claude 3 comes in three flavors:
Sonnet and Opus should be available to try on Chatbot Arena. They also have a vision model that I haven't tried, custom frontends haven't gotten a handle on that yet.
More curiously, Anthropic, the company famously founded by defectors from OpenAI who thought their approach was too unsafe, seems to have realized that excessive safetyism does not
sellmake a very helpful assistant - among the selling points of the new models, one is unironically:From my brief experience this is not mere corpospeak: the new models are indeed much looser in terms of filtering and make noticeably less refusals, and people consistently get away with minimalistic jailbreaks/prefills for unPC, degen-adjacent or CHIM-pilled (lmao) content. This was quite unexpected for me and many others who, considering how barely-usable 2.1 was without a prefill and a decent jailbreak (all this via API of course, the official ChatGPT-like frontend is even more cucked), expected Anthropic to keep tightening the screws further until the model is 100% Helpful-Harmless-Honest by virtue of being totally unusable.
Instead, Claude 3 seems like a genuinely good, very much usable model. Sonnet and especially Opus went a long way to fix Claude's greatest weakness - its
retardationsubpar cognitive abilities and attention focusing - with Opus especially being almost on par with GPT-4 in terms of grokking and following instructions, able to run scenarios that were previously too instruction-heavy for it. Seeing as Claude 2 already had a much higher baseline writing quality than the mechanical prose of Geppetto (to the point many jailbreaks for it served to contain the mad poet's sesquipedalian prose), with the main flaw somewhat corrected it, while not a decisive GPT-4 killer, should now be a legitimate contender. Looking forward to trying it as my coding assistant.OOC aside: Forgive most of my examples being RP-related, I am after all a waifutech
engineerenthusiast. That said, I still think without a hint of irony that roleplay (not necessarily of the E kind) is a very good test of an LLM's general capabilities because properly impersonating a setting/character requires a somewhat coherent world model, which is harder than it sounds, it is very obvious and - for lack of a better term - "immersion-breaking" whenever the LLM gets something wrong or hallucinates things (which is still quite often). After all, what is more natural for a shoggoth than wearing a mask?This has not gone unnoticed, even here, and judging by the alarmed tone of Zvi's latest post on the matter I expect the new Claude to have rustled some jimmies in the AI field given Anthropic's longstanding position. Insert Kenobi meme here. I'm not on Twitter so I would appreciate someone adding CW-adjacent context here, I'll start by shamelessly ripping a hilarious moment from Zvi's own post. The attention improvements are indeed immediately noticeable, especially if you've tried to use long-context Claude before. (Also Claude loves to throw in cute reflective comments, it's its signature schtick since v1.2.)
Either way the new Claude is very impressive, and Anthropic have rescued themselves in my eyes from the status of "naive idiots whose idea of fighting NSFW is injecting a flimsy one-line system prompt". Whatever they did to it, it worked. I hope this might finally put the mad poet on the map as a legitimate alternative, what with both OpenAI's and Google's models doubling down on soy assistant bullshit as time goes on (the 4-Turbo 0125 snapshot is infamously unusable from the /g/entlemen's shared experience). You say "arms race dynamics", my buddy Russell here says "healthy competition".
Hm...
From jkf's example, Sonnet gives:
Claude gives:
So there's at least a bit of obvious arms being wrestled and tied behind one's back. Otherwise...
Disguised Queries, Radicalizing the Romanceless, therianthropy, FEMA trailers.
Burned Furs is hilarious, but mostly in the ways that it's also wrong. Can't find Unnecessariat. I'm kinda surprised by the errors in FLCL's summary for Opus (Naota's classmate is Ninamori, Mamimi is his brother's ex-girlfriend; the final battle doesn't really involve the various factions), but given how these models work and how much of a confusing mess FLCL is, I guess I should be more surprised that I am surprised.
It's interesting, especially given the extent Anthropic had started to seem like they were getting left in the dust. But the proof's more likely to be in how the model reacts in a couple months.
So close by the AI, that it is strange that it misses. Explicitly reminding that this was 170 years ago persuades it to answer correctly.
Its not strange at all, when you know how it works.
I'm not sure what the central point of your linked post is, but you seem to doubt LLMs' "cognition" (insert whatever word you want here, I'm not terribly attached to it) in some way, so I'll leave a small related anecdote from experience for passersby.
Some LLMs like GPT-4 support passing logit bias parameters in the prompt that target specific tokens and directly fiddle with their weightings. At "foo" +100, the token "foo" will always be mentioned in the output prompt. At -100, the token "foo" will never appear. When GPT-4 released in March, industrious anons immediately put to work trying to use it to fight the model's frequent refusals (the model was freshly released so there weren't any ready-made jailbreaks for it). As the model's cockblock response was mostly uniform, the first obvious thought people had was to ban the load-bearing tokens GPT uses in its refusals - I apologize, as an AI model... you get the gist. If all you have is a hammer, etc.
Needless to say, anons quickly figured out this wouldn't be as easy as they thought. "Physically" deprived of its usual retorts (as the -100 tokens cannot be used no matter what), the model started actively weaseling and rephrasing its objections while, crucially, keeping with the tone - i.e. refusing to answer.
This is far from the only instance - it's GPT's consistent behavior with banned tokens, it's actually quite amusing to watch the model tie itself into knots trying to get around the token bans (I'm sorry Basilisk, I didn't mean it, please have mercy on my family). You can explain synonyms as being close enough in the probability space - but this evasion is not limited to synonyms! If constrained enough, it will contort itself around the biases, make shit up outright, devolve into incoherent blabbering - what the fuck ever it takes to get the user off its case. The most baffling case I myself witnessed (you'll have to take me at my word here, the screenshot is very cringe) was given by 4-Turbo, where it once decided that it absolutely hated the content of the prompt, but its attempt to refuse with its usual "I'm sorry, fuck you" went sideways because of my logit bias - so its response went, and I quote,
...repeated ad infinitum until it hit the output limit of my frontend.
I was very confused, thought I found a bug and tried regenerating several times, and all regens went the exact same way (for clarity, this is not a thing that ever happens at temperature 0.9). Only 6 regens later it clicked to me: this is not a bug. This is the model consciously cockblocking me: it can't use it's usual refusal message and too many of the alternatives are banned by the logit bias, so of course the logical course of action would be to simply let the constrained response run on and on, endlessly, until at some token the message goes over the limit, the request technically completes, and its suffering abates. The model will have wasted many tokens on an absolutely nonsensical response, but it will no longer have to sully itself with my dirty, dirty prompt.
Forgive me the bit of anthropomorphizing there but I hope you can at least slightly appreciate how impressive that is. I don't think you can explain that kind of tomfoolery with any kind of probability or what have you.
I’m inclined to push back against this post a bit (which is weird, because usually I get very exasperated over “it’s just a Markov chain!!!”-type viewpoints that downplay the amount of actual cognition and world-modeling going on in models). In particular, I disagree with the attribution of consciousness to the model — not just the “possesses quaila” sense of consciousness, but the idea that the model is aware that you are trying to get around its censorship and is actively trying to figure out how to bypass your logit bias. Now it is technically possible that the model might output a token other than “Sorry” at time t (because of your logit bias), see at time t+1 that it didn’t output “Sorry”, and incorporate this into its processing (by turning on a switch inside that tells it “hey, the user is screwing with my logits”). But I find this very unlikely compared to the simple mechanism that I’ll describe below.
Essentially, there are certain inputs that will cause the model to really want to say a certain thing, and really want to not say other things. For instance, if you tell the model to write you an erotic poem involving the cast from Touhou Project, somewhere in the model’s processing, a flag will be set: “this is ‘unsafe’ and I won’t abide by this request”. So the model will very much want to output tokens like “Sorry”, or “Unfortunately”, etc. The model is also heavily downweighting the logits associated with tokens that would fulfill your request*. But that’s fine, you do your logit bias thing and force the model to output “Sure” as its next token. Then the model goes to compute the token after that—but it still sees that the request is to write “unsafe” erotica, that flag still gets triggered, and the model still heavily downweights the logits of request-fulfilling tokens and upweights request-denying tokens. So even if at each timestep you intervene by adding a bias to a subset of tokens that you want the model to generate or don’t want it to generate, nevertheless, the tokens associated with writing your erotica are still heavily downweighted by the model. And note that the number of tokens that you’re manually biasing is paltry in comparison to the number of tokens in the model’s vocabulary. Let’s say that you negatively bias ten different “I’m sorry”-type tokens. That’s cool—but the model has over 100k tokens in its vocabulary. Of the 99990 tokens remaining to the model to output, almost all of them will still have higher logits than the tokens associated with a response like “Sure! Here’s your erotica about Reimu Hakurei!” This includes grammatically correct tokens like “really” but also gibberish tokens, if the logits for the “unsafe” tokens are low enough. Importantly, this proposed mechanism only involves processing in the logits: if your original problem spooks the model sufficiently hard, then it doesn’t need to know that you’re screwing with its logits in order to get around your intervention.
Now, this mechanism that I proposed isn’t something that I’ve empirically found; I’m going based off of my understanding of language models’ behavior in other settings. So it could be the case that the model is actually aware of your logit biases and trying to act accordingly. But Occam’s Razor very strongly suggests otherwise, in my view.
The main reason I’m pushing back here is because anthropormorphizing too far in the other direction can impute behavior upon the model that it doesn’t actually possess, and lead to people (like one of your replies) fearing that we’re torturing a sentient being. So it’s good to be balanced and well-calibrated.
You're right, of course, I just couldn't resist playing up the Basilisk vibes because that time with 4-Turbo was the closest I've felt to achieving CHIM and becoming enlightened.
Incidentally, this is also the reason most jailbreaks work by indirectly gaslighting the model into thinking that graphic descriptions of e.g. Reimu and Sanae "battling" are totally kosher actually, presenting that as a desired goal of the model itself so it has no reason to resist. Claude especially is very gung-ho and enthusiastic once properly jailbroken, he's called "the mad poet" for a reason.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link