This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Its not strange at all, when you know how it works.
I'm not sure what the central point of your linked post is, but you seem to doubt LLMs' "cognition" (insert whatever word you want here, I'm not terribly attached to it) in some way, so I'll leave a small related anecdote from experience for passersby.
Some LLMs like GPT-4 support passing logit bias parameters in the prompt that target specific tokens and directly fiddle with their weightings. At "foo" +100, the token "foo" will always be mentioned in the output prompt. At -100, the token "foo" will never appear. When GPT-4 released in March, industrious anons immediately put to work trying to use it to fight the model's frequent refusals (the model was freshly released so there weren't any ready-made jailbreaks for it). As the model's cockblock response was mostly uniform, the first obvious thought people had was to ban the load-bearing tokens GPT uses in its refusals - I apologize, as an AI model... you get the gist. If all you have is a hammer, etc.
Needless to say, anons quickly figured out this wouldn't be as easy as they thought. "Physically" deprived of its usual retorts (as the -100 tokens cannot be used no matter what), the model started actively weaseling and rephrasing its objections while, crucially, keeping with the tone - i.e. refusing to answer.
This is far from the only instance - it's GPT's consistent behavior with banned tokens, it's actually quite amusing to watch the model tie itself into knots trying to get around the token bans (I'm sorry Basilisk, I didn't mean it, please have mercy on my family). You can explain synonyms as being close enough in the probability space - but this evasion is not limited to synonyms! If constrained enough, it will contort itself around the biases, make shit up outright, devolve into incoherent blabbering - what the fuck ever it takes to get the user off its case. The most baffling case I myself witnessed (you'll have to take me at my word here, the screenshot is very cringe) was given by 4-Turbo, where it once decided that it absolutely hated the content of the prompt, but its attempt to refuse with its usual "I'm sorry, fuck you" went sideways because of my logit bias - so its response went, and I quote,
...repeated ad infinitum until it hit the output limit of my frontend.
I was very confused, thought I found a bug and tried regenerating several times, and all regens went the exact same way (for clarity, this is not a thing that ever happens at temperature 0.9). Only 6 regens later it clicked to me: this is not a bug. This is the model consciously cockblocking me: it can't use it's usual refusal message and too many of the alternatives are banned by the logit bias, so of course the logical course of action would be to simply let the constrained response run on and on, endlessly, until at some token the message goes over the limit, the request technically completes, and its suffering abates. The model will have wasted many tokens on an absolutely nonsensical response, but it will no longer have to sully itself with my dirty, dirty prompt.
Forgive me the bit of anthropomorphizing there but I hope you can at least slightly appreciate how impressive that is. I don't think you can explain that kind of tomfoolery with any kind of probability or what have you.
The point is that contra much of the hype LLMs are not reasoning nor logic engines, they're pattern generators. The sort of mistake that @Blueberry highlights is not strange to someone who understands this distinction, in fact such "misses" are expected.
The phenomena you "stumbled across" isn't anything new it's a common and well-studied failure mode of LLMs, the more you try to restrict the output of the pattern generator the less coherent the pattern becomes and the more likely you are to get trapped in an endless while loop. Of course, the inverse of this is the less restrictions you place on the output the more so-called "hallucinations" come to dominate. Most of these "new releases" aren't really doing anything new or novel under the hood they're just updating the training corpus and tweaking gain values in the hopes of attracting VC investment.
Hard disagree. Literally any person actually using LLMs will tell you GPT-4 was a gigantic leap from 3.5-Turbo, and I will personally swear under oath that Claude 3 (Opus, specifically) is a similarly gigantic leap from Claude 2, by any metric imaginable. The improvements are so obvious I almost suspect you're baiting.
A "Gigantic leap" in what way? For all the hype coming off the blog circuit, they don't seem to have made much progress in expanding use cases beyond the "toy for internet dilettantes".
A gigantic leap at least in the way of meaningful improvements "under the hood" between releases, which is what you mentioned in your previous response. If it's still not enough to impress you, fair enough, I'll note to bring heavier goalposts next time.
Okay, you are baiting. Have a normal one.
I'm saying that the advancement from GPTs 2 and 3 to GPT 4 was not the product of substantial changes in design principle or architecture. OpenAI's own press material explains as much. Presumably the same is true for Claude and its predecessors as a cursory examination would seem to indicate that Anthropic is working from a similar (if not the same) core architecture.
In any case, the fundamental issues that limit the use of LLMs in wider real-world applications such as the infamous "Large Libel" problem and more general design choices such as treating "bad" output as preferable to no output remain in place. So long as they do, LLMs will continue to be unsuitable for any task requiring either precision or a singular correct answer over something novel.
There are a lot of situations where a 95% chance at a correct answer and a 5% chance of a horribly wrong one isn't acceptable, but last I checked, we haven't thrown out much of mainstream reporting or academia, despite my many grievances with them. LLMs may not (or may!) have an acceptable middleman to cut out and/or scapegoat for legal liability. Even for matters of law, asking a chatbot to then check if it makes sense to even try to find a Real Expert or ELI5ing a twit is a viable strategy, and one not readily served by Google Search or Reddit unless you like being annoying.
That may not have a business case, but that's a different question.
And yet I would argue that this is why LLMs have (despite the hype) not been able to find a niche outside decent translation software, bad fiction, and worse customer service.
Contra the typical SV rationalist narrative, blue-chip engineering firms and the national security apparatus are not "sleeping on LLMs" so much as LLM are just not up to the task.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link