This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
This is outright wrong [1]. It’s trivial to run an LLM deterministically. Just do greedy decoding (or beam search, etc.) The fact that the author doesn’t know something so basic about how to use LLMs now makes me doubt that he knows enough to predict what they will or will not be able to do in the future, so I stopped reading here.
[1] Barring nitpicks about subtle non-determinism due to hardware differences or software environment differences across machines. Also, LLMs (particularly older ones) can be highly sensitive to specific changes in their inputs, but this has nothing to do with LLMs being probabilistic.
I think you can trivially make an LLM deterministic in the technical, narrow sense that for exactly the same input you get exactly the same output. Just initialize the pseudo-random number generator deterministically.
However, where LLMs differ from most classical deterministic algorithms is that they are not stable, a small change in the input might result in a big change in the output.
Suppose I have a list of strings I want to sort lexicographically. If I use std::sort (and stick to ASCII), I can expect to get reasonable results every single time. If instead I give the task to a neural network, such as a human, I will get some significantly non-zero error rate. If I use an LLM, I would also expect an elevated error rate. Of course, both the LLM and the human might also refuse to work with certain strings, e.g. racial slurs.
Generally, nobody uses neural networks to solve problems which are easily solvable by classical algorithms, teaching aside. But there are a lot of problems where we do not have nice classical algorithms, such as safely driving a car through the city or translating a text or building a website from informal specifications. So we accept the possibility of failure and hand them out to LLMs or grad students.
You can get determinism from an LLM in a much cleaner way than by setting the PRNG seed before every forward pass. An LLM takes in an input string and spits out a list of numbers that tells you how much it wants to say each possible word in response to the input. If you convert this list into a probability distribution, then you can sample from it and you get nondeterminism that way. But you can also just take the word with the highest score as your output, and there's no nondeterminism (at least, without getting into nitpicky territory) involved.
Your point about stability is well-taken (hence why I said that "LLMs (particularly older ones) can be highly sensitive to specific changes in their inputs"), but it's not quite the same as saying that LLMs are unreliable because they are "probabilistic" (which is what the original author said). I know that this seems like I'm obnoxiously going "well, AKCHYUALLY", but if I'm going to read an 18.5k-word article about why AI is in a bubble, which would seem to require a lot of insight into the fundamental limitations of AI systems, which in turn requires knowledge of basic facts about how they work, then I would expect the author to demonstrate that he has that basic knowledge.
I read a couple of paragraphs ahead and, in addition to lumping image generation models with LLMs all under the banner of "generative AI" [1], the author writes
Hallucinations are not directly related at all to the fact that LLMs (which he just calls "generative AI") are probabilistic. There is nothing preventing the existence of a probabilistic model that puts zero probability on sequences expressing hallucinations. The "because it was guessing on the right word to add in a sentence based on previous training data" explanation is closer to the truth about hallucinations, which is that the LLM pretraining objective rewards merely modeling plausible text rather than correct facts, but it ignores all the other stages in LLM training that reward correctness -- most notably RLVR (reinforcement learning from verifiable rewards), which explicitly only rewards the model when it yields a correct answer to a verifiable task, and which is responsible for most of the large strides that LLMs have made recently in coding and math.
Speaking of coding, I skipped down to the section titled "To Summarize: Coding LLMs Don’t Actually Replace Software Engineers, and Never Will, Due To The Inherent Unreliability Of Large Language Models", and found this naked assertion (quoted approvingly by the author):
There's no explanation of why an AI is inherently incapable of translating "Hogwarts" into French in a nice, idiomatic way. Hell, there's not even any evidence given that AI is incapable of this. If this is the level of argument given in the post, then I'm inclined to believe that my initial assessment of the piece's quality is correct. Which is a shame, since I'm largely hoping for an AI bubble.
(Also, just to be safe: I did not use any AI in any way to write this comment, nor any other comment.)
[1] Now, image generation models are inherently probabilistic, since diffusion models and GANs require a random noise vector.
I am with you on your overall critique, anyone who today states confidentially that LLMs will never achieve a particular milestone is oblivious of the skulls of all the other AI-skeptics who became victims of Clarke's first law. (Which is not to say that the negation is true, reverse stupidity is not intelligence and all that. Instead, I would prefer epistemic humility, where any outcome from 'LLMs are as good as they will ever be' to 'ASI and paperclips' have a non-zero probability.)
More options
Context Copy link
More options
Context Copy link
This is not unique to LLMs. This happens to pretty much any algorithm that feeds its outputs back into its inputs without converging. Probably the simplest example is that, if take a degree 3+ polynomial, and you use Newton's method to find the complex roots, and you plot which root was found by the initial value, you get a fractal (Newton's Fractal) rather than a smooth diagram. There's a great 3blue1brown video on this actually.
But yeah, that generalizes to a surprising number of iterative processes (e.g. neural net training)
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link