Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.
- 181
- 1
Jump in the discussion.
No email address required.
Notes -
If I gave you a snippet of Shakespeare and asked you to guess who wrote it, I expect the Bard would be one of your top choices. How are you doing that if you don't consult Google or your Shakespeare box set?
Each token that the model sees in training updates its view of what sort of things are associated and in what way. Elements of style or topics may be clustered somewhere in the high dimensional latent space with the corresponding authors.
I would have similar examples in my memory. What does that memory look like for an LLM if not reference to a database? I just asked the Edge default copilot for "the 3rd line from Shakespeare's 31st sonnet", and for "without searching the web, the 5th line from Shakespeare's 41st sonnet" and it produced both without any trouble.
Are you suggesting that Copilot is re-deriving particular lines from the sonnets from first principles?
This is a sincere question, I honestly don't know how the nuts and bolts of these things work.
The memory is the weights of the model. The first stage of training an LLM is next token prediction - the LLM is shown a block of text and is trained to produce the next word (technically, part of a word, but that's not important). Internally, the model manipulates the numeric representation of the input tokens as points in a high dimensional space. The model produces a kind of probability distribution over all the words it knows for what the next word might be, and the weights of the model are adjusted so that it's more likely to expect the correct word.
The result is that every token, with some context before it, that the model sees leaves some kind of "impression" upon the model. The details are fuzzy, but things like word choice or style are probably represented in some regions of that high dimensional space, which is what lets the model say that something "sounds like" Shakespeare.
Anthropic have some public research around trying to look into a language model to see what's going on in that latent space and how neuron activations relate to concepts. It may be difficult to read without any ML background, but fortunately you can now feed it in to an LLM and have it explain anything you don't understand.
LLMs being described as having ‘memory’ of things in the training set is almost certainly far closer to the colloquial, human understanding of what ‘memory’ is than either of the above concepts are to computer memory or an encyclopedia.
So if someone colloquially says the LLM has its training set in its memory this is no less accurate than saying that you remember what the water cycle is even though you cannot recall the precise page and content and diagram of the school textbook that you learned it from. Or why you can identify a line of text written in ‘Trump voice’ even though you cannot exhaustively list every Trump tweet you’ve ever seen.
It's one thing to say something is in the LLM's memory. It's quite another to say it's doing some kind of lookup in a database or into the training data.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link