Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.
- 106
- 2
Jump in the discussion.
No email address required.
Notes -
...
Weird. The Qwen distills don’t behave like that at all, they’re very po-faced and tend to give literary analysis essays about how the character would behave rather than just writing it. I haven’t tried the Llama distills or the main API.
One thing I have found (at least with the distills) is that it seems to work better for me as a one-shot story writer rather than as a chatbot. Giving it a set of logs and an instruction to ‘write the next part of the scene’ works a lot better than trying to talk to it conventionally.
Not weird whatsoever. V3 model that serves R1 as the foundation is very powerful and it has a vast breadth of knowledge thanks to being a giant ultrawide MoE, so it has no problem playing any role. Qwens are small and simply finetuned on some reasoning samples from R1, without even its reinforcement learning stage. Frankly it's a great surprise they learn math and code reasoning to such extent.
Though R1's writing ability is more of a surprise. Nobody expected reasoning to generalize in this manner, maybe not even DeepSeek. This is a big update about the capability of LLMs and general learning algorithms to produce some circuits that are actually just high-quality thinking and not narrow-domain memorized functions. It was always a spectrum of course but we've received evidence that LLMs are closer to "highly integrated cognition" end than assumed from their fragile behavior on new tasks.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link