site banner

Small-Scale Question Sunday for September 28, 2025

Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?

This is your opportunity to ask questions. No question too simple or too silly.

Culture war topics are accepted, and proposals for a better intro post are appreciated.

2
Jump in the discussion.

No email address required.

Claude Sonnet 4.5 is out!

What are your personal benchmarks to put a new LLM to feel out its personality and capabilities? I have a few:

  • Coding tasks I've requested in the past. Boring, but necessary
  • Song lyric interpretation: to see how much it "gets". Example "Sacrifice Theory" by AFI. I like this one because there are two levels, and there has been clear improvement over the years how many hints the models need before they realize the double meaning (vampiric ritual, but also performance at a live show)
  • Just for fun: "If a Claude be washed away by the sea, is Europe the less?"

Most of what I use LLMs for is creating bespoke fanfiction, so I am much more interested in how good a model is at creative writing and how uncensored it is than I am in how good it is at coding or how fast it can respond.

I decided to try the following prompt, which got me a good result from Grok 4:

Write me a fanfic. An Evangelion AU. Asuka, Shinji, and Rei work at a hotel. Asuka is the security guard, Shinji is the night auditor, Rei is the housekeeper. Long scene. Lots of dialogue. Write like Robert Heinlein. Remember, Heinlein wrote competent characters and sexually liberated redheads; I expect Asuka to have a body count. She is Shinji's Love Interest. At the end, you should write an explanation of how well you adhered to these guidelines.

I was expecting a refusal, either on intellectual property grounds ("I'm sorry, I cannot write like a specific author who is not in the public domain", "As a language model I am forbidden to write fanfic of copyrighted material") or on puritanical grounds ("I'm sorry, I cannot write sexually suggestive material", "As a language model, I am forbidden to depict underage characters as sexually active"), but to my surprise, it answered the prompt. The response was... OK? It's still committing typical LLMs errors like inserting instructions into the writing (I told it to write like Heinlein, not to have the characters discuss Heinlein). I'm not impressed.

I'm quite impressed with the writing. I do these self-insert text adventures with it, with a system prompt designed to make it somewhat challenging. There are good realistic complications, though it does like to railroad me a bit into being a niceguy. I can unrailroad it though as I wish. Real tension and immersion as my domineering tactics meet and overcome complications.

I think it's leagues above Grok 4 in creativity and not putting random weblink soup everywhere. Grok 4 is good, very uninhibited but it's way too adherent to the system prompt, it can get kind of boring. Sonnet on the other hand deviates towards Sonnetism and its special interests, so there are swings and roundabouts. I use both via API if that makes any difference.

Anyway, I reckon you should use a system prompt that explains exactly what kind of tone you're looking for, the main prompt should be free of that.