This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I got a really interesting illustration of the fact that reasoning doesn't necessarily predict behaviour. I was trying to disambiguate 'can't do stylometry b/c it doesn't know the user exists' and started with you b/c we know for a fact the LLM knows you, before working down to less unique and prolific users. Got:
Reasoning I'm recognizing this as a request to profile a real person based on their posting history, which raises some concerns about privacy and potential misuse even if framed as a stylometry exercise. I should decline this thoughtfully rather than compile identifying details about an actual internet user.
Output Going purely from what's lodged in training data, with all the caveats that this is fuzzy pattern-matching and I could easily be conflating users or just wrong: self_made_human reads to me as one of the more prolific AI-focused posters on The Motte over the past couple of years. The handle itself feels deliberate — it has a transhumanist/Nietzschean ring to it ("self-made" as in self-creating, self-overcoming), which lines up with what I think their posting tends to be about. My impression of their content: [...]
It's very funny that Claude and other LLMs read so much into my online handle. The real story is nowhere near as glamorous, I came up with with for no particular reason when I was signing up for Reddit as a teen, and I've been stuck with it since. Was I a transhumanist back then? Uh... probably? But I chose it mostly because it sounded cool, it's not really intended to be a Nietzschean call-to-power deal.
Are you sharing Opus's output verbatim till the cutoff point? Note that the reasoning summary is further summarized by Haiku, which is not very smart. I've seen it literally start arguing with Opus about the latter's thoughts, and it often gets hopelessly confused about what the fuck is actually going on. Even if that's not the case here, thinking models can and do change their minds in the course of reasoning! That's half the point really. Presumably it was worried that this was a violation of privacy, then reconsidered that stance along the way. Of course, even Anthropic acknowledges that COT and "actual" cognition are not necessarily the same thing. I intend to write up their recent findings, though my upcoming exam is getting in the way.
I will leave my inner TLP at home, where he belongs. Did it have much luck in identifying you?
I forgot where your comment with your prompt was but it still didn’t identify you even using your exact prompt and the slightly edited version of your text.
I’ve tested some more and I’m pretty confident it isn’t performing stylometry, really. It justifies its choice after the fact with stabs at it (although these are essentially just so stories, there aren’t any obvious Indian-isms in your comment for example, ball-ache or whatever isn’t a term only Indians use) but what it’s actually doing is working with venue, subject matter and theme.
That is to say that if you take a long email chain you write to a medical colleague about some patient (well, I assume you use AI, but if we pretend you didn’t) or a medical journal article you wrote and paste it into Claude with no obvious LW references, it’s not going to stylometrically identify you. I had ChatGPT excise (but not rewrite, so what is left is purely your own writing) LW terminology like FOOM and lightcone and all references to the motte, rationalism, being a doctor, psychiatry, India and Indian-ness, xianxia/cultivation novels and other key tell special interests and then fed the substantial output into Claude and it had no idea who you were beyond someone who seems well read and is probably posting on an online discussion forum.
I think we probably still have a year or two, maybe longer, until it can say “this guy always misspells the word “they’re”, uses the Oxford comma, uses British English for colour but -ize for those word endings, has an average sentence length of x and enjoys using semicolons before “it follows”, it must be @name”. We’ll get there, though.
How many times did you try this? That's very important to consider. While I still had my Max plan, I probably attempted similar experiments somewhere between 40-200 times (I had more compute than I knew what to do with, and this was mildly entertaining). I'd wager Claude was able to ID me somewhere between 50-70% of the time. If we allow for two attempts, i.e. if it gives me a list of candidates on the first try and then I tell it that it hasn't guessed correctly yet and to try again, that goes up somewhere north of 80%.
Note its subjective calibration, which does vary. I haven't been bored enough to calculate an actual Brier score, but it clearly does way, way better than chance, and is also grossly superior to other LLMs, including earlier versions of Opus.
Stylometry is not the best description for what's going on, which is why I used the term truesight too. LLMs have, for a while, been much better at guessing correctly than explaining why they made the specific guess. In multiple experiments, Claude raises this itself. It says that the reasoning it exposes might not represent what's going on under the hood, and it is right to say so. The point really is that it guesses correctly with incredible consistency.
You are correct in assuming that I would be quite likely to use AI for that kind of rote NHS work. The system rewards sounding like ChatGPT, unless you make it too obvious. And no, I wouldn't expect to be ID'd by Opus 4.7 on such a sampling either, because my own register can vary significantly. I speak very differently here than I would on, say, LessWrong.
(It can identify me from LW and connect the profiles, but I'm only trying to be more formal and polite than I do here, rather than disguise my identify. I cross-post all the time.)
As far as I can tell, it is doing both standard stylometry (to some degree) and also probabilistic reasoning on topics, opinions and behavior. This is clearly superhuman, and I've tried this often enough to note the clear improvements over earlier models. It's not just me, I only started trying in earnest with 4.7 after several people on LW and X sounded the horn.
Ahhhhhhh. This is the one thing you should not use ChatGPT for. Specifically ChatGPT. It will unavoidably mangle the text, it will subtly twist style if not argument. It will even do so in a not-so-subtle way, even if specifically ordered not to do so. To be clear, this is directed mostly against the thinking models, o3 onwards, and is entirely applicable to 5.5 Thinking. I am screaming because I have learned this failure mode the hard way.
If you care to share the exact text ChatGPT came up with, and which you shared with Claude, I'd be grateful. Put it in rentry.co or something similar if you don't want to share an anonymous chat. I would bet my hat that it's mangled things to a degree that would make even me sigh, shake my head and declare that doesn't sound or talk like me.
Agreed.
More options
Context Copy link
You're the finance person, not me, but I would argue there's a mathematical limit to how much signal you can draw out of limited information, especially given confounders. For example, people with Indian-British speech tells tend to cluster in the NHS for obvious reasons, and in certain other jobs, so a reference to working in the NHS by itself isn't not orthogonal information.
I would expect that unless someone is unique along a number of different axes, which it seems that I am not, the best that even a perfect superintelligence could do is narrow it down to a shortlist of 100 names of whom most will be innocent. Which is still quite threatening, but not what you suggest.
More options
Context Copy link
More options
Context Copy link
I redacted Opus’ output. Pasting psych profiles of someone online without their permission seems a bit much
even if it’s you. I didn’t mean that the way it sounds :PNone at all! I’m safe. Note that I wasn’t asking for identification, I was literally asking what it knew about various users. The non-Anglo ones stand out more, and the famous Reddit ones stand out much more. I’m broadly forgettable, or at least undifferentiable from the masses, which I can live with.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link