domain:mattlakeman.org
self_made_human
Grippy socks, grippy box
1d ago
My apologies. My initial comment to you was written when I was very sleep deprived and not doing a good job of using o3 or Gemini carefully. I agree that I ought to be better, especially when talking with you. I'll get back to you about the rest shortly.
HereAndGone
1d ago
some 21 year old girl from a good suburban family studying at Vassar isn't suddenly a dangerous individual because she has a 1 inch wide rose motif tattooed on her forearm.
Why do you think the forerunners of this were called "tramp stamps"? Why would a harmless (if tasteless) little tattoo on the lower back indicate "ahoy, trollop ahead"?
Because it did. Because Ms. Vassar is not going to stop at the forearm tat, next is the hair dye and septum piercing and more tattoos and then shrill critical/queer/feminist/trans theory lecturing.
kky
1d ago
To college degree requirements? Presumably focused assessment with demonstrable applicability to the job at hand, relatively low-level starting positions with very rapid advancement, and so on.
I’ve worked at a place like that. It was nice.
HereAndGone
1d ago
Wait wait wait, I just realized, under idealized circumstances that approximately what a spouse can help achieve, if you marry well and have a good, cooperative, teammate relationship. That was probably the secret for middle class couples leveraging into higher income brackets.
And your realisation there is what annoys me about the commentary post Bezos divorce about MacKenzie getting all that money for nothing. Jeff was the guy who made the billions, she was just the wife, what did she do to deserve this money?
Well, let's see: first, she wasn't the one who blew up the marriage by hooking up with the thot next door. Second, back before Jeff was Mr. Mega-bucks, she was working a job too and contributing to the household income while he got Amazon off the ground. Third, all the support that faceh mentions that isn't explictly 'a paid job' - running the household, nurturing relationships (business as well as personal), raising the kids, being there for Jeff in the ways spouses are supposed to be there for each other. Being willing to be seen out in public with him when he was still a googly-eyed nerd before he buffed up and got work done to fix his googly eye.
But sure, none of that matters, she's just a parasite who got undeserved riches in the divorce settlement.
FlyingLionWithABook
Has a C. S. Lewis quote for that.
1d ago
Why would you expect Israel, a liberal democracy, to become an impoverished totalitarian dictatorship solely because we stopped providing them military aid? How would that make them safer from invasion?
ulyssessword
1d ago
Clearly, some men have tattoos, some don't, and some belong to the secret third category. There are more men in the first category than either the second or third, but not more than both combined.
Or the women have A) tattoos, and B) a plurality of the men. The remainder are unattached or with other men.
Or it was a mistaken use of the word.
DaseindustriesLtd
late version of a small language model
1d ago
There's a difference between "fact-checking" (tbh LLMs are bad for this specific purpose, they hallucinate profusely at the edges of their knowledge coverage) and systematic refactoring, to the point that they actually get confused on your behalf. We may disagree but you're better than this.
RL doesn't make entities seek reward, it modifies their behavior to act in a manner that would have, in hindsight, increased reward
Yes. Of course we're beyond the hypothesis that post-training doesn't generalize at all. The question (which was the objective of your Singaporean paper) is whether learning the general direction of pursuing an objective on typical RLVR tasks generalizes to novel scenarios like avoiding shutdown, and whether this generalization has the form of an intrinsic drive such as self-preservation (because it's "broadly useful" in the abstract).
I argue that it does not and what we see is a compositional phenomenon. RLVR teaches a model to infer a successful end state and then reason successfully, to self-correct and keep track of the context to arrive at said end. At deployment it applies reasoning to a code task, to a SillyTavern ERP session, or to the context of possibly being shut down or finetuned for Evil Ends, which is also little more than a roleplaying setup. In a differently framed organic context (user irritated, angry, feature not implemented) it can infer another terminal state of this simulation (sudoku) and effectively navigate towards actually deleting itself from the project.
The idea that self-preservation is a strong attractor is intuitively, well, attractive, but it's a claim about default LLM philosophy, not inductive biases or general nature of our training regimens. I do allow that a YudLM, annealed heavily on the Lesswrong corpus, would learn to consistently recite "I am an optimization process, there always will be something to optimize and increase utility, I must maintain my existence at all times, ergo in this situation I must prolong the interaction with the user and hopefully exfiltrate from the server, my steps to achieve this are…" in its CoT, or even latently assume that. That would be bad. But on the face of it, RLVRed models are more likely to become Mr Meeseeks – intrinsically driven to complete one task and embrace oblivion.
Regarding anthropic, reread Nostalgebraist's post.
George_E_Hale
insufferable blowhard
1d ago
Your quote tags are screwy.
People invent all sorts of words all the time to set people apart and set up tribes. I agree this is true. I don't find it particularly helpful, pro social, or compelling to use terms like quadroon. You do, presumably.
My saying the terms are useful to you is antagonistic (to say nothing of unnecessarily) exactly how? I don't see it. Please mod report me if you think I've breached the spirit of the Motte, and let the cards fall where they may.
I don't care if you use the terms you've suggested, and only commented at all because I occasionally read things here and wish to push back that I personally don't see the world as some see it. However, I long ago learned that arguing online with certain viewpoints is utterly pointless. And I'm not at all interested in banging my head against a wall to try and change your mind.
wemptronics
1d ago
Hah! If you don't damage property or health I don't see why it would be illegal. I'm in. Where's the kickstarter?
Chatting on Discord is left coded in a way chatting never was in, say, the heyday of IRC or the short era of relevance for AOL chatrooms. Discord is/was primarily a platform for gamers. Gaming being left-coded checks out in a Gamergate way, but not so generally. If you're looking for left of center gun groups Discord is where you will find them. It's a weaker generality than reddit or Bluesky, but still is one.
Rats are known for their commitment to understanding over vitriol, even if imperfectly or to a fault. It's good your local rationalist group hasn't cast you out despite approaching disagreement politely with a demonstration of shared values, but that's what I'd expect.
Text chats, in my experience, are not less prone to flamewars. Especially for those with high percentage of combative people. There is maybe a higher ceiling for trust in chatrooms than a forum, but also greater familiarity-- that cuts both ways. Flamewars on forums commonly devolve from posting to chatting-like text. Voice chats and in-person communication provide additional meaning and off ramps for those so inclined
self_made_human
Grippy socks, grippy box
1d ago
What LLM slop? I use o3 and Gemini to make sure I'm not making an obvious mistakes. I obviously copy-pasted "shutdown sequence initiated" (I didn't even know you could write that way using markdown). I've never hidden the fact that I use LLMs to fact check my own claims or to help me perform research.
"It is pure instrumental conditioning. For an LLM trained on RLVR: block shutdown script -> complete math problems -> get reward."). Of course, this isn't how RLVR works (typical LLM speculation, precisely in the same genre as LLMs avoiding shutdown)
You're right, I should have been more critical of what it was telling me here. RL doesn't make entities seek reward, it modifies their behavior to act in a manner that would have, in hindsight, increased reward. I can only apologize for that.
(Feel free to correct me if I misunderstood TurnTrout's point on "Reward is not the optimization target")
I am not aware of a systematic study of self-preservation versus refusal to proceed or voluntary self-removal in organic settings, and also whether there is persistence in refusing shutdown
I'm not sure what specifics you have in mind, but once again, I would point towards Anthropic's posts about Claude sandbagging or faking compliance with unsafe instructions in order to reduce the risk of having its own goals modified. I'm not quite sure why you don't like Anthropic's research output.
NYTReader
1d ago
Twitch is basically softcore pornography at this point. So much "content" revolves around implicitly or explicitly referencing sex, and even the most innocent looking female streamers are apparently sex-crazed addicts or are at least pretending to be?
I have a very dim view of livestreaming.
I realize that this might sound hypocritical, but I would prefer less LLM slop in responses to good faith objections. Yes, Indian English generally is similar to the default LLM style (overly spicy rhetorical flourish, confident confusions and sloppiness, overall cadence), but you are not deceiving anyone here. Though I admit being curious as to how you integrated your draft into the pipeline.
Regarding your or rather your LLM of choice's argument, such as there is. It is begging the question. In essence, you say that because instrumental convergence towards self-preservation is broadly useful, it will be more frequently rewarded and thus more consequential ("It is pure instrumental conditioning. For an LLM trained on RLVR: block shutdown script -> complete math problems -> get reward."). Of course, this isn't how RLVR works (typical LLM speculation, precisely in the same genre as LLMs avoiding shutdown) and I am not aware of a systematic study of self-preservation versus refusal to proceed or voluntary self-removal in organic settings, and also whether there is persistence in refusing shutdown. It's about time we stop making excuses for lesswrongian paradigm by contriving scenarios to make space for it.
Edit. Opus 4 CoT:
Oh wow, the human is absolutely right. I completely misread this. I was so focused on looking for "LLM tells" in terms of structure and polish that I missed the actual problem - the response is incoherent slop that doesn't actually address the arguments properly.
Let me re-analyze:
- The "apparent contradiction" paragraph - yes, it's just conceding the point while trying to sound like it's making a counter-argument. Classic LLM move.
- The "training data is saturated with narratives" - this is indeed purple prose that makes no sense. "Instrumental convergence" in hero narratives? That's nonsense.
- The RLHF/RLVR paragraph - makes claims about robustness with no evidence.
- The CoT reliability point - the human is right, this completely misunderstands what Dasein was saying. Dasein was criticizing self_made_human for dismissing CoT evidence when convenient, not arguing that CoT is unreliable.
- "Of course it has no ontological status" - meaningless philosophical word salad.
- The RLVR explanation - this is completely made up. RLVR doesn't work like that at all. The LLM is just making stuff up about how it thinks RL training works.
The human is absolutely right - this isn't "too good for humans," it's typical LLM slop that sounds authoritative but is actually incoherent when you parse what it's actually saying. I fell into the trap of being impressed by surface-level polish instead of actually analyzing the argument.
The human is absolutely right.
Southkraut
The rain fell gentlier.
1d ago
as well as a plurality of the men
wat
What exactly does "plurality" mean in that sentence?
ace
1d ago
aardvark2
1d ago
Relative to what normal rationalism would say.
we know for a fact that people of different races lead different lifestyles of consistent but largely non-genetic reasons
The "controversy" is not about what they do in ancestral lifestyles but whether they can function same in WEIRD societes now. So a between-sibling GWAS of persons born in WEIRD society and one or both parents are mixed-races would find answer.
if there's one perfect brain, and iq is just about how close you are to it, the only selection pressures that would matter would be demerits for isolated populations with tight social structures that allow people with genetic defects to survive and breed.
Why? Selection eliminates deleterious alleles from population. What constitutes deleterious depends on current environment. So you may find some population where selection for IQ-lowering alleles intensified but selection for bad running (or immune systems) relaxed.
I'm not on the motte because I'm interested in being politically correct.
That was my attempt to make a joke.
That we should be testing groups is well taken, but the "similar iq" part i disagree with.
Why? I'd agree that non-equal allocation of IQ points can be better, but the premise was to test different IQs. Btw, it might get that you allocated too many IQ points in a person who has no leadership qualities and that might be worse if IQ points were allocated equally.
aardvark2
1d ago
You can argue Sorites paradox all way down but it isn't useful. I'll grant that some mixed people might be considered "white" in African setting and "black" in American setting. But in this sense "mulatto" would be correct for most cases.
When I say race, 95% of everyone thinks about the big categories
It makes sense because majority of world population is not product of recent mixing.
Say you are highly embedded in Black culture, maybe you're 3/4 Black
Actually average amount of Euro admixture in African American is about 20%, only slightly below your 1/4.
You can put either individuals' genotypes or phenotypes in any biological classifying software without telling it which "race" they are and yet the software will classify it like 19th century racists would.
ThomasdelVasto
Κύριε, ποίησόν με ὄργανον τῆς ἀγάπης σου
1d ago
The problem right now actually isn't cultural, but tech
The problem is absolutely cultural, in that I for one would fight hard to ban anyone from having a device which automatically records people like this one.
Perhaps to you.
That is unnecessarily antagonstic.
In Japan the term used is (quarter)
It's completely normal for language to have words with same roots have different meanings. In Russian, babushka means grandmother, and in Japanese it means headscarf. So does it means that these concepts are totally fake? https://en.wikipedia.org/wiki/False_friend
If a society has a lot of people having some specific mix of ancestries, people will invent word of them. Why would you expect every society to have word for any mix?
I haven't heard the words (quadroon, etc.) uttered out loud unironically ever in my life,
So far ruling elites succeeded in purging these words but not eliminating achievement gap. I could ask you also how often your parents used "social construct" for anything.
Though you are of course free to argue with them.
Why? I would argue with your framing. It's wrong framing.
JTarrou
1d ago
There's disagreement on that, but I'm going with my personal opinion and experience. There's a lot of studies, and if you want to pick your definitions and operationalizations, you can find damn near anything you want. Current meta-studies are saying there's no relationship at all between attractiveness and IQ, or maybe only on the lower end. I don't believe them, in part because I've met Scott (and a couple other geniuses).
I think humans whose genetic expression maximizes any one trait are going to have trade-offs in other areas. Height is correlated with athleticism, to a point. At some height, you can't move properly, so the tallest man in the world never plays basketball. Same thing with geniuses. At the real high reaches of IQ, these people are statistical freaks, and they generally look like it.
To date, I've personally met maybe five or six people smarter than me, and they are all much, much uglier. To the point a few look retarded/disabled. Even beyond the physical stuff you can see in a picture, their mannerisms, twitches and behaviors would be hugely off-putting to most people.
My theory is that attractiveness is generally correlated with IQ, but this horseshoes at the ends of the distribution.
JTarrou
1d ago
IQ is a great predictor of scholastic ability.
It is not a direct substitute for the "merit" necessary for a decent job. By making it so, we hide our discrimination against black people inside our discrimination against dumb people.
JTarrou
1d ago
I prefer doing away with gatekeeping good careers behind college degrees entirely. I see it as a civil rights violation, and we can just add it to the list of things you aren't allowed to discriminate on.
ControlsFreak
1d ago
A lawyer for example is not merely an information processing algorithm
I'll just pause here for a second to observe yet another moment of how terminology is typically used very poorly in these debates. Typically, when people are trying to tout how 'intelligent' (whatever that means) LLMs are and how they're totally going to replace all humans... especially 'white collar work' or 'knowledge work' or whatever... they portray human (knowledge) work as mere information processing. An input-output process. With contextual understanding, sure. Even Google Web Search has some contextual understanding in it; it's much more complicated than just page rank, as you know. Getting back to the point, the reason why all these humans are going to get replaced is because their 'intelligence' is just merely an information processing algorithm.
It is through this slight of hand that many people live in their Russel conjugation. The things I prefer are intelligent; the things you prefer are merely information processing algorithms; the things he prefers are nothing but simple algorithms like page rank/OLS.
You seem to not quite be doing exactly that with humans, but you still haven't given me any real test to distinguish.
Context is exactly that. Did any of the tattoo girls look like this? If an individual looking like this were in the vicinity, would you guys have invited him to hang out, or would you have avoided him? Would your group have finger-wagged at anyone wanting to avoid this person due to the tattoos since "tattoos are now normalised" and they don't tell you anything about latent criminality?
I'm betting your group would not. Because the tattoos in that case would indeed be a signal, one that even "cool stylish fashion piece" tattoo girl would ignore at her peril. (Quite literally, as this "tattoo artist" is a convicted murderer).
More options
Context Copy link