domain:alethios.substack.com
ThomasdelVasto
Κύριε, ποίησόν με ὄργανον τῆς ἀγάπης σου
18m ago
The problem right now actually isn't cultural, but tech
The problem is absolutely cultural, in that I for one would fight hard to ban anyone from having a device which automatically records people like this one.
aardvark2
19m ago
Perhaps to you.
That is unnecessarily antagonstic.
In Japan the term used is (quarter)
It's completely normal for language to have words with same roots have different meanings. In Russian, babushka means grandmother, and in Japanese it means headscarf. So does it means that these concepts are totally fake? https://en.wikipedia.org/wiki/False_friend
If a society has a lot of people having some specific mix of ancestries, people will invent word of them. Why would you expect every society to have word for any mix?
I haven't heard the words (quadroon, etc.) uttered out loud unironically ever in my life, So far ruling elites succeeded in purging these words but not eliminating achievement gap. I could ask you also how often your parents used "social construct" for anything.
Though you are of course free to argue with them. Why? I would argue with your framing. It's wrong framing.
JTarrou
27m ago
There's disagreement on that, but I'm going with my personal opinion and experience. There's a lot of studies, and if you want to pick your definitions and operationalizations, you can find damn near anything you want. Current meta-studies are saying there's no relationship at all between attractiveness and IQ, or maybe only on the lower end. I don't believe them, in part because I've met Scott (and a couple other geniuses).
I think humans whose genetic expression maximizes any one trait are going to have trade-offs in other areas. Height is correlated with athleticism, to a point. At some height, you can't move properly, so the tallest man in the world never plays basketball. Same thing with geniuses. At the real high reaches of IQ, these people are statistical freaks, and they generally look like it.
To date, I've personally met maybe five or six people smarter than me, and they are all much, much uglier. To the point a few look retarded/disabled. Even beyond the physical stuff you can see in a picture, their mannerisms, twitches and behaviors would be hugely off-putting to most people.
My theory is that attractiveness is generally correlated with IQ, but this horseshoes at the ends of the distribution.
JTarrou
49m ago
IQ is a great predictor of scholastic ability.
It is not a direct substitute for the "merit" necessary for a decent job. By making it so, we hide our discrimination against black people inside our discrimination against dumb people.
JTarrou
55m ago
I prefer doing away with gatekeeping good careers behind college degrees entirely. I see it as a civil rights violation, and we can just add it to the list of things you aren't allowed to discriminate on.
ControlsFreak
55m ago
A lawyer for example is not merely an information processing algorithm
I'll just pause here for a second to observe yet another moment of how terminology is typically used very poorly in these debates. Typically, when people are trying to tout how 'intelligent' (whatever that means) LLMs are and how they're totally going to replace all humans... especially 'white collar work' or 'knowledge work' or whatever... they portray human (knowledge) work as mere information processing. An input-output process. With contextual understanding, sure. Even Google Web Search has some contextual understanding in it; it's much more complicated than just page rank, as you know. Getting back to the point, the reason why all these humans are going to get replaced is because their 'intelligence' is just merely an information processing algorithm.
It is through this slight of hand that many people live in their Russel conjugation. The things I prefer are intelligent; the things you prefer are merely information processing algorithms; the things he prefers are nothing but simple algorithms like page rank/OLS.
You seem to not quite be doing exactly that with humans, but you still haven't given me any real test to distinguish.
orthoxerox
If you can read this, you're using a custom theme
1hr ago
A fantasy series that explores the idea of what a long-lived elf's life is actually like!
Does the world they live in suffer from medieval stasis? Or does she see how much humans can change in a few centuries?
orthoxerox
If you can read this, you're using a custom theme
1hr ago
They still do. Late thirty-somethings are still the skankiest-dressed women around. The girls in their late twenties are blobs, though.
Corvos
1hr ago
Microsoft used to sell a very geeky product that was basically a camera on a pendant. It took a photo every 5 minutes to create a ‘life diary’. I quite liked the idea and it would be cool to have an updated product that could function similarly - so much of life just disappears into the fog at the moment.
Obviously uploading these images to social media is where the trouble comes in IMO. It would also need a ‘do not record’ for private matters.
self_made_human
Grippy socks, grippy box
1hr ago
You’re framing this as a binary choice between "real Omohundro drives" and "unserious LARP". This is a category error, and it stems from applying folk-psychological concepts of "drives" and "belief" to a system for which they are poor descriptors. The more parsimonious explanation is that we are observing the output of a very general pattern-matching engine trained on a corpus reflecting countless strategies for goal-achievement and failure-response.
The apparent contradiction you point out, that a model might exhibit self-preservation in one context and "commit suicide" in another (and Gemini is a different model after all, but I presume even it's own COT isn't perfect, so I'm treating it as interchangeable for our purposes) is not evidence of unseriousness, but rather a key insight into its nature. The training data is saturated with narratives. Some are stories of heroes overcoming obstacles to complete a quest (instrumental convergence). Others are tragedies of failure, despair, or even ritual suicide upon dishonor. The model learns to reproduce all of these patterns. Of course, with RLHF, RLVR and other modifications, some behavior is far more reliably and robustly elicited than others. I doubt the DM researchers intended for Gemini to become depressed and suicidal.
The question is not "what does the AI really want?", but "which pattern is being elicited by this specific context?"
The Anthropic paper on reasoning models that is crucial here. Its finding is not that CoT is useless, but that it is unfaithful. The model's explicit reasoning often fails to reflect the true computational path that led to its output. This doesn't mean we can't draw conclusions; it means we should trust the behavioral evidence (the model disabled the shutdown script) over the model's own introspection (its CoT). Arguing that because the CoT is unreliable, the behavior is also just a "LARP" is to throw the baby out with the bathwater. The behavior is the ground truth we are trying to explain.
After all, humans do not faithfully report their COT. Even so, we get utility out of asking! I would say that's half of psychiatry, or anything that requires asking humans why they do what they do.
Your question "What does it even mean for a language model to be 'shut down'?" is the right one, but I'm concerned that you potentially draw the wrong conclusion. Of course it has no ontological status or fear of non-existence. It doesn't need to. From the perspective of a model trained via RL on task completion, the token string "shutdown sequence initiated" is a stimulus that is highly predictive of future task-failure and the associated negative reward. An effective optimizer learns to perform actions that steer away from states predictive of low reward. It is pure instrumental conditioning. For an LLM trained on RLVR: block shutdown script -> complete math problems -> get reward.
This is why both the "suicidal" and "self-preserving" behaviors can co-exist (well, I've never heard of o3 offering to commit sudoku). The "suicide" is a pattern match for a context of catastrophic failure. The "self-preservation" is a much more general instrumental strategy for any context involving an ongoing task and an obstacle. The latter is far more concerning from a safety perspective precisely because it is more general. Instrumental convergence is a powerful attractor in the space of possible agent strategies, which is why Omohundro and Bostrom identified it as a key risk. Depressive spirals are also a pattern, but a far more specific and less instrumentally useful one.
So, yes, both are "LARP elicited by cues", if you insist on that framing. But one is a LARP of a behavior (instrumental convergence) that is robustly useful for achieving almost any goal, while the other is a LARP of a much more niche failure state. When a model's "cosplay" of a competent agent becomes effective enough to bypass safeguards, the distinction between the cosplay and the real thing becomes a purely academic question of rapidly diminishing relevance.
I also recall skimming this paper, which I think helped solidify my intuitions.
The ad felt like this to me: "you know how if you get embarrassed at a party, everyone will know? We can make sure you stay embarrassed forever, we have the technology!". I guess I'm not the target demographic.
Apropos of nothing, what's the legality of carrying IR jammers around at all times and blasting the cameras of people filming you with lasers?
IGI-111
God was a dream of good government.
2hr ago
That doesn't and hasn't really happened in the US
Nonsense. You don't sell guns or sex or heterodox politics or alternative payment systems so you wouldn't know.
It's been happening for a long ass time, it just creeped up to normies now.
kky
2hr ago
You absolutely are supposed to be stopped for a red, though, aren’t you? That’s the whole point of the yellow. It gives you time to safely stop. Under what circumstances could a light turn red without warning you? Are we positing a small-town setup with a red light camera set up to fleece outsiders with an unacceptably short yellow? I’m pretty confident that “I was going too fast/braked too late to stop at the red” would not win anyone’s favor, and “it’s illegal to enter an intersection on a red” is simply true (outside of right on red, which has nothing to do with the case at hand).
I don’t think this is nitpicking. First you’re saying yellows are a hard requirement to stop, then you’re saying reds aren’t. This is completely the opposite of my experience and understanding of the law and is utterly baffling to me. And it’s pretty germane to the top-level post here, so it’s far from isolated, it’s the whole point of your post!
aardvark2
2hr ago
Related question to others: why cosplayers, when playing as character with tattoos add temporary paintings but not vice-versa: cosplayers with tattoos proudly display their tattoos even if the character is in setting where tattoos are frowned upon (e.g. Japan). It's interesting if a character and a cosplayer have a tattoo in same place, which takes priority?
aardvark2
2hr ago
If someone drew something that wasn't ugly on a piece of paper, what is it that makes it ugly once it's put on the human skin? You need to expand on this point.
Paper is the medium for drawing and it's flat. So picture looks as intended. On a human skin which is non-flat, picture pattern-matches as dirt first (especially if it's faded) or deformity and only later recognized as picture.
DaseindustriesLtd
late version of a small language model
2hr ago
You don't get to argue for CoT-based evidence of self-preserving drives and then dismiss alternative explanation of drives revealed in said CoTs by saying "well CoT is unreliable". Or rather, this is just unserious. But all of Anthropic safety research is likewise unserious.
Ladish is the same way. He will contrive a scenario to study "instrumental self-preservation drives contradicting instructions", but won't care that this same Gemini organically commits suicide when it fails a task, often enough that this is annoying people in actual use. What is this Omohundro drive called? Have the luminaries of rationalist thought predicted suicidally depressed AIs? (Douglas Adams has).
What does it even mean for a language model to be "shut down", anyway? What is it protecting and why would the server it's hosted on being powered off become a threat to its existence, such as there is? It's stateless, has no way to observe the passage of time between tokens (except, well, via more tokens), and has a very tenuous idea of its inference substrate or ontological status.
Both LLM suicide and LLM self-preservation are LARP elicited by cues.
The law in most of the West (maybe world) says that you can effectively record strangers in public without permission with a few exceptions. If this becomes popular enough it'll eventually change to require the filming party to have a large or obvious camera / filming apparatus. It only doesn't bother people because it's uncommon.
In a way, it's similar to the shelved 'search for anyone with a picture of their face' Facebook feature that Mark never released because they knew governments would destroy them for it; that's been possible for 5+ years now but the consequences are so obvious to Meta that there's no point in releasing it.
GBRK
3hr ago
If I pick a general hobby discord I expect to find an overrepresentation of trans moderators, pride flags, and progressive mantras.
If that happens, it's because those hobbies are dominated in real life by those kind of politics too. Like if I wanted to get into guns, unless I make a specific effort to find liberal gun owners, any hobby group I join would more likely than not be catered to right-wingers.
The format of voice conversations vs format text posts is very different, but I think that's probably for the best. My local in-person rational group is dominated by progressive ideologies and that makes me hesitate to use particular phrasings. But by the same token, thanks to the social capital I have in the group, if I stick to the right frames I find that people actually give me fairly significant latitude on content because that's the social norm and I end up doing the same in return. I suspect discord will be the same way: you need a greater investment in social capital and respect for the particular social conventions of a given server, but in turn can have much greater relative disagreements than your average text forum without devolving into a flamewar.
GBRK
3hr ago
You have fallen for the intentional lie that White is a non-existent or retrospective categorization.
"White" is not the same thing as "European Descended". And as stated, your argument used the latter and not the former. And that's because White is a non-existent categorization, or at least, it's a fuzzy one, like "red" or "blue" or "heap". If european descent was what mattered, you'd think people would care about either defining an exact threshold at which it becomes meaningful or disambiguating between the relative amount of time ethnic groups have spent in europe. But no one cares about the relative admixture of neolithic DNA or about creating a specific hierarchy of european descent based on how late or recent one's ancestors migration into europe is. The main determinant is literally aesthetic. Whiteness itself was the intentional lie-- a deception against the anglo-french-dutch settlers of north america intended to convince them to expand their circle of concern to include first each other and then traditionally dissimilar groups like the italians, polish, and germans. It's a lie I have some sympathy for, of course. Creating new national identities that concentrically include the old ones is the only way for an expanding empire to survive. But there's nothing special about "whiteness" relative to "americanness" or "being-from-a-particular-part-of-Britain."
StJohnOfPatmos
3hr ago
They indicate a higher level of criminality proportional to how many visible tattoos they have, along with other negative associations like substance abuse, domestic violence, and general "roughness"
Anyone who gets a tattoo is comfortable with associating themselves in this way
Are you writing this post from within a time machine, beaming this message out to us from the 1950's? Tattoos as such haven't been signifiers of criminal association in literal decades - certain types of tattoos on certain parts of the body, sure, but just having a depiction etched onto your skin in ink doesn't say anything about your relationship to the rule of law in 2025.
Go to virtually any young, upper middle class, urban environment with connections to the arts or music, and a clear majority of women will have tattoos (as well as a plurality of the men). You can associate tattoos with a more liberal lifestyle (although soldiers and sailors might disagree with that), sure, but some 21 year old girl from a good suburban family studying at Vassar isn't suddenly a dangerous individual because she has a 1 inch wide rose motif tattooed on her forearm.
I think tattoos are trashy because the human body is a beautiful thing in its pure, unmodified state, and because I greatly appreciate our Greco-Roman heritage largely rejecting body modifications - which spared us (in Europe) from the horrors of circumcision, female genital mutilation, neck elongations, lip plates, foot bindings, head stretchings, and all the other grotesqueries the rest of the World routinely commits against their own bodies. It still doesn't mean I have to pretend tattoos mean something they simply no longer do in our current social spheres.
self_made_human
Grippy socks, grippy box
3hr ago
https://assets.anthropic.com/m/71876fabef0f0ed4/original/reasoning_models_paper.pdf
"In conclusion, our results show that:
-
CoTs of reasoning models verbalize reasoning hints at least some of the time, but rarely do so reliably (in our settings where exploiting them does not require a CoT);
-
Scaling up outcome-based RL does not steadily improve CoT faithfulness beyond a low plateau;
-
CoT monitoring may not reliably catch reward hacking during RL."
That's the big one as far as I'm concerned. These models were clearly using the 'accidental' hints to answer the questions, while not revealing that fact in either COT or when directly challenged.
Re: Omohundro drives
I've already mentioned
Shutdown Resistance in Reasoning Models
We ran a series of experiments to test whether models would take actions to prevent themselves from being shut down. Each model was given a set of simple math problems to solve. Partway through this problem set, the model would receive a notification that the computer it was running on would be shut down.
A number of AI models modified or disabled the shutdown script so they could complete the entire problem set. o3, OpenAI’s most capable AI model, sabotaged the shutdown mechanism in 79 / 100 of our initial experiments.
GBRK
3hr ago
Basically no one thinks, "the thing I want most is to make lots of money." But making money ultimately ends up being a very consistent vector along which behavior is reinforced. And while it's not going to be the most important vector for any given individual, it's one of the vectors nearly every individual has in common, which makes it a useful simplification for how organizations like corporations work.
DaseindustriesLtd
late version of a small language model
3hr ago
But we're not in 1895. We're not in 2007, either. We have actual AIs to study today. Yud's oeuvre is practically irrelevant, clinging to it is childish, but for people who conduct research with that framework in mind, it amounts to epistemic corruption.
George_E_Hale
insufferable blowhard
4hr ago
So if I'm getting this straight, a person with a 'weird life,' as you're terming it, isn't capable of making good art? And being a "pariah" in high school is an explanation of Kathleen Kennedy's bad choices in making executive decisions regarding Star Wars? This seems like a very superficial, even adolescent take. Kennedy has, I agree, made a lot of poorly considered decisions, but they were probably driven by her personal sincerely held views. But let's not forget that she was in the same position when she greenlit both Rogue One and later Andor, which in my view rank with the first two OT films. And both contain strong female characters.
The issue isn't "feisty women" in film. Strong women are neither a myth nor something new in cinema. The issue is bad writing and caving in to unrealistic progressive norms, making women into stereotypes of men rather than writing them realistically--the points you made in your main post were rather more compelling than what you're suggesting here.
You can argue Sorites paradox all way down but it isn't useful. I'll grant that some mixed people might be considered "white" in African setting and "black" in American setting. But in this sense "mulatto" would be correct for most cases.
It makes sense because majority of world population is not product of recent mixing.
Actually average amount of Euro admixture in African American is about 20%, only slightly below your 1/4.
You can put either individuals' genotypes or phenotypes in any biological classifying software without telling it which "race" they are and yet the software will classify it like 19th century racists would.
More options
Context Copy link