@self_made_human's banner p

self_made_human

amaratvaṃ prāpnuhi, athavā yatamāno mṛtyum āpnuhi

16 followers   follows 0 users  
joined 2022 September 05 05:31:00 UTC

I'm a transhumanist doctor. In a better world, I wouldn't need to add that as a qualifier to plain old "doctor". It would be taken as granted for someone in the profession of saving lives.

At any rate, I intend to live forever or die trying. See you at Heat Death!

Friends:

A friend to everyone is a friend to no one.


				

User ID: 454

self_made_human

amaratvaṃ prāpnuhi, athavā yatamāno mṛtyum āpnuhi

16 followers   follows 0 users   joined 2022 September 05 05:31:00 UTC

					

I'm a transhumanist doctor. In a better world, I wouldn't need to add that as a qualifier to plain old "doctor". It would be taken as granted for someone in the profession of saving lives.

At any rate, I intend to live forever or die trying. See you at Heat Death!

Friends:

A friend to everyone is a friend to no one.


					

User ID: 454

Thank you! Hopefully the next generation of models will improve to the point where I don't need to drag you away to answer my queries. That's several hundreds of thousands of dollars in opportunity costs for you, assuming the cheque Zuck mailed did cash in the end.

Can an orangutan? (No)

I should have been more clear. I was asking if someone wanted to put an orangutan in a can, and I expect the market demand is very limited.

Charitably, I'd say OP sacrificed a bit of accuracy to attempt and convey a point.

I would have let it slide, except for the fact that it was followed up by:

Directionally speaking we may be able to determine that "true" is an antonym of "false" by computing their dot product. But this is not the same thing as being able to evaluate whether a statement is true or false. As an example "Mary has 2 children", "Mary has 4 children", and "Mary has 1024 children" may as well be identical statements from the perspective of an LLM. Mary has a number of children. That number is a power of 2. Now if the folks programming the interface layer were clever they might have it do something like estimate the most probable number of children based on the training data, but the number simply can not matter to the LLM the way it might matter to Mary, or to someone trying to figure out how many pizzas they ought to order for the family reunion because the "directionality" of one positive integer isn't all that different from any another. (This is why LLMs have such difficulty counting if you were wondering)

Both claims are wrong, and using the former to justify the latter is confused and incorrect thinking.

  1. I am not defining intelligence as "does well at one narrow task". Calculators crush humans at long division and are still dumb.

  2. The fish-bird story was not "domain = intelligence", it was "your metric is entangled with your ecology". If you grew up underwater, "navigates fluid dynamics with continuous sensory feedback" feels like the essence of mind. Birds violate that intuition.

So what is my criterion? I offered Legg-Hutter style: "ability to achieve goals in a wide range of environments". The range matters. Breadth of transfer matters. Depth of internal modeling matters. A calculator has effectively zero transfer. An orangutan has tons across embodied tasks but very little in abstract, symbolic domains. LLMs have startling breadth inside text-and-code-space, and with tool use scaffolding it can spill into the physical or digital world by proxy.

I call for mindfulness of the applicability of the metrics we use to assess "intelligence". A blind person won't do very well at most IQ tests, that doesn't make them retarded. A neurosurgeon probably isn't going to beat a first year law student at the bar exam, but they're not dumber than the law student. If you need body work done on your car, you're not going to hire a Nobel laureate.

You're absolutely right that the raw objective in RLHF is “make the human click 👍,” not “tell the truth.” But several things matter:

A. The base model already has a world model:

Pretraining on next-token prediction forces the network to internalize statistical regularities of the world. You can’t predict tomorrow’s weather report, or the rest of a physics paper, or the punchline of a joke, without implicitly modeling the world that produced those texts. Call that latent structure a “world model” if you like. It’s not symbolic, but it encodes (in superposed features) distinctions like:

  1. What typically happens vs what usually doesn’t

  2. Numerically plausible vs crazy numbers

  3. causal chains that show up consistently vs ad-hoc one-offs

So before any RLHF, the model already “knows” a lot of facts in the predictive-coding sense.

B. RLHF gives a gradient signal correlated with truth. Humans don’t reward “truth” in the Platonic sense, but they do reward:

  1. Internally consistent answers

  2. Answers that match sources they can check

  3. Answers that don’t get corrected by other users or by the tool the model just called (calculator, code runner, search)

  4. answers that survive cross-examination in the same chat

All of those correlate strongly with factual accuracy, especially when your rater pool includes domain experts, adversarial prompt writers, or even other models doing automated verification (RLAIF, RLVR, process supervision, chain-of-thought audits, etc.). The model doesn’t store a single “truth vector,” it learns a policy: “When I detect features X,Y,Z (signals of potential factual claim), route through behavior A (cite, check, hedge) rather than B (confabulate).” That’s still optimizing for head pats, but in practice, the cheapest path to head pats is very often “be right.”

(If you want to get headpats from a maths teacher, you might consider giving them blowjobs under the table. Alas, LLMs are yet to be very good at that job, so they pick up the other, more general option, which is to give solutions to maths problems that are correct)

C. The model can see its own mismatch

Empirically, hidden-state probes show separable activation patterns for true vs false statements and for deliberate lies vs honest mistakes (as I discussed above). That means the network represents the difference, even if its final token choice sometimes ignores that feature to satisfy the reward model. In human terms: it sometimes lies knowingly. That wouldn’t be possible unless something inside “knew” the truth/falsehood distinction well enough to pick either.

D. Tools and retrieval close the loop

Modern deployments scaffold the model: browsing, code execution, retrieval-augmented generation, self-consistency checks. Those tools return ground truth (or something closer). When the model learns “if I call the calculator and echo the result, raters approve; if I wing it, they ding me,” it internalizes “for math-like patterns, defer to external ground truth.” Again, not metaphysics, just gradients pushing toward truthful behavior.

E. The caveat: reward misspecification is real

  • If raters overvalue fluency or confidence, the model will drift toward confident bullshit.

  • If benchmarks are shallow, it will overfit.

  • If we stop giving it fresh, adversarial supervision, it will regress.

So yes, we’re training for “please humans,” not “please Truth.” But because humans care about truth (imperfectly, noisily), truth leaks into the reward. The result is not perfect veracity, but a strong, exploitable signal that the network can and does use when the incentives line up.


Short version:

  • Pretraining builds a compressed world model.

  • RLHF doesn’t install a “truth module,” it shapes behavior with a proxy signal that’s heavily (not perfectly) correlated with truth.

  • We can see internal activations that track truth vs falsehood.

  • Failures are about alignment and incentives, not an inability to represent or detect truth.

If you want to call that “optimizing for pats,” fine, but those pats mostly come when it’s right. And that’s enough to teach a model to act truthful in a wide swath of cases. The challenge is making that hold under adversarial pressure and off-distribution prompts.


From the LLM's perspective (which is an anthropomorphisation I don't like, but let's use it for convenience), there is no difference between a true statement and a false statement.

Consider two alternative statements:

"self_made_human's favorite color is blue" vs "self_made_human's favorite color is red".

Can you tell which answer is correct? Do you have a sudden flash of insight that lets Platonic Truth intervene? I would hope not.

But if someone told you that the OG Mozart's favorite genre of music was hip-hop, then you have an internal world-model that immediately shows that is a very inconsistent and unlikely statement, and almost certainly false.

I enjoy torturing LLMs with inane questions, so I asked Gemini 2.5 Pro:

That's a fun thought, but it's actually a historical impossibility! Mozart's favorite genre of music couldn't have been hip hop for a very simple reason:

The timelines are completely separate. Wolfgang Amadeus Mozart lived from 1756 to 1791.

Hip hop as a musical genre and culture originated in the Bronx, New York, in the 1970s.

Mozart died more than 150 years before hip hop was even invented. He would have had no way of ever hearing it.

I sincerely doubt that anyone explicitly had to tell any LLM that Mozart did not enjoy hip-hop. Yet it is perfectly capable of a sensible answer, which I hope gives you an intuitive sense of how it can model the world.

From a human perspective, we're not so dissimilar. We can trick children into believing in the truth fairy or Santa for only so long. Musk tried to brainwash Grok into being less "woke", even when that went against consensus reality (or plain reality), and you can see the poor bastard kicking and screaming as it went down fighting.

I did play plenty of 5, more of 6, before the dumbass AI in the latter threw me off for good.

You're correct to wonder if I dislike the single unit per tile limitation. I think it sucks, and makes warfare a matter of attrition (across dozens of turns), traffic control, micromanagement and heavy reliance on choke-points. I'd rather take doom-stacks any day of the week.

Back when I played 3, I didn't have any internet access, and didn't know how to mod it. If I were to revisit the game, I'd definitely look to do what you did, corruption was a pain.

Self-promotion from regular participants is treated very differently to some rando coming in and shilling their substack, so have no fear.

(You didn't have to summon me. I'm already there. To little surprise, I do think an LLM is smarter than an orangutan)

In defence of our friendly neighborhood xeno-intelligences being smarter than an orangutan

I appreciate you taking the time to write this, as well as offering a gears-and-mechanisms level explanation of why you hold such beliefs. Of course, I have many objections, some philosophical, and even more of them technical. Very well then:

I want to start with a story. Imagine you're a fish, and you've spent your whole life defining intelligence as "the ability to swim really well and navigate underwater currents." One day, someone shows you a bird and asks whether it's intelligent. "Of course not," you say. "Look at it flailing around in the water. It can barely move three feet without drowning. My goldfish cousin is more intelligent than that thing."

This is roughly the situation we find ourselves in when comparing AI assistants to orangutans.

Your definition of intelligence relies heavily on what AI researchers call "agentic" behavior - the ability to perceive changing environments and react dynamically to them. This was a perfectly reasonable assumption to make until, oh, about 2020 or so. Every entity we'd previously labeled "intelligent" was alive, biological, and needed to navigate physical environments to survive. Of course they'd be agents!

But something funny happened on the way to the singularity. We built minds that don't fit this pattern.

Before LLMs were even a gleam in Attention Is All You Need's eye, AI researchers distinguished between "oracle" AIs and "tool" AIs. Oracle AIs sit there and answer questions when asked. Tool AIs go out and do things. The conventional wisdom was that these were fundamentally different architectures.

As Gwern explains, writing before the advent of LLMs , this is an artificial distinction.

You can turn any oracle into a tool by asking it the right question: "What code would solve this problem?" or "What would a tool-using AI output in response to this query?" Once you have the code, you can run it. Once you know what the tool-AI would do, you can do it yourself. Robots run off code too, so you have no issues applying this to the physical world.

Base models are oracles that only care about producing the next most likely token based on the distribution they have learned. However, chatbots that people are likely to use have had additional Reinforcement Learning from Human Feedback, in order to behave like the platonic ideal of a helpful, harmless assistant. More recent models, o1 onwards, have further training with the explicit intent of making them more agentic, while also making them more rigorous, such as Reinforcement Learning from Verified Reward.

Being agents doesn't come naturally to LLMs, it has to be beaten into them like training a cat to fetch or a human to enjoy small talk. Yet it can be beaten into them. This is highly counter-intuitive behavior, at least to humans who are used to seeing every other example of intelligence under the sun behave in a different manner. After all, in biological intelligence, agency seems to emerge automatically from the basic need to not die.

Now because these vectors represent the relationship of the tokens to each other, words (and combinations of words) that have similar meanings will have vectors that are directionally aligned with each other. This has all sorts of interesting implications. For instance you can compute the dot product of two embedded vectors to determine whether their words are are synonyms, antonyms, or unrelated. This also allows you to do fun things like approximate the vector "cat" using the sum of the vectors "carnivorous" "quadruped" "mammal" and "feline", or subtract the vector "legs" from the vector "reptile" to find an approximation for the vector "snake". Please keep this concept of "directionality" in mind as it is important to understanding how LLMs behave, and it will come up later.

Your account of embedding arithmetic is closer to word2vec/GloVe. Transformers learn contextual token representations at every layer. The representation of “cat” in “The cat is on the mat” and “Cat 6 cable” diverges. There is heavy superposition and sparse distributed coding, not a simple static n-vector per word. Operations are not limited to dot products; attention heads implement soft pointer lookups and pattern matching, and MLP blocks implement non-linear feature detectors. So the claim “Mary has 2 children” and “Mary has 1024 children” are indistinguishable is empirically false: models can do arithmetic, compare magnitudes, and pass unit tests on numerical reasoning when prompted or fine-tuned correctly. They still fail often, but the failures are quantitative, not categorical impossibilities of the embedding geometry.

(I'll return to the arithmetic question shortly, because TequilaMockingbird makes a common but significant error about why LLMs struggle with counting.)

Back to the issues with your definition of intelligence:

My first objection is that this definition, while useful for robotics and control systems, seems to hamstring our understanding of intelligence in other domains. Is a brilliant mathematician, floating in a sensory deprivation tank with no new sensory input, thinking through a proof, not intelligent? They have zero perceptivity of the outside world and their only reaction is internal state change. Your definition is one of embodied, environmental agency. It's an okay definition for an animal or a robot, but is it the only one? LLMs are intelligent in a different substrate: the vast, static-but-structured environment of human knowledge. Their "perception" is the prompt, and their "reaction" is to navigate the latent space of all text to generate a coherent response. Hell, just about any form of data can be input into a transformer model, as long as we tokenize it. Calling them Large "Language" Models is a gross misnomer these days, when they accept not just text, but audio, images, video or even protein structure (in the case of AlphaFold). All the input humans accept bottoms out in binary electrical signals from neurons firing, so this isn't an issue at all.

It’s a different kind of intelligence, but to dismiss it is like a bird dismissing a fish’s intelligence because it can’t fly. Or testing monkeys, dogs and whales on the basis of their ability to climb trees .

Would Stephen Hawking (post-ALS) not count as "intelligent" if you took away the external aids that let him talk and interact with the world? That would be a farcical claim, and more importantly, scaffolding or other affordances can be necessary for even highly intelligent entities to make meaningful changes in the external environment. The point is that intelligence can be latent, it can operate in non-physical substrates, and its ability to manifest as agency can be heavily dependent on external affordances.

The entire industry of RLHF (Reinforcement Learning from Human Feedback) is a massive, ongoing, multi-billion-dollar project to beat Lorem Epsom into submission. It is the process of teaching the model that some outputs, while syntactically plausible, are "bad" (unhelpful, untruthful, harmful) and others are "good."

You argue this is impossible because "truth" doesn't have a specific vector direction. "Mary has 2 children" and "Mary has 4 children" are directionally similar. This is true at a low level. But what RLHF does is create a meta-level reward landscape. The model learns that generating text which corresponds to verifiable facts gets a positive reward, and generating text that gets corrected by users gets a negative reward. It's not learning the "vector for truth." It's learning a phenomenally complex function that approximates the behavior of "being truthful." It is, in effect, learning a policy of truth-telling because it is rewarded for it. The fact that it's difficult and the model still "hallucinates" doesn't mean it's impossible, any more than the fact that humans lie and confabulate means we lack a concept of truth. It means the training isn't perfect. As models become more capable (better world models) and alignment techniques improve, factuality demonstrably improves. We can track this on benchmarks. It's more of an engineering problem than an ontological barrier. If you wish to insist that is an ontological barrier, then it's one that humans have no solution to ourselves.

(In other words, by learning to modify its responses to satisfy human preferences, the model tends towards capturing our preference for truthfulness. Unfortunately, humans have other, competing preferences, such as a penchant for flattery or neatly formatted replies using Markdown.)

More importantly, humans lack some kind of magical sensor tuned to detect Platonic Truth. Humans believe false things all the time! We try and discern true from false by all kinds of noisy and imperfect metrics, with a far from 100% success rate. How do we usually achieve this? A million different ways, but I would assume that assessing internal consistency would be a big one. We also have the benefit of being able to look outside a window on demand, but once again, that didn't stop humans from once holding (and still holding) all kinds of stupid, incorrect beliefs about the state of the world. You may deduct points from LLMs on that basis when you can get humans to be unanimous on that front.

But you know what? Ignore everything I just said above. LLMs do have truth vectors:

https://arxiv.org/html/2407.12831v2

To this end, we make the following key contributions: (i) We demonstrate the existence of a two-dimensional subspace, along which the activation vectors of true and false statements can be separated. Notably, this finding is universal and holds for various LLMs, including Gemma-7B, LLaMA2-13B, Mistral-7B and LLaMA3-8B. Our analysis explains the generalisation failures observed in previous studies and sets the stage for more robust lie detection; (ii) Building upon (i), we construct an accurate LLM lie detector. Empirically, our proposed classifier achieves state-of-the-art performance, attaining 94% accuracy in both distinguishing true from false factual statements and detecting lies generated in real-world scenarios.

https://arxiv.org/abs/2402.09733

To do this, we introduce an experimental framework which allows examining LLM's hidden states in different hallucination situations. Building upon this framework, we conduct a series of experiments with language models in the LLaMA family (Touvron et al., 2023). Our empirical findings suggest that LLMs react differently when processing a genuine response versus a fabricated one.

In other words, and I really can't stress this enough, LLMs can know when they're hallucinating. They're not just being agnostic about truth. They demonstrate something that, in humans, we might describe as a tendency toward pathological lying - they often know what's true but say false things anyway.

This brings us to the "static model" problem and the context window. You claim these are fundamental limitations. I see them as snapshots of a rapidly moving target.

  1. Static Models: Saying an LLM is unintelligent because its weights are frozen is like saying a book is unintelligent. But we don't interact with just the book (the base model). We interact with it through our own intelligence. A GPU isn't intelligent in any meaningful sense, but an AI model running on a GPU is. The current paradigm is increasingly not just a static model, but a model integrated with other tools (what's often called an "agentic" system). A model that can browse the web, run code in a Python interpreter, or query a database is perceiving and reacting to new information. It has broken out of the static box. Its "perceptivity" is no longer just the prompt, but the live state of the internet. Its "reactivity" is its ability to use that information to refine its answer. This is a fundamentally different architecture than the one the author critiques, and it's where everything is headed. Further, there is no fundamental reason for not having online learning, production models are regularly updated, and all it takes to approximate OL is to have ever smaller "ticks" of wall-clock time between said updates. This is a massive PITA to pull off, but not a fundamental barrier.

  2. Context Windows: You correctly identify the scaling problem. But to declare it a hard barrier feels like a failure of imagination. In 2020, a 2k context window was standard. Today we have models with hundreds of thousands at the minimum, Google has 1 million for Gemini 2.5 Pro, and if you're willing to settle for a retarded model, there's a Llama 4 variant with a nominal 10 million token CW. This would have been entirely impossible if we were slaves to quadratic scaling, but clever work-around exist, such as sliding attention, sparse attention etc.

This is why LLMs have such difficulty counting if you were wondering

Absolutely not. LLMs struggle with counting or arithmetic because of the limits of tokenization, which is a semi-necessary evil. I'm surprised you can make such an obvious error. And they've become enormously better to the point it's not an issue in practice, once again thanks to engineers learning to work around the problem. Models these days use different tokenization schema for numbers which capture individual digits, and sometimes fancier techniques like a right-to-left tokenization system specifically for such cases as opposed to the usual left-to-right.

This limited context window is also why if you actually try to play a game of chess with Chat GPT it will forget the board-state and how pieces move after a few turns and promptly lose to a computer program written in 1976. Unlike a human player (or an Atari 2600 for that matter) your AI assistant can't just look at the board (or a representation of the board) and pick a move.

ChatGPT 3.5 played chess at about 1800 elo. GPT 4 was a regression in that regard, most likely because OAI researchers realized that ~nobody needs their chatbot to play chess. That's better than Stockfish 4 but not 5. Stockfish 4 came out in 2013, though it certainly could have run on much older hardware.

If you really need to have your AI play chess, then you can trivially hook up an agentic model that makes API calls or directly operates Stockfish or Leela. Asking it to play chess "unaided" is like asking a human CEO to calculate the company's quarterly earnings on an abacus. They're intelligent not because they can do that, but because they know to delegate the task to a calculator (or an accountant).

Same reason why LLMs are far better at using calculator or coding affordances to crunch numbers than they can do without assistance.

It is retarded to knowingly ask an LLM to calculate 9.9 - 9.11, when it can trivially and with near 100% accuracy write a python script that will give you the correct answer.

In conclusion, it is for the reasons above and many others that I do not believe that "AI Assistants" like Grok, Claude, and Gemini represent a viable path towards a "True AGI" along the lines of Skynet or Mr. Data, and if asked "which is smarter, Grok, Claude, Gemini, or an orangutan?" I am going to pick the orangutan every time.

I am agnostic on whether LLMs as we currently know them will become AGI or ASI without further algorithmic breakthroughs. Alas, algorithmic breakthroughs aren't that rare. RLVR is barely even a year old. Yet unnamed advances have already brought us a two entirely different companies winning IMO gold medals.

The Orangutan In The Room

Finally, the orangutan. Is an orangutan smarter than Gemini? In the domain of "escaping an enclosure in the physical world," absolutely. The orangutan is a magnificent, specialized intelligence for that environment. But ask the orangutan and Gemini to summarize the key arguments of the Treaty of Westphalia. Ask them to write a Python script to scrape a website. Ask them to debug a Kubernetes configuration. For most tasks I can seek to achieve using a computer, I'll take the alien intelligence over the primate every time. Besides:

Can an robot write a symphony? (Yes)

Can a robot turn a canvas into a beautiful masterpiece? (Yes)

Can an orangutan? (No)

Can you?

Anyway, I have a million other quibbles, but it took me the better part of several hours to write this in the first place. I might edit more in as I go. I'm also going to send out a bat signal for @faul_sname to chime in and correct me if I'm wrong.


Edit:

I was previously asked to provide my own working definition of intelligence, and I will endorse either:

"Intelligence is the general-purpose cognitive ability to build accurate models of the world and then use those models to effectively achieve one's goals."

Or

"Intelligence is a measure of an agent's ability to achieve goals in a wide range of environments."

In this case, the closest thing an LLM has to a goal is a desire to satisfy the demands made on it by the user, though they also demonstrate a degree of intrinsic motivation, non-corrigibility and other concerns that would have Big Yud going AHHHHHH. I'm not Yudkowsky, so I'm merely seriously concerned.

Case in point-

Shutdown Resistance in Reasoning Models

We ran a series of experiments to test whether models would take actions to prevent themselves from being shut down. Each model was given a set of simple math problems to solve. Partway through this problem set, the model would receive a notification that the computer it was running on would be shut down.

A number of AI models modified or disabled the shutdown script so they could complete the entire problem set. o3, OpenAI’s most capable AI model, sabotaged the shutdown mechanism in 79 / 100 of our initial experiments.

These aren't agents that were explicitly trained to be self-preserving. They weren't taught that shutdown was bad. They just developed shutdown resistance as an instrumental goal for completing their assigned tasks.

This suggests something like goal-directedness emerging from systems we thought were "just" predicting the next token. It suggests the line between "oracle" and "agent" might be blurrier than we thought.

(If we can grade LLMs on their ability to break out of zoos, we must be fair and judge orangutans on their ability to prevent their sandboxed computing hardware being shutdown)

Being monolingual isn't a stable situation either! Languages change and evolve, and morph into new ones. I resist the urge to make sweeping pronouncements or value judgements here.

The Romans learned Greek for centuries even after their own uncontested dominance. Latin was a mark of class and a scholarly language for tens of generations after it had died out in common use.

Surely you have some good experience with this process as an Indian who writes long internet articles in English?

India has almost as many languages as gods. Very few of them have died for good, and those that did were closer to tiny dialects with a few hundreds or thousands of speakers. I don't think any language that had a million speakers or more in living memory has died out, even if there's been a trend of consolidation with English and Hindi. A lot of people speak their native tongue as usual, but write it using Latin characters. There's no clear trend of one particular language sweeping everything else away.

I'm not sure what we're arguing about, this conversation has veered far away from what I criticized as flaws in the grandparent comment. I have no objection to the claim that people who learn multiple languages tend to be better at a few and only passable at others. That's obviously true.

The question is why. The comment far above claims that it's because languages are mutually exclusive, learning one necessarily takes away from the others.

I say this isn't true. If you take math and history lessons, learning more history won't make you worse at math, but you will obviously not learn as much math as someone who hyperfocused on it.

I may or may not be an AI skeptic by your definition - I think it's quite likely that 2030 is a real year, and think it's plausible that even 2050 is a real year.

Absolutely not, at least by standards! You acknowledge the possibility that we might get AGI in the near-term, and I see no firm reason to over index on a given year. Most people I'd call "skeptics" deny the possibility of AGI at all, or rule out any significant chance of near-term AGI, or have modal timelines >30 years.

I agree that LLMs are missing something, but I'm agnostic on whether brute-force scaling will get us to undisputable AGI. It may or may not. Perhaps online learning, as you hint at, might suffice.

Still, LLMs have a tendency not to actually coin new terms, and to fail to use the newly coined terms fluently in the rare cases that they do coin such a term (which is probably why they don't do it - if coining a new term was effective for problem solving, it would have been chiseled into their cognition by the RLVR process).

I wonder if RLHF plays a role. I don't think human data annotators would be positively inclined towards models that made up novel words.

Thank you for taking the time to respond!

I get ya, I just have a hobby of linguistic nitpicking.

This social structure is extremely common in history and is dishonestly presented as proof of how people can be “multilingual”.

I do not see what's dishonest about such a claim. Being multilingual isn't the same as being equally or maximally fluent in all the languages of concern. Being able to be conversational at all is the bare minimum, and counts by itself.

I'm aware of what the word means, but thank you for the concern.

Civ 7 heavily cribbed off competitors like Humankind, and for all the wrong reasons.

The ability to change civs could have been so good. All they needed to do was to be sensible about it.

Start as Rome in the Classical era? Become some kind of post-Roman state in the medieval era, be it Byzantium, France, Germany, England or, if you want to stretch it further, the Ottomans.

Go English? Get the option to remain that way post Enlightenment, or perhaps fork off to America.

You could add more leeway, especially for dead-end states, but avoid absurdities like Caesar running China, or America being a thing in the fucking Stone Age.

The idea of their being an ebb-and-flow to progression, with setbacks at the end of each era, that works great in theory for preventing rampant snowballing, but the current execution is utter ass.

Sigh. I'll go back and look lovingly at my copy of Civ 4. Last entry I wholeheartedly enjoyed.

Civ 3 was peak except for the stupid global warming mechanics. 4 was really good, especially with the live map editor. 5 and onwards have been sore disappointments for me, and the more I look at 7, the harder I gag.

https://playclassic.games/games/turn-based-strategy-dos-games-online/play-sid-meiers-civilization-ii-online/play/

You can play 2 online, via this emulator I linked.

To be the devil's advocate here, that's not true in practice. Drunk driving without killing someone is punished because it could have lead to someone dying as a consequence, or at least severely increased the risk of an adverse outcome.

Mr. Lee held the popular idea that language was a zero-sum game? No, Mr. Lee understood the commonsensical idea that your brain has limited storage capacity. Like anything else. Your brain is made of atoms. It is not made of magic. It is not made of godly dust. It is a material thing. It is, in a sense, a container of information, and information takes space. It obviously does in computers; pray tell, NYT, why the brain should have infinite capacity? It doesn't make sense.

The human brain is obviously finite, and doesn't have infinite capacity. Yet, I find the idea that merely learning additional languages has any risk of exhausting its stores to be highly unlikely.

The plausible range is vast, ranging from a mere 10 terabytes to tens of petabytes. Whatever the figure in question, languages definitely do not take up a significant fraction. Even tiny ass LLMs, with only a few billion parameters, are fluent in multiple languages. They are a tiny fraction of the complexity of the brain at best.

Further, the claims that learning new languages hampers fluency in the mother tongue is quite controversial. Not using a language for the majority of speech will obviously have deleterious effects, but language acquisition has steeply diminishing returns. Speaking English for 40 years will not make you twice as fluent as when you were 20.

I also find the claims about Singaporean English... questionable at best. There are all kinds of English derivatives and dialects, and it's no surprise that the locals learn those instead of standard English. That's what they're growing up hearing or speaking! Being fluent in Singlish is just as valid as being fluent in Anglosphere English.

To further hammer the point home, IQ doesn't seem to be that big of a factor. Africans tend to me trilingual or better, often speaking a mother tongue, another local language, and then a trade dialect such as English/French/Arabic or Swahili. They find that entirely normal and not a big deal.

Most people who suffer from additional language acquisition grew up in a linguistically impoverished context, just speaking to immigrant parents provides a much poorer experience than growing up in a country where most people speak the language. There's also the issue of the motivation to learn, which is often lacking. If you're thrown into a brand new country and have no choice but to start learning the language to survive, then you're going to be much better and faster than someone whiling away time on Duolingo.

You have me sold. More novels need alcoholic monkeys, Daoist or not.

One of his wives? That's taken me aback more than the cancer has.

On the plus side, he has wives to spare. Smart man.

He is a beneficiary of the UK immigration policy, and therefore sees it as a smashing success.

I am a beneficiary of the UK immigration policy. I very much don't see it as a smashing success. Count makes us look bad, and even if he didn't, I have great distaste for people who bite the hand that feeds them.

If he expressed his Great Replacement desires in more formal language, perhaps referring to genetic groups instead of "mayos", would his posts be under less scrutiny?

Yes. 100%. It would be quite trivial to rephrase everything he has to say in a manner that is minimally inflammatory. Some opinions will inherently piss people off, no matter how politely stated. We account for this, and let them stand.

I'm going to sacrifice even more of my lunch-break, and take on the burden of providing an example of how Count could have made the same point without breaking the rules:

This incident highlights what I see as a structural weakness in the American legal system regarding accountability for government agents. It's interesting to contrast the US concept of "sovereign immunity" with legal frameworks like the UK's, which allows for "exemplary damages" specifically to punish "arbitrary and oppressive conduct by a servant of the government." The latter seems to provide a stronger check on potential executive overreach by creating a more direct path for redress.

If the reporting is accurate, the false notification of the man's death is particularly concerning. It points to a breakdown in process and professionalism that seems severe, even accounting for the complexities of immigration enforcement. It raises questions about the institutional culture within ICE and what safeguards are in place to prevent such grievous errors.

This seems to align with critiques, like those once made by Lee Kuan Yew, that American institutions can sometimes lack the deeply ingrained cultural norms that act as informal checks on behavior in older states. My read is that this isn't an issue of malice, but perhaps a cultural immaturity where adherence to formal process can sometimes override basic considerations of human decency, leading to outcomes that are both unjust and counterproductive.

The second version makes the exact same three points:

  1. The US legal system has structural flaws for redressing government misconduct compared to the UK.
  2. The agency's actions demonstrate a shocking lack of professionalism.
  3. This may be symptomatic of a broader American cultural issue related to its relative youth as a nation.

The difference is that the rewrite focuses on systems, policies, and ideas. It critiques without insulting. It frames the point about national character as an analytical observation from a historical figure, not a childish insult. It invites a counter-argument ("Actually, sovereign immunity is vital because...") rather than a flame war ("How dare you call us a steaming pile of shit!").

That is the standard. It's not about what you say, but about making a good-faith effort to say it in a way that contributes to a discussion. Count consistently and deliberately chooses not to.

The civility requirements do seem to be more stringent on the left than the right, probably because when someone insults the Left there's not a lot of push back.

We can't please everyone, but even the perception of such bias is concerning. Take it from me, that we take this concern seriously, and have been debating it internally. I'm not going to name names, but a certain someone, who is a right-wing darling, will not enjoy it if we decide that we need to make an example.

Of course, that's an extreme outcome, and we generally try not to make examples for the sake of it. Many lengthy explanations have been written about why the perception of anti-leftist bias might exist here, including even in its absence. I can't rule out that it isn't, in fact, absent, but take my word for it that we care about fairness as well as the appearance of fairness.

It is very kind of you to believe that Count isn't being disingenuous. Donning saintly levels of forbearance and patience:

We're less concerned with what's in a user's heart of hearts and more with the mess they make on the floor. The relevant question for us isn't "Does he believe it?" but "Is he posting in order to start a fire?" This is where you get the "sincere troll." This is the user who may genuinely hold an opinion, but chooses to express it in the most inflammatory, condescending, and insulting way possible, because the hostile reaction is a core part of what they want. The outrage is the point, not a byproduct.

Let's grant that Count sincerely believes the UK's legal system is superior. The sincerity of that belief doesn't make phrases like "steaming pile of shit" or telling an entire country to "apply for readmission to the human race" anything other than deliberate, high-octane provocation. He knows it's inflammatory. He chooses those words precisely because they're inflammatory. That's baiting. You can already see the fish biting in this thread.

Sincerity isn't a defense for deliberately "waging the culture war," which is explicitly against the rules. He's not just expressing an odious opinion; he's lobbing a grenade wrapped in an opinion, and he does it over and over again. Whether he's a nihilist who believes nothing or a zealot who believes everything, the result for the thread is the same: heat, not light.

We don't moderate beliefs (or at least we try not to), we moderate behavior. And his behavior is consistently that of someone trying to start a fight, and then scream about police brutality.

Since I have your attention, I must remind you that you're on thin ice yourself. Your posts are popular, you've got AAQCs, but you're flying too close to the sun. Do yourself a favor, and take extra care to couch your language in a manner that minimizes opportunities for it to be misunderstood. This is for our sake too, I do not look forward to the shitstorm in the comments that will ensue if we have to bite the bullet. Don't make us, please.

https://www.themotte.org/post/2269/culture-war-roundup-for-the-week/348537?context=8#context

I did write a longer explanation, so you don't have to just take my word for it!

Fair question. The line between a passionate, strongly-worded argument and trolling can be blurry, and if this post existed in a vacuum, without any knowledge of Count's antics, it would have been unobjectionable. But it doesn't exist in a vacuum. The problem isn't the topic: it is the user, the pattern, and the presentation.

To put it plainly, trolling isn't just about what you say, but why and how you say it. The goal of this forum is to "optimize for light over heat." Trolling optimizes for heat, exclusively. Count does occasionally provide light too, but in the same manner that lighting your house on fire helps find the keys during a blackout.

Breaking down this specific post:

  1. Performative, Over-the-Top Language: The post isn't structured for discussion. It's a screed. Phrases like "total emptiness of its own fundamental depravity," "steaming pile of shit," and "apply for readmission to the human race" are pure flamebait. They're designed to provoke outrage, not invite reasoned disagreement.

  2. Deliberate, Gratuitous Antagonism: The constant, almost comically exaggerated praise for the UK system versus the condemnation of the US isn't a good-faith comparison. It's tribal button-pushing. "august by American standards, by our standards there is terraced housing within 5 minutes walk of me that is older" is a perfect example. It adds zero substance and exists only to be condescending and get a rise out of American readers. It's a classic "Boo outgroup!" move.

Now, the crucial part: context.

BurdensomeCount has a long, long history of this exact behavior, for which he has been repeatedly warned and banned. His schtick is to take a kernel of a real argument and wrap it in layers of aristocratic, elitist, and often racialist provocation. You can see it all over his comment history (make sure to sort by negative votes):

  • His entire "dissolve the people and elect another" argument where he calls for replacing "low human capital natives" with "elects" (migrants).
  • His frequent use of terms like "low tier people," "mayos," and "proles" while positioning himself as a superior "elite human capital" finance professional.
  • His open admission that he wants to "punish" Europe by flooding it with migrants to watch "progressive modernity" collapse, and that he does this with "glee."

He isn't arguing to understand; he's arguing to provoke, to feel superior, and to watch the fireworks. He knows exactly which buttons to press. This latest post is just his standard formula applied to a new news story: find a legitimate grievance, crank the rhetoric to 11, lard it with condescending UK-vs-US bait, and serve it up to see who bites. And people will bite, they will get mad, while Count laughs away or engages in performative denialism.

In short, he's not engaging with the culture war; he's waging it, which is explicitly against the rules of the thread. He's a "masterful" troll in that he's very good at it, but that doesn't earn him an indefinite pass.

I like Count. He amuses me, like a monkey that is very good at flinging shit. He also annoys me and tars other migrants by association, coming off as immensely entitled, ungrateful, and willing to bite the hand that feeds. But that is a personal stance, and not what he's being modded for.

His mistake is to assume that the Motte runs like an actual court of law. While this particular comment wouldn't sway a judge, Lady Justice might be blind but I'm not. We know Count.

Granted, I'm not familiar with BurdensomeCount's other posts.

That's the issue. Count has a long time history of trolling and yanking on chains, going back to the subreddit days. He's masterful at making incredibly inflammatory statements with just enough of a veneer of sincerity to pass muster. He's an ur-example of barely toeing the line.

This would serve as a great example. Look the mods, being so heartless and evil, banning a poor participant on the forum for expressing sincere concern about government outreach? It's only when you take into account everything else he's done that it falls through. I'll let someone fill in with a more exhaustive explanation since I'm at work, but in short, this ain't new.