This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Here some people have expressed interest in my take on AI broadly, and then there's Deepseek-Coder release, but I've been very busy and the field is moving so very fast again, it felt like a thankless job to do what Zvi does and without his doomer agenda too (seeing the frenetic feed on Twitter, one can be forgiven for just losing the will; and, well, I suppose Twitter explains a lot about our condition in general). At times I envy Iconochasm who tapped out. Also, this is a very niche technical discussion and folks here prefer policy.
But, in short: open source AI, in its most significant aspects, which I deem to be code generation and general verifiable reasoning (you can bootstrap most everything else from it), is now propped up by a single Chinese hedge fund (created in the spirit of Renaissance Capital) which supports a small, ignored (except by scientists and a few crackpots on Twitter) research division staffed with some nonames, who are quietly churning out extraordinarily good models with the explicit aim of creating AGI in the open. These models happen to be (relatively) innocent of benchmark-gaming, but somewhat aligned to Chinese values. The modus operandi of DeepSeek is starkly different from that of either other Chinese or Western competitors. In effect this is the only known group both meaningfully pursuing frontier capabilities and actively teaching others how to do so. I think this is interesting and a modest cause for optimism. I am also somewhat reluctant to write about this publicly because there exist lovers of Freedom here, and it would be quite a shame if my writing contributed to targeted sanctions and even more disempowerment of the small man by the state machinery in the final accounting.
But the cat's probably out of the bag. The first progress prize of AI Mathematical Olympiad had just been taken by a team using their DeepSeekMath-7B model, solving 29 out of 50 private test questions «less challenging than those in the IMO but at the level of IMO preselection»; Terence Tao finds it «somewhat higher than expected» (he is on the AIMO Advisory Committee, along with his fellow Fields medalist Timothy Gowers).
The next three teams entered with this model as well.
I. The shape of the game board
To provide some context, here's an opinionated recap of AI trends since last year. I will be focusing exclusively on LLMs, as that's what matters (image gen, music gen, TTS etc largely are trivial conveniences, and other serious paradigms seem to be in their embryonic stage or in deep stealth).
- We have barely advanced in true out-of-distribution reasoning/understanding relative to the original «Sparks of AGI» GPT-4 (TheDag, me); GPT-4-04-29 and Sonnet 3.5 were the only substantial – both minor – steps forward, Gemini was a catch-up effort, and nobody else has yet credibly reached the same tier. We have also made scant progress towards consensus on whether that-which-LLMs-do is «truly» reasoning or understanding; sensible people have recoursed to something like «it's its own kind of mind, and hella useful».
- Meanwhile there's been a great deal of progress in scaffolding (no more babyAGI/AutoGPT gimmicry, now agents are climbing up the genuinely hard SWE-bench), code and math skills, inherent robustness in multi-turn interactions and responsiveness to nuanced feedback (to the point that LLMs can iteratively improve sizable codebases – as pair programmers, not just fancy-autocomplete «copilots»), factuality, respect of prioritized system instructions, patching badly covered parts of the world-knowledge/common sense manifold, unironic «alignment» and ironing out Sydney-like kinks in deployment, integrating non-textual modalities, managing long contexts (merely usable 32K "memory" was almost sci-fi back then, now 1M+ with strong recall is table stakes at the frontier; with 128K mastered on a deeper level by many groups) and a fairly insane jump in cost-effectiveness – marginally driven by better hardware, and mostly by distilling from raw pretrained models, better dataset curation, low-level inference optimizations, eliminating architectural redundancies and discovering many "good enough" if weaker techniques (for example, DPO instead of PPO). 15 months ago,"$0.002/1000 tokens" for gpt-3.5-turbo seemed incredible; now we always count tokens by the million, and Gemini-Flash blows 3.5-turbo out of the water for half that, so hard it's not funny; and we have reason to believe it's still raking in >50% margins whereas OpenAI probably subsidized their first offerings (though in light of distilling and possibly other methods of compute reuse, it's hard to rigorously account for a model's capital costs now).
- AI doom discourse has continued to develop roughly as I've predicted, but with MIRI pivoting to evidence-free advocacy, orthodox doomerism getting routed as a scientific paradigm, more extreme holdovers from it («emergent mesaoptimizers! tendrils of agency in inscrutable matrices!») being wearily dropped by players who matter, and misuse (SB 1047 etc) + geopolitical angle (you've probably seen young Leopold) gaining prominence.
- The gap in scientific and engineering understanding of AI between the broader community and "the frontier" has shrunk since the debut of GPT-4 or 3.5, because there's too much money to be made in AI and only so much lead you can get out of having assembled the most driven AGI company. Back then, only a small pool of external researchers could claim to understand what the hell they did above the level of shrugging "well, scale is all you need" (wrong answer) or speculating about some simple methods like "train on copyrighted textbooks" (spiritually true); people chased rumors, leaks… Now it takes weeks at most to trace a yet another jaw-dropping magical demo to papers, to cook up a proof of concept, or even to deem the direction suboptimal; the other two leading labs no longer seem desperate, and we're in the second episode of Anthropic's comfortable lead.
- Actual, downloadable open AI sucks way less than I've lamented last July. But it still sucks. And that's really bad, since it sucks most in the dimension that matters: delivering value, in the basest sense of helping do work that gets paid. And the one company built on the promise of «decentralizing intelligence», which I had hope for, had proven unstable.
To be more specific, open source (or as some say now, given the secretiveness of full recipes and opacity of datasets, «open weights») AI has mostly caught up in «creativity» and «personality», «knowledge» and some measure of «common sense», and can be used for petty consumer pleasures or simple labor automation, but it's far behind corporate products in «STEM» type skills, that are in short supply among human employees too: «hard» causal reasoning, information integration, coding, math. (Ironically, I agree here with whining artists that we're solving domains of competence in the wrong order. Also it's funny how by default coding seems to be what LLMs are most suited for, as the sequence of code is more constrained by preceding context than natural language is).
To wit, Western and Eastern corporations alike generously feed us – while smothering startups – fancy baubles to tinker with, charismatic talking toys; as they rev up self-improvement engines for full cycle R&D, the way imagined by science fiction authors all these decades ago, monopolizing this bright new world. Toys are getting prohibitively expensive to replicate, with reported pretraining costs up to ≈$12 million and counting now. Mistral's Mixtral/Codestral, Musk's Grok-0, 01.Ai's Yi-1.5, Databricks' DBRX-132B, Alibaba's Qwens, Meta's fantastic Llama 3 (barring the not-yet-released 405B version), Google's even better Gemma 2, Nvidia's massive Nemotron-340B – they're all neat. But they don't even pass for prototypes of engines you can hop on and hope to ride up the exponential curve. They're too… soft. And not economical for their merits.
Going through our archive, I find this year-old analysis strikingly relevant:
I think successful development of a trusted open model rivaling chatgpt in capability is likely in the span of a year, if people like you, who care about long-term consequences of lacking access to it, play their cards reasonably well. […] Companies whose existence depends on the defensibility of the moat around their LM-derived product will tend to structure the discourse around their product and technology to avoid even the fleeting perception of being a feasibly reproducible commodity.
That's about how it went. While the original ChatGPT, that fascinating demo, is commodified now, competitive product-grade AI systems are not, and companies big and small still work hard to maintain the impression that it takes
- some secret sauce (OpenAI, Anthropic)
- work of hundreds of Ph.Ds (Deepmind)
- vast capital and compute (Meta)
- "frontier experience" (Reka)
– and even then, none of them have felt secure enough yet to release a serious threat to the other's proprietary offers.
I don't think it's a big exaggerion to say that the only genuine pattern breaker – presciently mentioned by me here – is DeepSeek, the company that has single-handedly changed – a bit – my maximally skeptical spring'2023 position on the fate of China in the AGI race.
II. Deep seek what?
AGI, I guess. Their Twitter bio states only: «Unravel the mystery of AGI with curiosity. Answer the essential question with long-termism». It is claimed by the Financial Times that they have a recruitment pitch «We believe AGI is the violent beauty of model x data x computing power. Embark on a ‘deep quest’ with us on the journey towards AGI!» but other than that nobody I know of has seen any advertisement or self-promotion from them (except for like 70 tweets in total, all announcing some new capability or responding to basic user questions about license), so it's implausible that they're looking for attention or subsidies. Their researchers maintain near-perfect silence online. Their – now stronger and cheaper – models tend to be ignored in comparisons by Chinese AI businesses and users. As mentioned before, one well-informed Western ML researcher has joked that they're the bellwether for «the number of foreign spies embedded in the top labs».
FT also says the following of their parent company:
Its funds have returned 151 per cent, or 13 per cent annualised, since 2017, and were achieved in China’s battered domestic stock market. The country’s benchmark CSI 300 index, which tracks China’s top 300 stocks, has risen 8 per cent over the same time period, according to research provider Simu Paipai.
In February, Beijing cracked down on quant funds, blaming a stock market sell-off at the start of the year on their high-speed algorithmic trading. Since then, High-Flyer’s funds have trailed the CSI 300 by four percentage points.
[…] By 2021, all of High-Flyer’s strategies were using AI, according to manager Cai Liyu, employing strategies similar to those pioneered by hugely profitable hedge fund Renaissance Technologies. “AI helps to extract valuable data from massive data sets which can be useful for predicting stock prices and making investment decisions,” …
Cai said the company’s first computing cluster had cost nearly Rmb200mn and that High Flyer was investing about Rmb1bn to build a second supercomputing cluster, which would stretch across a roughly football pitch-sized area. Most of their profits went back into their AI infrastructure, he added. […] The group acquired the Nvidia A100 chips before Washington restricted their delivery to China in mid-2022.
“We always wanted to carry out larger-scale experiments, so we’ve always aimed to deploy as much computational power as possible,” founder Liang told Chinese tech site 36Kr last year. “We wanted to find a paradigm that can fully describe the entire financial market.”
In a less eclectic Socialist nation this would've been sold as Project Cybersyn or OGAS. Anyway, my guess is they're not getting subsidies from the Party any time soon.
They've made a minor splash in the ML community eight months ago, in late October, releasing an unreasonably strong Deepseek-Coder. Yes, in practice an awkward replacement for GPT-3.5, yes, contaminated with test set, which prompted most observers to discard it as a yet another Chinese fraud. But it proved to strictly dominate hyped-up things like Meta's CodeLLaMA and Mistral's Mixtral 8x7b in real-world performance, and time and again proved to be the strongest open baseline in research papers. On privately designed, new benchmarks like this fresh one from Cohere it's clear that they did get to parity with OpenAI's workhorse model, right on the first public attempt – as far as coding is concerned.
On top of that, they shared a great deal of information about how: constructing the dataset from Github, pretraining, finetuning. The paper was an absolute joy to read, sharing even details on unsuccessful experiments. It didn't offer much in the way of novelty; I evaluate it as a masterful, no-unforced-errors integration of fresh (by that point) known best practices. Think about your own field and you'll probably agree that even this is a high bar. And in AI, it is generally the case that either you get a great model with «we trained it on some text… probably» tech report (Mistral, Google), or a mediocre one accompanied by a fake-ass novel full of jargon (every second Chinese group). Still, few cared.
Coder was trained, it seems, using lessons of the less impressive Deepseek-LLM-67B (even so, it was roughly Meta's LLaMA-2-70B peer that also could code; a remarkable result for a literally-who new team), which somehow came out a month after. Its paper (released even later still) was subtitled «Scaling Open-Source Language Models with Longtermism». I am not sure if this was some kind of joke at the expense of effective altruists. What they meant concretely was the following:
Over the past few years, LLMs … have increasingly become the cornerstone and pathway to achieving Artificial General Intelligence (AGI). … Guided by the scaling laws, we introduce DeepSeek LLM, a project dedicated to advancing open-source LMs with a long-term perspective.
- …Soon, we will release our technique reports in code intelligence and Mixture-of-Experts(MoE), respectively. They show how we create high-quality code data for pre-training, and design a sparse model to achieve dense model performance.
- At present, we are constructing a larger and improved dataset for the upcoming version of DeepSeek LLM. We hope the reasoning, Chinese knowledge, math, and code capabilities will be significantly improved in the next version.
- Our alignment team is dedicated to studying ways to deliver a model that is helpful, honest, and safe to the public. Our initial experiments prove that reinforcement learning could boost model complex reasoning capability.
…I apologize for geeking out. All that might seem normal enough. But, a) they've fulfilled every one of those objectives since then. And b) I've read a great deal of research papers and tech reports, entire series from many groups, and I don't remember this feeling of cheerful formidability. It's more like contemplating the dynamism of SpaceX or Tesla than wading through a boastful yet obscurantist press release. It is especially abnormal for a Mainland Chinese paper to be written like this – with friendly confidence, admitting weaknesses, pointing out errors you might repeat, not hiding disappointments behind academese word salad; and so assured of having a shot in an honest fight with the champion.
In the Coder paper, they conclude:
…This advancement underscores our belief that the most effective code-focused Large Language Models (LLMs) are those built upon robust general LLMs. The reason is evident: to effectively interpret and execute coding tasks, these models must also possess a deep understanding of human instructions, which often come in various forms of natural language. Looking ahead, our commitment is to develop and openly share even more powerful code-focused LLMs based on larger-scale general LLMs.
In the Mixture-of-Experts paper (8th January), they've shown themselves capable of novel architectural research too, introducing a pretty ingenuous «fine-grained MoE with shared experts» design with the objective of «Ultimate Expert Specialization» and economical inference: «DeepSeekMoE 145B significantly outperforms Gshard, matching DeepSeek 67B with 28.5% (maybe even 14.6%) computation». For those few who noticed it, this seemed a minor curiosity, or just bullshit.
On 5th February, they've dropped DeepSeekMath,of which I've already spoken: «Approaching Mathematical Reasoning Capability of GPT-4 with a 7B Model». Contra the usual Chinese pattern, it wasn't a lie; no, you couldn't in normal use get remotely as good results from it, but in some constrained regimes… The project itself was a mix of most of the previous steps: sophisticated (and well-explained) data harvesting pipeline, scaling laws experiments, further «longtermist» continued pretraining from Coder-7B-1.5 which itself is a repurposed LLM-7B, and the teased reinforcement learning approach. Numina, winners of AIMO, say «We also experimented with applying our SFT recipe to larger models like InternLM-20B, CodeLama-33B, and Mixtral-8x7B but found that (a) the DeepSeek 7B model is very hard to beat due to its continued pretraining on math…».
In early March they released DeepSeek-VL: Towards Real-World Vision-Language Understanding, reporting some decent results and research on building multimodal systems, and again announcing new plans: «to scale up DeepSeek-VL to larger sizes, incorporating Mixture of Experts technology».
III. Frontier minor league
This far, it's all been preparatory R&D, shared openly and explained eagerly yet barely noticed by anyone (except that the trusty Coder still served as base for labs like Microsoft Research to experiment on): utterly overshadowed in discussions by Alibaba, Meta, Mistral, to say nothing of frontier labs.
But on May 6th, 2024, the pieces began to fall into place. They released «DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model», which subsumed all aforementioned works (except VL).
It's… unlike any other open model, to the point you could believe it was actually made by some high-IQ finance bros from first principles. Its design choices are exquisite, just copying minor details can substantially improve on typical non-frontier efforts. It pushes further their already unorthodox MoE and tops it off with a deep, still poorly understood modification to the attention mechanism (Multi-head Latent Attention, or MLA). It deviates from industry-standard rotary position embeddings to accomodate the latter (a fruit of collaboration with RoPE's inventor). It's still so unconventional that we are only beginning to figure out how to run it properly (they don't share their internal pipeline, which is optimized for hardware they can access given American sanctions). But in retrospect, it's the obvious culmination of the vision announced with those first model releases and goofy tweets, probably a vision not one year old, and yet astonishingly far-sighted – especially given how young their star researchers are. But probably it's mundane in the landscape of AI that's actually used; I suspect it's close to how Sonnet 3.5 or Gemini 1.5 Pro work on the inside. It's just that the open-source peasants are still mucking around with stone age dense models on their tiny consumer GPUs.
I understand I might already be boring you out of your mind, but just to give you an idea of how impressive this whole sequence is, here's a 3rd April paper for context:
Recent developments, such as Mixtral (Jiang et al., 2024), DeepSeek-MoE (Dai et al., 2024), spotlight Mixture-of-Experts (MoE) models as a superior alternative to Dense Transformers. An MoE layer works by routing each input token to a selected group of experts for processing. Remarkably, increasing the number of experts in an MoE model (almost) does not raise the computational cost, enabling the model to incorporate more knowledge through extra parameters without inflating pre-training expenses… Although our findings suggest a loss-optimal configuration with Emax experts, such a setup is not practical for actual deployment. The main reason is that an excessive number of experts makes the model impractical for inference. In contrast to pretraining, LLM inference is notably memory-intensive, as it requires storing intermediate states (KV-cache) of all tokens. With more experts, the available memory for storing KV caches is squeezed. As a result, the batch size – hence throughput – decreases, leading to increased cost per query. … We found that MoE models with 4 or 8 experts exhibit more efficient inference and higher performance compared to MoE models with more experts. However, they necessitate 2.4x-4.3x more training budgets to reach the same performance with models with more experts, making them impractical from the training side.
This is basically where Mistral.AI, the undisputed European champion with Meta and Google pedigree (valuation $6.2B), the darling of the opensource community, stands.
And yet, apparently DeepSeek have found a way to get out of the bind. «4 or 8»? They scale to 162 experts, reducing active parameters to 21B, cutting down pretraining costs by 42.5% and increasing peak generation speed by 5.76x; and they scale up the batch size via compressing the KV cache by like 15 times with a bizarre application of low-rank projections and dot attention; and while doing so they cram in 3x more attention heads than any model this size has any business having (because their new attention decouples number of heads from cache size), and so kick the effective «thinking intensity» up a notch, beating the gold standard «Multihead attention» everyone has been lousily approximating; and they use a bunch of auxiliary losses to make the whole thing maximally cheap to use on their specific node configuration.
But the cache trick is pretty insane. The hardest-to-believe, for me, part of the whole thing. Now, 2 months later, we know that certain Western groups ought to have reached the same Pareto frontier, just with different (maybe worse, maybe better) tradeoffs. But those are literally inventors and/or godfathers of the Transformer – Noam Shazeer's CharacterAI, Google Deepmind's Gemini line… This is done by folks like this serious-looking 5th year Ph.D student, in under a year!
As a result, they:
- use about as much compute on pretraining as Meta did on Llama-3-8B, an utter toy in comparison (maybe worth $2.5 million for them); 1/20th of GPT-4.
- Get a 236B model that's about as good across the board as Meta's Llama-3-70B (≈4x more compute), which has the capacity – if not the capability – of mid-range frontier models (previous Claude 3 Sonnet; GPT-4 on a bad day).
- Can serve it at around the price of 8B, $0.14 for processing 1 million tokens of input and $0.28 for generating 1 million tokens of output (1 and 2 Yuan), on previous-gen hardware too.
- …and still take up to 70%+ gross margins, because «On a single node with 8 H800 GPUs, DeepSeek-V2 achieves a generation throughput exceeding 50K tokens per second… In addition, the prompt input throughput of DeepSeek-V2 exceeds 100K tokens per second», and the going price for such nodes is ≤$15/hr. That's $50 in revenue, for clarity. They aren't doing a marketing stunt.
- …and so they force every deep-pocketed mediocre Chinese LLM vendor – Alibaba, Zhipu and all – to drop prices overnight, now likely serving at a loss.
Now, I am less sure about some parts of this story; but mostly it's verifiable.
I can see why an American, or a young German like Leopold, would freak out about espionage. The thing is, their papers are just too damn good and too damn consistent over the entire period if you look back (as I did), so «that's it, lock the labs» or «haha, no more tokens 4 u» is most likely little more than racist cope for the time being. The appropriate reaction would be more akin to «holy shit Japanese cars are in fact good».
Smart people (Jack Clark from Anthropic, Dylan Patel of Semianalysis) immediately take note. Very Rational people clamoring for AI pause (TheZvi) sneer and downplay: «This is who we are worried about?» (as he did before, and before). But it is still good fun. Nothing extreme. There slowly begin efforts at adoption: say, Salesforce uses V2-Chat to create synthetic data to finetune small Deepseek-Coder V1s to outperform GPT-4 on narrow tasks. Mostly nobody cares.
The paper ends in the usual manner of cryptic comments and commitments:
We thank all those who have contributed to DeepSeek-V2 but are not mentioned in the paper. DeepSeek believes that innovation, novelty, and curiosity are essential in the path to AGI.
DeepSeek will continuously invest in open-source large models with longtermism, aiming to progressively approach the goal of artificial general intelligence.
• In our ongoing exploration, we are dedicated to devising methods that enable further scaling up MoE models while maintaining economical training and inference costs. The goal of our next step is to achieve performance on par with GPT-4 in our upcoming release.
In the Appendix, you can find a lot of curious info, such as:
During pre-training data preparation, we identify and *filter out contentious content, such as values influenced by regional cultures, to avoid our model exhibiting unnecessary subjective biases on these controversial topics. Consequently, we observe that DeepSeek-V2 performs slightly worse on the test sets that are closely associated with specific regional cultures. For example, when evaluated on MMLU, although DeepSeek-V2 achieves comparable or superior performance on the majority of testsets compared with its competitors like Mixtral 8x22B, it still lags behind on the Humanity-Moral subset, which is mainly associated with American values.
Prejudices of specific regional cultures aside, though, it does have values – true, Middle Kingdom ones, such as uncritically supporting the Party line and adherence to Core Values Of Socialism (h/t @RandomRanger). The web version will also delete the last message if you ask something too clever about Xi or Tiananmen or… well, nearly the entirety of usual things Americans want to talk to Chinese coding-oriented LLMs about.
And a bit earlier, this funny guy from the team presented at Nvidia's GTC24 with the product for the general case – «culturally sensitive», customizable alignment-on-demand: «legality of rifle» for the imperialists, illegality of Tibet separatism for the civilized folk. Refreshingly frank.
But again, even that was just a preparatory.
IV. Coming at the king
Roughly 40 days later they release DeepSeek-V2-Coder: Breaking the Barrier of Closed-Source Models in Code Intelligence, where they return to the strategy announced at the very start: they take an intermediate checkpoint of V2, and push it harder and further on the dataset enriched with code and math (that that've continued to expand and refine), for 10.2 trillion tokens total. Now this training run is 60% more expensive than Llama-3-8B (still a pittance by modern standards). It also misses out on some trivia knowledge and somehow becomes even less charismatic. It's also not a pleasant experience because the API runs very slowly, probably from congestion (I guess Chinese businesses are stingy… or perhaps DeepSeek is generating a lot of synthetic data for next iterations). Anons on 4chan joke that it's «perfect for roleplaying with smart, hard-to-get characters».
More importantly though, it demolishes Llama-3-70B on every task that takes nontrivial intelligence; bests Claude 3 Opus on coding and math throughout, Gemini 1.5-Pro on most coding assistance, and trades blows with the strongest GPT-4 variants. Of course it's the same shape and the same price, which is to say, up to 100 times cheaper than its peers… more than 100 times, in the case of Opus. Still a bitch to run, but it turns out they're selling turnkey servers. In China, of course. To boot, they rapidly shipped running code in browser (a very simple feature but going most of the way to Claude Artifacts that wowed people do much), quadrupled context length without price changes (32k to 128k) and now intend to add context caching that Google boasts of as some tremendous Gemini breakthrough. They have... Impressive execution.
Benchmarks, from the most sophisticated and hard to hack to the most bespoke and obscure, confirm that it's «up there».
- Aider (2nd, 1st at release)
- LMSYS Arena (low on Overall, but 5th rank on Coding and 7 ranks above Google and Meta's open source alternatives, respectively 11th and 3+ ranks above on Hard subsample)
- Arena-Hard-Auto(7th, surprisingly 2 more Chinese models narrowly get ahead)
- %compilable Golang programs (2nd)
- Livebench (7th, by virtue of being 5th-6th in coding and reasoning and 2nd in Math; everyone above is OpenAI/Anthropic)
- LiveCodeBench (4th, same order)
- BigCodeBench(2nd)
- Gaokao-Math, released days before its deployment (roughly above GPT-4o)
Etc etc, and crucially, users report similar impressions:
So I have pegged deepseek v2 coder against sonnet 3.5 and gpt4o in my coding tasks and it seems to be better than gpt4o (What is happening at OpenAI) and very similar to Sonnet 3.5. The only downside is the speed, it's kinda slow. Very good model and the price is unbeatable.
I had the same experience, this is a very good model for serious tasks. Sadly the chat version is very dry and uncreative for writing. Maybe skill issue, I do not know. It doesn't feel slopped, it's just.. very dry. It doesn't come up with things.
Some frustrating weak points, but they know of those, and conclude:
Although DeepSeek-Coder-V2 achieves impressive performance on standard benchmarks, we find that there is still a significant gap in instruction-following capabilities compared to current state-of-the-art models like GPT-4 Turbo. This gap leads to poor performance in complex scenarios and tasks such as those in SWEbench. […] In the future, we will focus more on improving the model’s instruction-following capabilities…
Followed by the list of 338 supported languages.
Well-read researchers say stuff like
DeepSeek-Coder-V2 is by far the best open-source math (+ coding) model, performing on par with GPT4o w/o process RM or MCTS and w/ >20x less training compute. Data contamination doesn't seem to be a concern here. Imagine about what this model could achieve with PRM, MCTS, and other yet-to-be-released agentic exploration methods. Unlike GPT4o, you can train this model further. It has the potential to solve Olympiad, PhD and maybe even research level problems, like the internal model a Microsoft exec said to be able to solve PhD qualifying exam questions».
Among the Rational, there is some cautious realization («This is one of the best signs so far that China can do something competitive in the space, if this benchmark turns out to be good»), in short order giving way to more cope : «Arena is less kind to DeepSeek, giving it an 1179, good for 21st and behind open model Gemma-2-9B».
And one more detail: A couple weeks ago, they released code and paper on Expert-Specialized Fine-Tuning, «which tunes the experts most relevant to downstream tasks while freezing the other experts and modules; experimental results demonstrate that our method not only improves the efficiency, but also matches or even surpasses the performance of full-parameter fine-tuning … by showing less performance degradation [in general tasks]». It seems to require that «ultimate expert specialization» design of theirs, with its supporting beam of generalist modules surrounded by meaningfully task-specific shards, to automatically select only the parts pertaining to some target domain; and this isn't doable with traditional dense of MoE designs. Once again: confident vision, bearing fruit months later. I would like to know who's charting their course, because they're single-handedly redeeming my opinion of the Chinese AI ecosystem and frankly Chinese culture.
V. Where does this leave us?
This might not change much. Western closed AI compute moat continues to deepen, DeepSeek/High-Flyer don't have any apparent privileged access to domestic chips, and other Chinese groups have friends in the Standing Committee and in the industry, so realistically this will be a blip on the radar of history. A month ago they've precluded a certain level of safetyist excess and corporate lock-in that still seemed possible in late 2023, when the argument that public availability of ≈GPT-4 level weights (with the main imaginary threat vectors being coding/reasoning-bottlenecked) could present intolerable risks was discussed in earnest. One-two more such leaps and we're… there, for the vague libertarian intuition of «there» I won't elucidate now. But they're already not sharing the silently updated Deepseek-V2-Chat (that somewhat improved its reasoning, getting closer to the Coder), nor the promised materials on DeepSeek-Prover (a quiet further development of their mathematical models line). Maybe it's temporary. Maybe they've arrived to where they wanted to be, and will turtle up like Stability and Mistral, and then likely wither away.
Mostly, I honestly just think it's remarkable that we're getting an excellent, practically useful free model with lowkey socialist sensibilities. Sadly, I do not foresee that this will inspire Western groups to accelerate open source and leave them in the dust. As Google says in Gemma-2 report:
Despite advancements in capabilities, we believe that given the number of larger and more powerful open models, this release will have a negligible effect on the overall risk landscape.
Less charitably, Google is not interested in releasing anything you might use to enhance your capabilities and become less dependent on Google or other «frontier company», and will only release it if you are well able of getting better stuff elsewhere. In my view, this is closer to the core value of Socialism than withholding info about Xinjiang reeducation camps.
I remain agnostic about the motivations and game plan of DeepSeek, but I do hope they'll maintain this policy of releasing models «with longtermism», as it were. We don't have many others to rely on.
Edits: minor fixes
We live in a country of 350 million people. At any given time, around 1.2 million people serve in the military, or about a third of a percent of the general population. Of that third of a percent, the "tooth to tail" ratio is 8:1 (150k), but "tooth" is combat arms which includes artillery, tankers, cav, engineers etc. Even within the Infantry there are mortarmen, mechanized, anti-tank squads etc. It's hard to say for certain, but the number of dudes in the military whose job it is to kick the doors and shoot the faces is likely no more than 50k at a given time. Of that, only a minority will deploy and a minority of those see combat. In the twenty years of war in Iraq and Afghanistan, the military has awarded right about 77k CIBs and 47k Purple Hearts out of 2-3 million total deployed over that time. Obviously, there is some overlap between those awards.
For the uninitiated, the Combat Infantryman Badge is intended to identify those members of the infantry who have actually "close(d) with and destroy the enemy with direct fires.", and the PH is known colloquially as the Enemy Marksmanship Badge. But there are some caveats. For reasons I will explain, the numbers we pull from awards like this are likely a high-end estimate, rather than a minimum, but counting the Marine 0300s and whoever all else could push the numbers up a bit.
The CIB is awarded at the company level, and only in the Army. This means if one dude in an infantry company (~100 men) gets in a firefight, the whole company gets their CIB. Much of the fighting these days is very small unit engagements, so it is quite common for only a few squads or teams in a CIB company to have actually seen the combat. OTOH, more people than infantry get into shooting scrapes, I know a couple cooks with multiple engagements just because they used to volunteer to fill out patrols that were short on people. Given the nature of the conflict, a lot of people who weren't infantry, or even combat arms have seen combat. However, if they do, it's usually because of bad luck or the military ran out of infantry to do that job. It's impossible to say definitively given the data available to me at the moment, but I very much doubt the total number exceeds the number of infantrymen with CIBs. Lots of cav guys saw action as route security, lots of random MOS people got blown up en route or pressed into some role they weren't trained for. But the guys who do the job day in and day out of locating, fixing and killing the enemy is a rather select group.
So too the Purple Heart has gone to a lot of people who don't do that sort of job. Mortars dropped into a FOB can hit anyone, and roadside IEDs don't care if you're on the road to take water to an outpost or heading out on a raid. But they are more common among the people who are in the most dangerous situations more regularly.
Let's bring it all those numbers and assumptions together for a moment, because I'm describing a group of people who are very, very abnormal, and very far out on the distribution tail of the violence bell curve. Let's round up to make the math easy and account for POGs and Marines and call it a hundred thousand men over twenty years (and yes, to the closest approximation, it is all men). It's three ten thousandths of one percent of the general population. That's the high estimate, the real number could be significantly lower still. And the number who deploy multiple times is much, much lower.
When American foreign policy decides some poor dirt farmers on the other side of the globe need some freedom in their lives, maybe one ten thousandth of one percent of the population is who gets sent to do the actual violence of empire. I am one of those men. I have a CIB and a purple heart. Within the rarified community of professional actual soldiers, I am a small fish in a tiny pond. I was not special forces (or, technically I was briefly, but not really). I was a reasonably high-speed infantryman with a penchant for guns who worked himself into a sniper platoon in a fairly trash unit. I made sergeant, ate an IED and got med-boarded out of the military. A short, somewhat spicy but relatively unremarkable military career for an infantryman.
Much of what the general public hears about combat, even "first hand accounts" is not from people who actually do this job. As I have hopefully established, this is a very small, very highly selected and very abnormal group of people. Most of what you read ore hear in war accounts is from the middle classes, which in the military means officers. Officers are not soldiers. They are managers of soldiers. During conscription, it was at least possible for a southern gentleman of letters like Eugene Sledge or a Junker scion like Junger to write an account of actual enlisted combat, if unlikely. In today's volunteer military, this is almost never the case. The venn diagram of actual front-line soldiers and people who can write competently in an educated manner for general consumption is essentially two separate circles. These are not generally guys with college degrees, because if they had one they'd be an officer. There are exceptions, but we'll perhaps get into that at another time.
This hopefully will go some way to explaining my arrogance in writing about the topic. IQ and violent tendencies tend to be negatively correlated, and I am way out on the right tail of of the distribution on both. If that seems self-aggrandizing, rest assured that neither has done me any good.
So who are these men? Who carries the torch of empire into the barbarian wastes of the Korangel? Who sits behind the machine gun of a HMMWV? Who donkey-kicks the doors off their hinges and plunges into the black interior following the blinding light of his Surefire? Who sits in a ditch for three days waiting to shoot a retarded teenager whose dad got paid $200 to have him drop an IED in a pothole?
In short, they're degens. Poor and working class kids, half of which aren't old enough to buy beer. Mostly rural whites and hispanics. Roughly a quarter are from Texas alone. The South more generally provides well over half, maybe two thirds of the total. Most of the rest are from the Midwest and West. They self select. These are guys who asked to be in the Infantry, and survived the training and indoc. Nobody winds up on the pointy end of the spear by accident. It's the worst job in the military, so the people who volunteer for it are driven by very different considerations to most. It's also the highest status within the violence hierarchy.
It's a weird group. There's a lot of immigrants, not all of them hispanic. A surprising number of professional soldiers from other countries come to the US just for the action. If you're just itching for a fight but are born somewhere too peaceful, coming to the US will greatly increase your likelihood of getting into the shit. I've met British Marines, African princes, a German seminarian who dropped out to join the US infantry. There's some tiny minority communities that are heavily overrepresented though still small in total numbers, like Native Americans, the Samoans, the Hmong and the Sikhs. East asians are rare, and black Americans, while overrepresented in the military generally, are underrepresented in this part but still common.
We are united by a few characteristics, most generally the privileging of suffering as a badge of honor, physical violence, and a death wish. One does not join the Infantry to live a quiet life to a ripe old age. Mostly, I think we want to fuck around and find out. Like the movie says, how much can you know about yourself if you've never been in a fight? I had a lot of reasons for joining up, but I think the most basic was a desire to test myself and find out if I had what it took to face another man in mortal combat. To "see the elephant", or any of the other thousand euphemisms we use for fighting and killing our fellow men. To take the ultimate risk.
A recruit can get all the benefits of military service in a safer and more sustainable career. The infantry chews up men, minds and bodies, training alone eliminates hundreds of thousands. A twenty-five year old infantryman is probably middle management, a thirty-year old is probably out of the field as a platoon sergeant. The incessant road marching ruins feet, ankles and knees. The heavy packs wreck spines and shoulders. The heat, cold, wet and sleep deprivation cull the sensitive and the civilized. The social aggression removes the timid and the hazing removes the bitches.
There's an old glib saying that captures the esprit of the group. "The cowards never started and the weak quit along the way. That just leaves us.".
As a composite character, I give you the US infantryman. He is nineteen or twenty years old, grew up in a trailer park, has a kid or two with women he's not married to, is married to a woman with kids that are not his. He'll be divorced in a year. His family are construction workers, nurses, truck drivers, retail workers, garbage collectors, heavy machine operators, drug dealers, petty criminals, major criminals. He is dumber than average, hated school, has never read a book not assigned in class. He's been in jail multiple times, and probably will be again, mostly for low level stuff like underage drinking, vandalism and fighting. He binge drinks and smokes when in garrison, dips in the field. He gets in fistfights on a regular but extended basis with members of his own unit, in group conflict with other units, or with civilians on liberty. His politics, if he has any, are somewhere to the right of Genghis Khan. He drives a pickup truck, a Mustang, or a heavily riced-out import and is dead broke most of the time. He is, in short, perilously close to the underclass of our society, and there's a lot of crossover. His life is boredom, fear, pain and the brotherhood of those who live in fear and pain. His values are foreign, rude and frightening to those not of his group.
He is the world elite of the violent class. The modern equivalent of a knight, loaded down with many years' wages' worth of technology, weapons and armor. Far better trained, supplied and equipped than his adversaries. The big stick that the world hegemon swings in the anarchic world of international politics. The very tip of the spear. The point of empire.
In our modern peaceful society, that point has become very fine indeed.
Let me tell you a tale, young private, of my time after the service. Back in civilian land, a stranger to my own people. I got drunk for the better part of a decade. You think you want this life, listen up, because there's a price to pay for those stories.
When you're a drunk, your social circle is mostly in bars. I had my local. It wasn't the nicest or the grubbiest, it wasn't a meat market. It was a quiet, steady bar with a slightly older clientele. Dark inside, even darker booths. Stamped copper ceiling, left over from a wealthier time. A bullet hole over the bar that the owner claims happened during Prohibition, but I heard was from a drive-by in 1996.
This is the Nasty, and it's right on the strip. Hamilton Street. Five blocks of bars, restaurants and coffee shops with assorted tattoo parlors, barbers, bike shops and bail offices. Just across the river, the East side. One of the worst square miles in the US. Perennial murder capital contender, desperately poor, the abandoned urban underclass in the wake of de-industrialization. There were once over a hundred factories in Saginaw. Now there's three.
That's one three hundred yard bridge away from Hamilton Street, and it's the meeting place for the three sides of town. To the south, the Mexican quarter, to the west, the Township. The business owners on Hamilton are young and old. Some old proprietors hanging on for dear life. Some new ones filled with vision and ambition. All of them with more hope than sense. The town is dying.
I'd only been in town about four years at the time, fresh out the service and living with my younger brother, also just back from Iraq. It was a rough town, but we'd come from rougher.
Down to the pub, I like to think I was an excellent patron. Quiet, polite, my tab got paid in full, the staff got tipped every time, and well. A good pub can take years to get “in” to, but it's faster if you don't stiff the staff. You join the community. Learn the rules. Follow the code. The regulars and the semi-regulars. You start to learn about people slowly, over time.
John, an ever-so-slightly aged queen holds magnificent court with his coterie of fag-hags. It's always a fun time when they're in. Dom is rich or something and is forever buying rounds. Regulars stalk him. Neil, the manager, is forty and dating a different twenty-year-old every week. Nan, the heroically ugly and cantankerous bartender. She once threatened to “slap the dipshit out of your [my] face” for calling her “ma'am”. I wasn't going to try her.
But this story is about another regular. Name of Cowboy. He only comes in early in the day or late at night. A bit shorter than average, rail thin, all corded muscle, bad ink and long scraggly goattee. Works second shift, and hustles afterward. He makes the rounds at the bars late at night, sells beef jerky. Looks sketchy as hell, comes in a ziplock bag, but it is excellent. Kind of like Cowboy.
Now, Cowboy likes a drink, and I like drinks and beef jerky and we can both smell blood on the other. He's wizened, old before his time. Hard living, no doubt. He has full dentures, teeth knocked out in a prison fight. He's been in three times, he rides for the Outlaws, has the badges, has the ink. He's on parole for another ten years. I still don't know the details.
We bond over telling scar stories, every old soldier's favorite game. The stories start funny and get dark. There's some things you can't discuss with someone who can't directly relate, and when you find someone like that, there's a context to it. You don't understand now, you may later. It takes a while. You don't kiss on the first date with a guy like that. You gotta feel it out, get comfortable. Here we were, relative nobodies to the rest of the world, a broke-dick soldier and a lifelong criminal. But within our respective tiny subcultures, we were powerful and respected elders. The reason you're listening to my stories, young private.
Three terms, three deployments, three war zones.
Let me tell you son, my stories got nothing on old Cowboy's! I mean, mine might be crazier, and happen in a more exotic location, but his were so much more traumatic. It's one thing to go into battle with the might and money of a world superpower at your back, and another to have nothing and no one but yourself, and no win but more prison time with your opponents at the other end. Dude was a hard, hard man.
Generally kept my nose clean, by infantry standards. A bit of jail here and there for fighting, public drunkenness, stealing the flags off a golf course once...nothing serious.
So anyway, back to the story. The bar. The community.
I've been drinking in this bar for four years. I'm on my third set of owners. I've been here before most of the bartenders started. Now, one of the benefits of being a trusted regular is the lock-in.
So the boys hold a lock-in without us one night, lo and behold the cash is all gone in the morning. The whole night's takings. And half the liquor. None of this is discovered until Neil opens in the afternoon, and my brother and I wander in not ten minutes later.
The place is a bit of a mess, we help clean up and start figuring out what happened. Neil checks the security footage, and there it is, plain as day. The night bartender left the keys with one of his friends to lock up, and the guy cleaned the place out. We know this guy. We know his address. Nobody can think of his name right off, but who gives a shit?
The cops are no help, they'll take a statement but don't have time to waste on a few thousand in loose cash. This is bad for the bar, but it's really bad for those of us who like lock-ins.
Cowboy turns up, we brief him on what happened and he has a suggestion. For ten percent of the lost cash, he'll go over to the kids house with a couple of his Outlaw boys, put a bit of a fright into him, get him to give back the stuff. Way faster and cheaper than the cops!
Neil calls the owner, he says handle it. Cowboy goes outside to clear it with his boss in the MC. Comes back with bad news, the club won't sanction its guys for debt collection, and apparently this counts. Plan B.
What are the odds this popped-collar fuckwit knows what a proper Outlaw cut looks like? Are we outsmarting ourselves? Dan rides, he has a cut from a veteran MC. I used to ride, but sold my bike in Cali. We get on the phone.
Raoul is sixty-four, single, alcoholic. Looks like hispanic Colonel Sanders. I think he's been drunk since the seventies. He had a tough tour in Vietnam, I met him at the Purple Heart meetings. Sweet guy, a quiet and melancholy drunk, but good humored when roused. An inveterate poon-hound. He's a good dude, and hooks us up with a bike and a convincing-looking vest.
Cowboy has to work, so Dan and I do a basic recce. Walk the street, check the alley, count the exits. Windows are small and mostly high up, he's not gonna crawl out through those most likely. Three doors, one to the garage.
Back to our apartment. We talk over the plan, the scope. We leave weapons at home, pepper spray only. We're in sketchy legal area here. We aren't committing a crime, but we're going to be on his property and not necessarily friendly. If things go sideways, we don't want to escalate any more than necessary to break contact. We can always come back with more hardware, or men.
Cowboy turns up at midnight, and we roll out into the damp, dark night. Loud. This isn't a sneak operation, this is about intimidation. The whole neighborhood is going to peek out at this little show.
Fuck, I nearly forgot how much fun motorcycles are. And how incredibly scary they are when you ride with maniacs. I'm a highway cruiser, Dan has a death wish, and Cowboy was born on a bike or something. We come down that quiet cul-de-sac like thunder, line all three bikes up with the headlights pointed at the front door. My adrenaline is off the charts, I am not that good on a bike.
Dan splits off to the back, the garage is to our right, but the external door is closed. Cowboy mounts the steps to the porch, I stay one step back and to his left. The cut is a bit loose, Raoul is a lot thicker. Three loud raps at the door. Just enough to bounce the hinges a bit, you know? Take note, young private. Your knock should loosen a screw or two. Makes a good first impression.
Fuckwit comes stumbling to the door, must have slept the day. Queasy looking. Comes out of the door! Ok, we got this, this dude is not going to be a problem. If he barricaded, we might have had a time. He's quite a bit bigger than any of us, but that won't matter now.
He's disoriented, blinded by the lights behind us. He looks for a long moment at Cowboy, then at me. He's outside, the door is behind him. He's wearing sweatpants and flip flops. We're both within four feet of him. He knows us, but not by name, and not with biker gear on. We're both holding our helmets. He turns to go back through the door, and he can see straight down the hallway through the back sliding glass door, to where Dan is standing on his patio.
Cowboy puts his hand softly on Fuckwit's shoulder.
“Put everything back in your car, and take it back to the pub. We'll follow you.”
Fuckwit looks back at me. I give him my best evil grin.
He packs his car. People are looking. It's nearly one AM now, and suddenly people are wandering up and down the sidewalks, cell phones in hand. Whatever, Nasty PD ain't crossing the river for a noise complaint. It takes an uncomfortable amount of time though. That was a lot of hooch.
The escort is a good excuse for me to fall back and tail Fuckwit's ancient Buick. Dan and Cowboy are blasting up and down the wet, empty streets, popping wheelies, Charlie Mike. We take the scenic route through the neighborhood. It's last call when we roll back into the pub.
Neil boots the stragglers, makes Fuckwit restock the liquor and bans him from the bar. The whole of Hamilton Street already knows. Cowboy gets his ten percent, Dan and I get what turns out to be quite a lot of free drinks, and a special reward. We call Raoul to come down to pick up his bike and join us for a lock-in.
And that, ladies and gentle privates, is how I got a seat at the bar.
They've torn the place down now, but somewhere there is a small brass plaque that used to be nailed in front of the back corner stool of the pub.
Excuse me, but that's my seat. Is my name on it? Yes, yes it is. Would you like to speak to the manager?
Coda:
Cowboy died last year. Mid fifties,god knows of what. My brother rode up from North Carolina for the funeral. Six hundred people joined his wake. I had to borrow a bike. Older now, softer and relatively sober. I gave up the seat for a wife and a quiet life in the Nasty burbs. Dan has two kids and travels for work. Raoul is dead, years ago. There were no women at his funeral. I still see Neil from time to time. He's still dating twenty-year-olds and managing a bar. I never saw Fuckwit again. Still don't know his name.
I sometimes go back to the old neighborhood and walk its mountainous sidewalks, check in on old neighbors. Maybe have a drink at Neil's new place. Remind the streets. We're only old, we ain't dead yet.
It's not just that my clients lie to me a lot, which will only hurt them --- it's that they're really, really bad at it.
[Originally posted on Singal-Minded]
My job as a public defender puts me in a weird place. I am my clients' zealous advocate, but I'm not their marionette. I don't just roll into court to parrot whatever my clients tell me --- I make sure I'm not re-shoveling bullshit. So for my sake and theirs, I do my homework. I corroborate. I investigate.
A significant portion of my job ironically mirrors that of a police detective. Every case I get requires me to deploy a microscope and retrace the cops' steps to see if they fucked up somehow (spoiler: they haven't). Sometimes I go beyond what the cops did to collect my own evidence and track down my own witnesses.
All this puts some of my clients of the guilty persuasion in a bind. Sure, they don't want me sitting on my ass doing nothing for their case, but they also can't have me snooping around on my own too much. . . because who knows what I might find? So they take steps to surreptitiously install guardrails around my scrutiny, hoping I won't notice.
You might wonder why any chicanery from my clients is warranted. After all, am I not professionally obligated to strictly maintain client confidentiality? It's true, a client can show me where they buried their dozen murder victims and I wouldn't be allowed to tell a soul, even if an innocent person is sitting in prison for their crimes. Part of my clients' clammed-up demeanors rests on a deluded notion that I won't fight as hard for their cases unless I am infatuated by their innocence. Perhaps they don't realize that representing the guilty is the overwhelmingly banal reality of my job.[1] More importantly, it's myopic to forget that judges, prosecutors, and jurors want to see proof, not just emphatic assurances on the matter.
But clients still lie to me --- exclusively to their own detriment.
Marcel was not allowed to possess a firearm. And yet mysteriously, when the police arrested him --- the details are way too complicated to explain, even by my standards --- in his sister's vehicle, they found a pistol under the passenger seat.
"The gun is not mine. I don't even like guns. I'm actually scared of guns." He told me this through the jail plexiglass as I flipped through his remarkable résumé of gun-related crimes. Marcel spent our entire first meeting proselytizing his innocence to me. Over the next half hour he went on a genealogy world tour, swearing up and down on the lives of various immediate and extended members of his family that he never ever ever touched guns.
I was confused why he perseverated so much, but I just nodded along as part of my standard early precarious effort to build rapport with a new (and likely volatile) client. What he was telling me wasn't completely implausible --- sometimes people are indeed caught with contraband that isn't theirs --- but there was nothing I could do with his information at that early stage. Maybe he thought if he could win me over as a convert, I'd then ask for the case to be dismissed on the "he says it's not his" precedent.
Weeks later, I got the first batch of discovery. I perused the photographs that documented the meticulous search of his sister's car. I saw the pistol glistening beneath the camera flash, nestled among some CDs and a layer of Cheetos crumbs. And on the pistol itself, a sight to behold: to this day the clearest, most legible, most unobstructed fingerprints I have ever seen in my legal life. If you looked closely enough, the whorls spelled out his name and Social Security number.
Public defenders are entitled to ask the court for money to pay for private investigators, digital forensic specialists, fingerprint examiners, or whatever else is needed to ensure a defendant in a criminal case is provided with his constitutionally guaranteed legal bulwark. The photographed prints here were so apparent that an examiner could easily rely on the photos alone to make a comparison.
Marcel had earned himself some trolling from me. I went back to see him at the jail, faked as much enthusiasm as I could muster, and declared, "Good news! They found fingerprints on the gun!" He stared at me stunned and confused, so I continued.
"Well, when we first met, you told me that you never touched the gun," I reminded him with an encouraging smile. "Obviously you wouldn't lie to your own lawyer, and so what I can do is get a fingerprint expert to come to the jail, take your prints, then do a comparison on the gun itself. Since you never touched the gun, the prints won't be a match! This whole case will get dismissed, and we can put all this behind you!"[2]
He was still reeling but realized I was waiting for a response. "You. . . don't need to do that," he muttered. I had the confirmation I was looking for, but I pressed him while maintaining the facade of earnest congeniality.
"But why not?" I sang in staccato, smile wide. "You told me. That. You. Never. Touch any guns."
Turned out Marcel might have accidentally touched the gun. So his prints could be on it. I had made my point, so I dropped the act. I explained to Marcel that the only thing lying to me accomplishes is to slow things down and worsen his own prospects --- how could I pursue any potentially helpful leads for his defense when I couldn't be sure I wasn't about to bumble into an incriminating revelation?
Marcel nodded sagely and claimed to understand, but he went on to lie to me many more times over the next two years that I remained his attorney. Marcel has and will spend the majority of his adult life in prison --- not necessarily because he lied to me but that certainly didn't help.
My first meeting with Kyle was useless. He insisted throughout that it wasn't him, that he wasn't even there. Now, personally speaking, if several witnesses claimed to have seen someone who looks like me, in my car, with my girlfriend in the front seat, commit a drive-by shooting in broad daylight, I would summon slightly more curiosity about who this apparent doppelganger might be. But Kyle gave me no leads, pantomiming an internal agony about not wanting to be a snitch, clutching at his stomach as if the mere thought was physically unbearable.
His tune eventually changed. "I need you to tell the prosecutor who was driving my car," he said."His name is Richie Bottoms." If the name hadn't given it away, I already knew where this was going,[3] and I was excited for the coming entertainment. I pretended to be enthused by his revelation, and let Kyle know that I had a "really great" investigator who's phenomenal at tracking "anyone" down --- even the elusive Dick Bottoms.
Based on his reaction, that wasn't the response Kyle expected; another illustration of a myopic theory of mind (not uncommon among the interpersonally inept) incapable of simulating anything but affirmation. He tensed up momentarily, but realized that he'd already committed himself to acting out a demeanor congruent with the "innocent client responds to helpful attorney" fantasy. Yet the only excuse he could muster up in the moment was that Richie wouldn't be found because he fled to Los Angeles.
I maintained what must have been an obnoxious level of optimism, explaining how "perfect" that was because my investigator "knew lots of people" there. My job affords me few if any moments of joy, and so forgive me if I overindulged in Kyle's vexation. I'll spare you a full accounting of the myriad reasons he gave why tracking down Sir Bottoms was a lost cause. Suffice to say that in addition to being out of state, Richie had maybe fled the country; also, Richie happens to look almost identical to Kyle, but also we might not even know his real name since he went by "Arby," and no one had his phone number, et cetera. . .
Even when we moved on to other topics, Kyle couldn't let it go, interrupting whatever we were talking about to repeat warnings about how tracking down Richie was going to be a total waste of time for my investigator and me. He was palpably angry, but had no viable outlet for his frustration, and so he just stewed, stuck with his lie. I kept my poker face. It's a stark contrast to my factually innocent clients, who cannot help but drown me with leads to pursue in the hopes that any are helpful.
The whole thing reminded me of Carl Sagan's parable of the dragon in his garage as a critique of certain unprovable religious beliefs. Can I see the dragon? No, it's invisible. Can I detect its fire's thermal image? No, the fire is heatless. Can I find Dick in Los Angeles? No, because now he fled the country.
There's always some excuse --- there's always some eject button allowing my defendants to evade specific evidence demands. No matter how ridiculous.
It's banal for my clients to deny the accusations, but a special breed takes denial to the next level by waging total jihad against their accusers. It's a sort of a reverse counterpart to the Narcissist's Prayer:
If they claim I was driving during the hit-and-run, they're lying. And if they're liars, then they exaggerated their injuries. And they're exaggerating because they're after an insurance payday. And we know they're after a payday because they sued their dry cleaners in 1993. And they're framing me to get money, which is how we know they're lying.
In these clients' telling, nothing is their fault. The random bystanders who randomly drew the unlucky witness card become a convenient scapegoat. Yet these clients are so myopically overwhelmed by the desire to bounce the rubble on a witness's credibility, they don't notice how implausible their story becomes with each new clause they tape onto their fabulist's scrapbook.[4]
Sometimes clients are self-aware enough to couch their denials in innuendo. Ivan, who was accused of [redacted], was waging the same Total War approach against Cindy, a social worker at the homeless shelter where Ivan regularly stayed. Cindy was a dangerous witness --- an uninvolved, respected professional who severely undercut Ivan's alibi defense about having never left the shelter to go on his [redacted] spree.
In yet another of our jail rendezvous, Ivan expounded at length about how Cindy's testimony was invalid because, as a social worker, she would be violating HIPAA.[5] The glaze over my eyes must have gotten too obvious for me to hide, so he switched tack, shuffled through his jail-sanctioned filing system (read: pile), and slid a flyer across the table about trash cleanup day at the shelter, with a smiling cartoon trash can picking up a baby garbage bag while announcing "Pick up a little trash, talk a little trash." It's cute, but what the fuck was I supposed to be looking at? Ivan stared at me grinning and expectant, but his demeanor quickly turned into disappointment at my ongoing silence. He snatched the flyer out of my hand and jammed his finger at the "talk a little trash" clause. "This!" he shouted, and then just stared at me again. I looked at the words that meant so much to him and nothing to me and just said, "Huh?"
His disappointment transmogrified into astonished anger. "Do I have to fucking spell it out for you?" he screamed. "I thought you were the lawyer here!" We had been ping-ponging across various aspects of his case for the last hour or so and I gave up on any posturing and reiterated my ignorance at the significance of the cartoon flyer. Ivan snapped, "Cindy is encouraging people to trash talk!" For, you see, she wrote the flyer. "I'm trying to show you that she's a fucking punk! And a liar!"
I immediately understood why Ivan was so attached to remaining within the realm of innuendo. Because as soon as he gave his claim some body ("We should infer lack of credibility from individuals when they author flyers that include garbage-related puns"), he knew how much of a dumbass he would sound like out loud.
Ivan moved on from the flyer, and instead asked how to disqualify a witness "for being a liar." I tell him that's not a thing,[6] which sent him into a further rage. "I need you to be on my side here but all I hear from you is 'NO.' Why are you working for the prosecutors?"
The manipulation attempts we just cataloged were comically inept, and fell apart with far less effort than it took to create them. Slightly more polished versions of these charades are regularly deployed within the Discourse™ but they're equally hollow and just as pathetic. So those are some of my clients --- individuals who cannot rise to the level of your average internet troll.
[1] There is a kernel of an exception that is almost not worth mentioning. The Rules of Professional Conduct 3.3 obligates me with the duty of candor. I am not allowed to present evidence that I "know" is false, which encompasses witness testimony. Some jurisdictions make exceptions to this rule for defendants testifying in their criminal trial (correctly, IMO) but not all. So assuming that a client truthfully confesses to me, assuming we go to trial, assuming they decide to testify, and assuming I "know" they're going to lie, then yes, this could indeed spawn a very awkward situation where I'm forced to withdraw in the middle of proceedings.
[2] I'm told I put on a good poker face.
[3] There was no Richie Bottoms.
[4] For example, Kyle asked if it was possible to present self-defense evidence on behalf of "Richie Bottoms," just in case.
[5] Does this sound familiar to anyone?
[6] During the editing process, Jesse was skeptical of this. "Wait," he asked me in a Google Doc comment, "there's NO way for one side to prove to a judge that a witness is so untrustworthy the jurors/judge shouldn't consider their testimony?" Correct. The closest rule is disqualifying a witness as incompetent, either for being too young, severely mentally ill or mentally retarded, or too intoxicated (on the witness stand!). Credibility is up to the judge/jury to decide, and if a witness has a history of lying, then it makes for a very easy credibility impeachment. Theoretically, in extremely rare circumstances, a judge could strike the testimony of a witness or find them in contempt, but they'd have to be seriously flagrant about their lying under oath. I have never heard of this happening.
Contrary to well-established popular opinion, I'm not actually right about everything. I'm human and, sometimes, I make mistakes or otherwise fundamentally change my opinion on a topic. When I first sat down to write this list, one of the items was substantive enough to inflate into its own stand-alone post (Defunding My Mistake). Although unintentional, this does carry the misleading implication that the mistakes I make are exclusively of the rare and soul-searching variety. My original intent was to analyze errors in order to showcase how banal or even reasonable they can be. Part of my goal here is to nudge the act of acknowledging one's errors into the realm of the common & boring, and away from the tearful confession elicited only through torture. I hope to encourage others that it's OK and maybe even admirable to admit errors.
What follows is an incomplete listing, and my primary goal here isn't just to delineate what but to provide a detailed account for why. When picking examples to highlight, I wanted to cover a diverse palette of failure scenarios and so they're not intended to be a representative sample. Also, please note that if I offer an explanation for why I made a mistake, it shouldn't be interpreted as an excuse to shirk responsibility.
Paper Rips 4 Allah
I'll spare you the novel I could write about how and why I abandoned Islam and instead I'll focus on one particular incident. I must have been around 14 years old or so, wandering the stacks at the local library, when I encountered a Chick tract about Islam called Allah Had No Son. Chick tracts were widely distributed pocket-sized short comic book strips intended to impart evangelical Christian messages, typically through combative and antagonistic messaging.
The tract basically argues that Islam is a false religion because it was based on repurposed tribal moon deities. I have no idea how much of this is true and don't care, but my reaction at the time was livid anger. Here I was encountering some new information about a topic I was (fanatically) enthusiastic about but instead of "hmm that's interesting" I responded by making it my mission to scour the rest of the library and rip up any other Chick tracts I could find. I remember my heart racing and this distasteful feeling that I had somehow been mind-poisoned by a comic strip, and I did all I could to wipe my thoughts clean as if it was a radioactive waste clean-up mission.
That cleanliness desire is what I remember most, the notion that I couldn't even entertain the "noxious" ideas even just to mount a rebuttal because the risk of an incurable infection was too great. And so my only recourse was to suppress and bury. I see this burial reflex in full-grown adults today and all it reminds me of is a shaken 14 year-old thinking he's saving the world from damnation by ripping up paper.
Wrong About Wrong About
I was a big fan of Sarah Marshall & Michael Hobbes's You're Wrong About podcast, listened to dozens of their episodes, and heartily recommended it to others. My impression of the two is that they were unusually diligent reporters who devoted an incalculable amount of research behind each episode. I recall at one point they claimed each individual 1 hour-ish episode took 8 to 10 hours to record and was preceded by several weeks of research. This claim seemed and remains totally credible to me, because I can't imagine how else they would have been able to release sixteen hour-long episodes on the OJ Simpson case full of obscure minutiae without first having read several books on the topic.
The problem here is Hobbes specifically, his selective devotion to the truth, and why I didn't notice it before. Freddie deBoer's list reserved a scathing paragraph for Hobbes:
The quintessential 2022 liberal is someone who does not want to achieve anything, but rather to be something - an ally, a friend to the movement, one of the good ones. Achieving is beyond the point; the point is to occupy a space of existential goodness. For people like Hobbes, politics is not a thing you do but a thing you are. And what Hobbes is, naturally, is a guy who already knows the answer to every question.
As an illustrative example, see how credulous Hobbes is towards spurious claims which just happen to flatter his preconceived conclusion that Jesse Singal Bad. By far Hobbes's most telling confession comes from the 2018 You're Wrong About episode on the murder of Matthew Shepard. Amazingly, the transcript remains up (emphasis added):
My longtime obsession with this case and the debunking is about our use of symbols and our use of cases to illustrate larger phenomena. You saw this a lot with Michael Brown actually, and with Trayvon Martin. That those cases come out. It's horrible. That's used as a tag to talk about police killing African Americans at wildly disproportionate rates. And then everybody pops out of a trashcan and is like, actually Michael Brown, it looks like he fought back against the officer. Or maybe Trayvon Martin was shoplifting that day. And they try to complicate the narrative of this anecdote on which we've hung this larger trend. And frankly, who fucking cares? Maybe everything that the racists say about the Michael Brown case is true, and maybe everything they say about the Trayvon Martin case is true. That does not negate the fact that statistically speaking African Americans are more likely to be killed by police than white people. So, it really doesn't matter whether they are correct about their "debunking" of these cases. But to make a trend interesting, to make a trend important, you have to tie it to these events. And then we get into these events being more complicated than they seem at first, which fucking every event is more complicated than it seems at first. [...] And so then we start to complicate this narrative and then the entire edifice of the social problem falls apart. They say that cops are killing black people at disproportionate rates, but I read on Breitbart that like this Michael Brown kid was fighting with the officer, and the whole thing gets swept away. And I think it's just something human and a huge weakness of journalism that you have to tie bigger trends to these stories. And then once the story gets debunked, the trend gets debunked. [...] The thing that I think is really hard for people to incorporate is that even if all of the debunking about Matthew Shepard was true, or even more true, let's say he was trying to sell them meth and he was this huge meth kingpin, and he's just this terrible human being, it still doesn't stop the fact that he's gay and he got murdered. And it still doesn't stop the fact that homophobia in 1998 in America was a huge problem. And that many gay people were killed or beaten up or harassed or whatever due to their sexuality. So even if the debunking of the Matthew Shepard case was true, it doesn't negate the larger point.
Hobbes, an alleged journalist, admits it is acceptable to circulate factually false narratives if they happen to be in service towards a broader morally true mission. I hadn't listened to that episode but if that was admitted to in 2018, why didn't I notice the problem earlier? Partly it's because I assumed that diligence is completely incompatible with dishonesty. The other part is that Hobbes is not uniformly averse towards questioning sacred cows. In 2019 for example, during Pride month no less, he was willing to unambiguously reject the "A transwoman threw the first brick at Stonewall" canard, although admittedly not without some gratuitous and familiar excuse-making:
I think a lot of this putting Sylvia and Marsha back into the Stonewall narrative is completely understandable because they are much more representative of Stonewall, then the hot white 2% body fat people that have typically been celebrated for this kind of event.
So what now? By Hobbes' own admission, I can't trust his work on any subject since I can never know if he's relaying something factually true or just morally true. But did I go back and scrutinize everything else I picked up from listening to YWA? No. That kind of forensics is just not practical and also, Hobbes isn't just operating an opposite day machine where he reflexively relays the opposite of whatever his research says. I'm willing to wager that he's factually accurate the overwhelming amount of the time, but all you need to fully pulp your credibility is admit you're willing to bend the truth sometimes.
Legal Forecasting
I wrote about the bombshell revelation during the Proud Boys trial of an FBI agent caught lying in her testimony. I included a prediction of sorts: "My assumption is that the prosecutor will dismiss charges against Nordean in a feeble attempt to make this go away." Gattsuru righteously pointed out that this did not happen; the trial continued and all defendants were found guilty.
Obviously I cannot see the future so why should this failed prediction be on me? Well it's mostly a reminder that I should stay in my lane. One of the things I (hopefully) offer in my writing on legal topics as a criminal defense attorney is the background experience necessary to contextualize events, like how there's nothing at all remarkable about a defendant pleading not guilty at arraignment (dramatic headlines notwithstanding). I frame conversations with my real clients with similar qualifications, something like "While I can't predict the future, I have done hundreds of sentencings and I would be very surprised if X happened instead of Y." Neither my readers nor my real clients should have any business listening to what I have to say if I continue to fuck up my crystal ball.
It's possible for a prediction to be wrong but still be reasonable when offered at the time, and it remains possible that I would be vindicated by some future appeal decision. Even so, that would be an instance of being accidentally correct. I should not have made that prediction (no matter how weakly-worded it was) for a couple of reasons:
-
My criminal defense experience is overwhelmingly in state court, not federal court. I lack the necessary context to confidently interpret events in the latter. Let's just say that it's much easier to catch a state prosecutor tripping with their pants down.
-
My own bias as a defense attorney (and really, virtually the only time I get to do something useful at work) is to make hay out of the government's fuck-ups, only to thereafter be dispelled of the festivities once the prosecutor's reply brief comes in. In the Proud Boys case I relied entirely on just the defense motion as the prosecutor's response had not yet been filed.
Hopefully I can keep my limitations in mind...but who can predict the future?
Overestimated Immunity
In the same post above, I claimed that Qualified Immunity was "practically speaking, basically absolute immunity with a few extra steps". QI is definitely one of my hobby horses that I've written extensively about and yet, curiously, I never looked into how prevalent it is. Had I been asked at the time to predict how often QI is granted as a shield against §1983 civil lawsuits, I probably would have said around 80%. The real answer (thanks to Gdanning) is somewhere between 57% and 3.7%.
Regardless of what the real answer is, the fact that I never bothered to look it up was a big mistake on my end. All it took to answer the question was the same cursory research that I regularly excoriate others for not doing. I think this error was paradoxically the result of my enthusiastic interest on the topic. Once you're drowning within an issue it's much easier for the availability heuristic to take over. Something similar happened to Matt Walsh when he erroneously claimed on Rogan's show that "millions of kids" were on puberty blockers.
DoNotTrust
DoNotPay used to advertise itself as the World's First Robot Lawyer, now it's has rebranded into just Your AI Consumer Champion. The reason for the rebranding might have something to do with how DoNotPay's CEO, Joshua Browder, was exhaustively exposed as a flagrant fabulist by Kathryn Tewson, and he's the target of a lawsuit by the same.
When I first heard of DoNotPay, it was within the context of deploying a chatGPT-like agent on a company's customer support chat system in order to dispute bills and the like. That idea was and remains perfectly plausible (customer service reps are trained to follow a script after all) and so when Browder made news with his ridiculous $1 million SCOTUS offer I said that the stunt risked hurting DoNotPay's "promising product".
It was a tweet that barely got 100 views but I didn't have enough information to make that declaration. I also feel a bit sore about this one because I shelved my usual skepticism on a topic within my wheelhouse and got outscooped by Tewson on a major story. Darn.
"The Law That Created The Internet"
I already wrote about this a while ago. I used to be a §230 devotee but reading Gilad Edelman's article changed my mind about whether the federal law is as necessary to the existence of the internet as I thought it was. There's no shortage of arguments in favor of §230 but one errant thought I completely failed to follow up on is investigating how exactly the rest of the world handled the issue. Presumably not every country in the world copied §230 verbatim and yet the world-wide web still exists. I didn't dwell too long on that question and shelved it away with some glossed-over "maybe that's why all the tech companies are in the U.S.".
The other question I failed to pursue was if we were to assume that a world without §230 would be as cataclysmic as its proponents argue, why would it stay that way? The whole point of the internet was allowing people across the globe to communicate. It seems patently implausible that if §230 did not exist everyone would just shrug and stoically accept a world where everyone is too spooked by the threat of defamation lawsuits to allow any user-generated content. Admittedly this is on dodgy aspirational ground in the vein of "we'll figure something out" but it illustrates how helpful it can be to contemplate how exactly people (including legislators) will respond and not just assume they'll sit and helplessly awwshucks while the fire burns. I still think §230 is a good solution but it was a failure of the imagination to assume it was the only solution.
The other mistake I made on this subject was to reflexively reject §230 criticism, even in areas where I lacked the subject-matter familiarity. I did this in response to a claim that §230 overruled anti-discrimination law; a claim I confidently rejected as patently ludicrous but one which ended up being correct.
See? That wasn't that bad was it? I am still alive. Please call me out on any errors I haven't acknowledged! I am so grateful towards the people that do this. There's my entire Substack archive and here's also a spreadsheet with all my Motte posts from Reddit if that's easier to search. Never hesitate from flogging that whip, wah-pah!
Regime-banned books are in school libraries and on indigo bookshelves at eye level for children.
REAL banned books are often decades out of print, going for hundreds of dollars used on eBay, they've been disappeared by publishers and distributors in spite of interest and demand. Others have authors who've died or been imprisoned for their ideas, yet more have been removed from city or university-wide library systems so that their "Misinformation" and "Lies" do not poison impressionable scholars.
Yet more are suppressed algorithmically, not appearing on the author's wikipedia page and not appearing in Google search if you type the author and "book" or "memoirs"... but only appearing when you already know the full title of the work (try this yourself: Type in "Pinochet Memoirs", and then type in "Pinochet: A journey through a life")
Yet others are explicitly banned, some to the point where a mere PDF on your hard drive can result in a decade-long sentence... IN THE UNITED KINGDOM, NEW ZEALAND, and AUSTRIA.
This has been a massive project. over 200 titles on the full list and 10,000 words in my "Cursory" survey.
Let me take you on a journey into the heart of the forbidden
UPDATE: Also Checkout My Addendum to The Real Banned Book list on Holocaust Revisionist Liturature
I live pretty close to the university I attended roughly a decade ago, and I’m very frequently on or near campus. Over the past couple of years, especially since we’ve had some nasty hot summers here in San Diego, it has become somewhat common for me to see young women walking on public sidewalks wearing skimpy bikinis, including occasionally thong swimsuit bottoms. Like, ass cheeks fully out for the world to see. When I see this, obviously my lizard brain is thoroughly captivated, but my higher-functioning brain is then immediately scandalized and appalled.
When I attended this same university, it was strange and tantalizing enough to see so many women walking around in sheer leggings and booty-shorts. This was not allowed at my high school, and I doubt that many of the girls would have availed themselves of the option even if it had been allowed. So, for me, being surrounded by women in (compared to what I was used to) revealing clothing made me feel frustrated and constantly distracted. It also, as I continued through college without having any romantic/sexual success with women during that time, began to make me feel desperate and invisible. Look at all of these hot people all around me! Am I the only person on campus who is not attractive? Does anyone even notice that I exist? Like a penniless man walking down a bustling commercial boulevard arrayed with shiny advertisements of wonderful products I couldn’t hope to purchase, I felt like having all of these unattainable women showing off their bodies to me but not giving me the time of day was infinitely worse than not having women around at all.
Still, it would have been unheard of at that time for one of those young women to walk around in public in broad daylight wearing nothing but a thong bikini. Regardless of any legal penalties or school policies regarding such an action, it would have been seen, by both women and men, as simply unacceptably slutty. I can imagine that such an act would have led dozens of captivated male passers-by to walk head-first into trees or crash their cars while rubbernecking, like when Sue-Ellen Mischke walked down an NYC sidewalk wearing a bra as a top. Now apparently this is normal behavior in 2023.
When I see one of these women, I’m struck by the thought, “The Taliban are right about women.” Now, this is not a rational and considered policy endorsement. It’s just an atavistic cri de couer of a man who does not want to have such a thing dangled in my face unexpectedly while trying to have a normal public outing. It honestly makes me a tiny bit sympathetic to the Middle Eastern and African guys who come into Europe and end up sexually assaulting local women because they misinterpreted the women’s loose and revealing manner of dress for an obvious and intentional public invitation to sexual contact. Where those men come from, no woman would dream of dressing like that, unless she were a particularly brazen prostitute. Having made it to adulthood without cultivating any coping mechanisms for dealing with the level of sexual frustration generated by being surrounded by countless beautiful unaccompanied women in revealing outfits, they lash out in a brutish act of desperate catharsis. Now obviously I do not actually condone the actions of these men, and I wish to see them punished unimaginably harshly for their depraved violations of European women; I also wish that immigration policies were such that these men were not in Europe in the first place to experience such a brutal culture shock.
Still, I can’t help but think that the Islamic world basically has the right idea in terms of their approach to strictly enforcing conservative female attire. I can quibble with the specifics - certainly a burqa is excessive, and I’m not sure that things like niqabs and hijabs are really necessary. But, of course, that’s my western culturally-liberal background talking; I’ve been born centuries after the multiple turns of the ratchet which normalized women walking about with exposed hair and legs and arms, so it seems normal to me, and with the way things are going it looks like in a few more decades the ratchet will have turned here in America such that people will be seen as wildly prudish for thinking it off to see women with their entire asses out on the sidewalk. Hell, perhaps by 2050 American women will be strutting around like the women of the early Bronze Age Minoan civilization, -titties out for the world to see, if their vases are accurate - and the prudes of that era will be asking why we can’t just go back to when women were classy and didn’t wear anything more revealing than a bikini.
Speaking of the Minoans, they are one of the few ancient civilizations for which we have any concrete persuasive evidence that a matriarchal order may have prevailed for a substantial length of time. In Neolithic European civilizations, prior to the Indo-European (Aryan) conquests, a harsh sexual order appears to have prevailed in which the vast majority of men did not reproduce, and may have simply been worked to death in salt mines or massive farm complexes while the women could spend their time advertising their beauty and sexual competitiveness to a small elite of men. I’m far from the first commentator to notice that our societies appear to be lurching in a similar direction; the woman strutting around my local sidewalk in a thong, with no fear of repercussions nor even social censure, content that any frustration or angst she generates in nearby males is highly unlikely to redound negatively toward her, strikes me as symptomatic of this development.
In such a sociosexual regime, assuming we don’t have any massive salt mines for all of our sexually-unsuccessful beta males to expire in, it seems that it may be high time to reintegrate into our society a male archetype which has decidedly fallen by the wayside over the past few centuries: that of the monk or ascetic. While rightwing Twitter (uh, sorry, “X”) embraces the total hegemony of the conquering warrior archetype, it remains the case that there are hundreds of millions of men like me who are never going to ride a chariot into battle or build a homestead from the ground up. For guys like us, maybe it’s time to look toward the monastic lifestyle as an alternative option.
I recently spent a week visiting the U.K. I spent a substantial amount of my time there visiting cathedrals and abbeys. While all of them were breathtaking, I found myself particularly captivated - haunted, really - by Tintern Abbey. Walking within the shattered exoskeleton of a once-thriving monastery is a truly unique experience. Reading more about the Cistercian brotherhood of Monks who founded and operated Tintern for centuries, much about their lifestyle sounded quite appealing to me. To live apart from the world of carnality and temptation, sequestered away with your geeky and serious-minded brothers, translating old Greek and Latin texts, tending a garden, eating simple meals and enjoying simple but meaningful pursuits while the outside world roils and burns around you… what’s not to like? I can imagine how I would fit in with the other monks; I think I’d have a solid chance of being the best singer in the Gregorian chant choir, and I bet I’d be appreciated for giving the most spirited reading of Bible passages during dinner of any of the monks there. I wouldn’t conquer any lands or hear the lamentation of my enemies’ women - ideally I wouldn’t encounter women at all - but maybe I’d end up being the primary author of some groundbreaking historical compendium that would still be useful to people a millennium in the future.
Of course, no such life is really available for western men in our age. Sure, Buddhist monks still exist, as do the Hare Krishnas and other assorted oddball ascetic cults, but they remain the sole province of foreigners, and only the oddest of western oddballs would join one of them. Far more importantly, I have already tasted the fruits of modernity. I have been with women. I know what it’s like to have an infinite universe of porn and other superstimuli at my fingertips. Giving that all up to go withdraw into the monastic life would be impossibly difficult and depriving, because I would know what I’m missing. Sure, it would be a blessed release from the sexual rat race, in which I have fallen far behind, but I would never be able to escape the nagging feeling that I could have done better for myself. The only way to make the monk life work is to identify, early in life, the boys who would be best served by that life path, and plucking them away from the temptations of the world before they’ve developed any strong taste for them. For those of us who’ve already been exposed to modernity, the genie is out of the bottle and he’s not going back in.
If the monastic option is going to make a real return in our culture, it will have to be undergirded by a genuine status infrastructure undergirding it. Such men must not be seen as losers and washouts, crawling in shame away from a life of failure and grasping tightly a pathetic consolation prize. It must be seen as a noble and important life path, every bit as valid as the warrior’s role, and genuinely rewarding in and of itself rather than simply an escape from suffering. It seems like for shape rotators, the life of the shut-in programmer, the “digital nomad”, or the mad scientist are still viable life paths that offer real status and material rewards, but for male wordcels who wish to check out of the lottery lifestyle of academia or entertainment, the pickings seem significantly slimmer. What is the modern wordcel monk to do? AI seems to be rapidly devouring what few paths had remained, leaving beta wordcels no path forward but to cope and seethe, dreaming of living a simple but failure-proof life in an abbey which now lies in ruins.
To offset the more serious post, I thought I'd throw in something a bit more narrative and fun. These stories are true, but composite. It didn't all happen the same day, not with all the people mentioned, and not in the same units and places. That said:
It's midnight on a Tuesday. I'm pleasantly drunk in a VFW, arguing politics with Doc and Malo. They're the only people in the unit who would rather do that than chase girls and fight the locals. Doc short, confident, prematurely bald and ugly as a gnome. Hilarious dude, a Napoleon of dick jokes. Malo black and huge, his big dumb face concealing a keen intellect and a first rate education. He speaks the Queen's English with a slight african accent and an aristocratic tone. Me rail-skinny, argumentative and mean. We're a strange looking group, but our military IDs buy us a quiet corner where we close it down, then start collecting our other guys. Pasco and Healy have struck out at the bars, and Po is hiding behind a gas station near the highway. He got in a fight and the cops are looking for him. We pick him up on our way out of Victorville. It's almost two hours back to base, and morning formation is at 0430. Our team leaders will kill us if we're not there by 0415, so we have barely enough time to get back.
I get about an hour of sleep on the ride. Malo drives, he's a bad muslim but doesn't drink. The gate guard wakes me up with his flashlight, checking the long line of cars snaking into the base in the dark early morning. We awkwardly change into our PT uniforms, lots of butts in faces if you want to imagine six drunk dudes getting changed in a moving Chevy Tahoe. Lots of commentary and propositioning. Ahh, the infantry. Gayest pack of straight dudes in the world.
Parked, we stagger out into the company area where the NCOs are already forming us up. We jog to our spot in the formation, our little group splitting. Doc to Headquarters, Malo to first squad, Pasco and Healy to Second and Po and I to Third. New guy, B team, Third squad, Third platoon (Dirty Third!). I'm at the end of the line, farthest from the NCOs. Po is one slot from the squad leader. I look down the line, and I see we weren't the only ones out on the town. The whole squad is swaying forward and back, everyone trying hard not to look as hammered as they are. The First Sergeant is walking out, but I can hear Sgt. Mac's vicious hiss: “I can smell you motherfuckers”.
We're all technically drunk on duty, a court martial offense. We're relying on the informal rule, which is that if we can gut out the morning PT, they won't breathalyze us. There's a thought to sharpen your mind on a cold desert morning. Fall out, and you'll be in the brig by lunch. And first Sar is looking pissed. He's not going to make it easy for us. Yup, he's leading PT personally, and we're running. Fuck my life.
At least I'm a strong runner. No weight to slow me down. I can run this company into the ground drunk or sober. If I'm in the brig, I won't be alone. I wonder if that would warm the guys up to me?
Three miles outside of base, I have a new problem. Somehow, I've hit the perfect level of intoxication, and am feeling no pain on this run. I do need to shit though, and that greasy bar food is not going to wait for this run to end. I spot my chance as we pass over a large culvert where the road crosses a wadi. I dive off the side, slide down the embankment. I can hear my team leader's startled yell behind me, but I'm already down to the culvert. It's dark, I crawl in and paint the walls brown.
No TP, no vegetation, and with this run, I'll probably need my socks. I rip the bottom off my PT shirt to wipe myself with, leaving me with a very fetching crop-top. The unit is hundreds of yards ahead now. I start making up the time, closing the distance. I come upon Healy who has fallen out and is struggling. He looks at me in surprise, “I thought you fell out?”. I grin as I come along side him, pointing to my midriff. “Needed a shit”. He laughs and quickens his pace a bit. We catch up the unit just before it's light enough for people to see us. Hernandez, my TL, is relieved, but I'll still probably catch some smoke later. There's much giggling, whistling and shit talk about my new fashion statement. For now, we're five miles from base, and we still have to run back. Six miles. Seven. Jesus, is first Sar gonna run us twenty miles? At seven and a half we turn back. Fifteen miles and I am flying. A third of the unit drops out, but they're all the family guys who weren't at the bars last night. The rest of us have to finish.
We form back up on the company area. First sergeant stares us down for a long minute before he releases us. I don't know if that's good or bad. From there, it's a sprint back to the barracks, quick shower, uniform of the day, then off to the chow hall, packed this time of the morning. I wolf down two quite good omelets, four very bad biscuits, and something they passed off as sausage gravy. Got fifty bucks says no pigs were harmed in the making of that product.
Back to formation, where we are sent to “PMCS the vehicles”, standard infantry code for “we don't have shit for you to do today, so check the oil on the lieutenant's vehicle for the eighth time this month”. But there are benefits. A pair of boots sticking out from under a vehicle won't raise suspicion, so I get a nice nap on the cool concrete as the outside temperature spikes. Supposed to hit a hundred and ten today, but it's barely hundred in the shade! Fuck Fort Irwin.
Hernandez kicks me awake, and gives me a perfunctory smoke session for dropping out of formation and damaging government property (my PT shirt). His heart's not in it, he doesn't care, but he has to be able to tell Mac that he did it so Mac can tell his boss, and so I do ten minutes of pushups. If that's all the worse I come out of this, it will be a fucking miracle. “Good job on the run” he says “go to lunch”. He can't come down on me too hard for drinking, I buy him beer. He's still only twenty.
For lunch I go back to my room and catch another nap. My roommate isn't there, but I can see the ants crawling under his bedroom door. Filthy fucker. How does he keep getting away with it? I get my room inspected weekly, Lacava has never been inspected. And he's supposedly in a strict company. I've put in for a different barracks room three times already, but they say nothing is available. This motherfucker has pizza stuck to the ceiling, rotting food and dirty clothes everywhere, his room stinks. You can smell it from the sidewalk. I should probably just kick his ass until he puts in for a transfer, but he's the only guy around smaller than me. Not a good look. And I'm new, no leverage, no real friends, no idea of the lay of the land. Might have to ruminate on that one.
Back to the motor pool. We're a light unit, so our platoon only has one vehicle. The PMCS took fifteen minutes back in the morning, but if we go back to the company, they'll find something even dumber for us to do. There's fifteen guys laying around, playing spades, napping, bullshitting. It's a great afternoon. The glories of the previous night are dissected in detail, lies are told, bullshit is called. I come in for a good bit of comedy about my shit-shirt that morning. I play it off, call them fags for getting all excited at a man's belly button. My timing is off, the jokes are flat. But soon enough it's someone else, Pasco getting grilled about that totally hot chick who supposedly blew him in the bathroom that no one else can remember seeing.
1600 the other units are leaving the motor pool, but Hernandez keeps us there. Our formation will be late today, he says, and he wants to see if we can steal some gear from other units' trucks. We find an unlocked victor two lines behind ours from some other unit and strip it bare. Seat cushions, pioneer gear, a donkey dick (long funnel) etc. We'll outfit our PL's vehicle and trade the surplus to other trucks, maybe even the guys we stole it all from. There's only one thief in the Army, the rest of us are just getting our shit back the saying goes....
Formation finally happens around seven, after several more hours of sitting around not doing shit. Infantry in garrison rarely have much to do. Our general work cycle is two weeks in the field doing training, then two short weeks in garrison punctuated by a three-day and a four-day weekend. Then back out to the field, ranges, gunnery etc. Once we get out of the field and get all our gear cleaned and turned in, it's bullshit details and “PMCS” to try to keep all those joes busy. For our part, we learn to stretch every job out as long as possible, because they're only going to find something else when you're done. The stories about hand-painting the gravel are true. When everything conceivable has been cleaned, mopped, polished and checked it's time to get creative.
The infantryman considers garrison to be basically one big weekend, which is why we've been drinking every night. This is not our job, this is the bullshit period before our job. There's nothing important to do, so we try to stay out of the way and enjoy ourselves as much as possible. We'll be sleeping in the dirt again soon enough. There will be no bars, no girls, not even the indifferent but at least hot chow hall slop. I'm gonna miss those omelets.
I get pulled aside at formation, everyone else goes home. Hernandez, Mac and I have to report to the First Sergeant. Just for scale, I'm a boot PFC, Hernandez is my team leader, my daily boss. His boss is Mac, the squad leader. Mac is the Hand of God, and the First Sergeant is his bosses' boss. Literally the Almighty Himself. I do not want to be here, this will not be good. I'm worried about jail, but it turns out someone has finally complained about my roommate. Which means they complained about me. Mac and Hernandez get interrogated about how often they inspect my room. Because Lacava isn't their soldier, they can't touch him, so they say nothing about that part. They do hedge a bit because I've built a recording studio in my closet. I checked the regs and it's technically legal, but first Sar needs a reason to be angry and that's it. He's getting complaints that his soldiers have dirty rooms, so someone is gonna suffer and that someone is apparently me.
The three of them march me over to the barracks and wait outside while I pull every scrap out of my room and lay it out on the sidewalk. Bed, cabinet, clothes, everything. It takes hours. I have to disassemble the bed and cabinet, reassemble them outside, clean the room. The boys are headed back out, they're clowning me, asking if I want to pop down to San Bernardino for a quick one. I give them both fingers and keep working. The cleaning is easy because my room was already clean. My only consolation is Lacava doing the same thing a few yards away. His NCOs ain't saying shit. Now that we're all here, it's pretty obvious who has the problem, and who is just a weirdo. I re-disassemble the furniture, and move back into my own barracks room. It's midnight by the time I finish, twenty-four hours in the life of an infantryman.
Mac and Hernandez have had to sit there and watch me all night. They didn't get to go home, have dinner, see their families. They'll be pissed about that, but hopefully they understand.
In the morning, I'm going to kick Lacava's ass.
I think that UN manipulating it's own index is not culture wars even if the index is related to gender. Let me know if I am wrong.
Human development
The Gender Development Index (GDI), along with its more famous sibling Human Development Index (HDI) is a an index published annually by UN's agency, the United Nations Development Programme (UNDP). Whether an index is manipulated or not can be judged only against a precise definition of what the index claims to be measuring. So how do you measure human development? Whatever you do, you will never capture all nuances of the real world - you will have to simplify. The UNDP puts it this way:
The Human Development Index (HDI) was created to emphasize that people and their capabilities should be the ultimate criteria for assessing the development of a country, not economic growth alone.
So the UNDP defines the Human Development Index as a geometric mean of three dimensions represented by four indices:
| Dimension | Index |
|---|---|
| Long and healthy life | Life expectancy at birth (years) |
| Knowledge | Expected years of schooling (years) |
| Mean years of schooling (years) | |
| Decent standard of living | Gross National Income (GNI) per capita (2017 PPP$) |
Source: https://hdr.undp.org/data-center/human-development-index#/indicies/HDI
Gender Development
So far so good. Next, on it's website the Gender Development Index (GDI) is defined like this:
GDI measures gender inequalities in achievement in three basic dimensions of human development: health, measured by female and male life expectancy at birth; education, measured by female and male expected years of schooling for children and female and male mean years of schooling for adults ages 25 years and older; and command over economic resources, measured by female and male estimated earned income.
Source: https://hdr.undp.org/gender-development-index#/indicies/GDI
While in the actual report HDI it is simply defined as a ratio of female to male HDI values:
Definitions - Gender Development Index: Ratio of female to male HDI values.
Source: https://hdr.undp.org/system/files/documents/global-report-document/hdr2021-22pdf_1.pdf
Let's look, for instance, at the Gender Development Index of United Kingdom. The value 0.987 means that despite longer life and more education, in UK, females are less developed than males.
| Dimension | Index | Female value | Male value |
|---|---|---|---|
| Long and healthy life | Life expectancy at birth (years) | 82.2 | 78.7 |
| Knowledge | Expected years of schooling (years) | 17.8 | 16.8 |
| Mean years of schooling (years) | 13.4 | 13.4 | |
| Decent standard of living | Gross National Income (GNI) per capita (2017 PPP$) | 37,374 | 53,265 |
Source: https://hdr.undp.org/system/files/documents/global-report-document/hdr2021-22pdf_1.pdf
Wait, what?? What does it mean that females in UK have command over economic resources of post Soviet Estonia (GNI Estonia=38,048) while males in UK have command over economic resources of EU leader Germany (GNI Germany=54,534)?
The manipulation
The UNDP calculates separate command over economic resources for females and males, as a product of the actual Gross National Income (GNI) and two indices: female and male shares of the economically active population (the non-adjusted employment gap) and the ratio of the female to male wage in all sectors (the non-adjusted wage gap).
The UNDP provides this simple example about Mauritania:
Gross National Income per capita of Mauritania (2017 PPP $) = 5,075
| Indicator | Female value | Male value |
|---|---|---|
| Wage ratio (female/male) | 0.8 | 0.8 |
| Share of economically active population | 0.307 | 0.693 |
| Share of population | 0.51016 | 0.48984 |
| Gross national income per capita (2017 PPP $) | 2,604 | 7,650 |
According to this index, males in Mauritania enjoy the command over economic resources of Viet Nam (GNI Viet Nam=7,867) while females in Mauritania suffer the command over economic resources of Haiti (GNI Haiti=2,847).
Let's be honest here: this is total bullshit. There are two reasons why you cannot use raw employment gap and raw wage gap for calculating the command over economic resources:
Argument 1
Bread winners share income with their families. This is a no brainer. All over the world, men are expected to fulfil their gender role as a bread winer. This does not mean that they keep the pay check for themselves while their wives and children starve to death. Imagine this scenario: a poor father from India travels to Qatar where he labours in deadly conditions, so that his family can live a slightly better life. According to UNDP, he just became more developed, while the standard of living his wife is exactly zero.
Argument 2
Governments redistribute wealth. This is a no brainer too. One's command over economic resources and standard of living is not equal to ones pay check. There are social programs, pensions, public infrastructure. Even if you have never earned a pay check yourself, you can take a public transport on a public road to the next public hospital. Judging by the Tax Freedom Day, states around the world redistribute 30% to 50% of all income. And while men pay most of the taxis (obviously, they have higher wages) women receive most of the subsidies (obviously, they have lover wages). But according the UNDP, women in India (female GNI 2,277) suffer in schools and hospitals of the war-torn Rwanda, while men in India (male GNI 10,633) enjoy the infrastructure and social security of the 5-times more prosperous Turkey.
Don't get me wrong, the employment gap and pay gap are not irrelevant for the standard of living and command over economic resources. Pensions and social security schemes mostly do not respect the shared family income and as a result the partner doing less paid work - usually a women - gets lower pension, unemployment benefit etc. What's worse, the non-working partner is severely disadvantaged in case of divorce or break up. But while this has an impact on each gender's standard of living it certainly does not define 100% of that value.
Argument 3
You may argue that the command over economic resources measured by estimated earned income is some kind of proxy for all other disadvantages women face in society. But do you remember what I said in the beginning?
Whether an index is manipulated or not can be judged only against a precise definition of what the index claims to be measuring.
The HDI measures "people and their capabilities" and the GDI is a ratio of these capabilities measured separately for men and women. The economic dimension of the GDI is supposed to be standard of living or command over economic resources - neither of which can be represented by earned income alone.
The taboo
Wikipedia says: "For most countries, the earned-income gap accounts for more than 90% of the gender penalty." (I have not verified this.) This is important, because when we look at the other two dimensions it becomes clear that while men have shorter and less health lives they also increasingly fall behind in mean and expected years of schooling. Without the misrepresentation of the command over economic resources value, the index would show something very uncomfortable: that according to UN's own definition of Human Development men are the less developed gender.
PS: Is there a way to give those tables some borders and padding?
I've been wrong, again, pooh-poohing another Eurasian autocracy. Or so it seems.
On 29 August 2023, to great jubilation of Chinese netizens («the light boat has passed through a thousand mountains!», they cry), Huawei has announced Mate 60 and 60 Pro; the formal launch is scheduled for September 25th, commemorating the second anniversary of return of Meng Wanzhou, CFO and daughter of Huawei's founder, from her detainment in Canada. Those are nice phones of course but, specs-wise, unimpressive, as far as flagships in late 2023 go (on benchmarks, score like 50-60% of the latest iPhone while burning peak 13W so 200% of power). Now they're joined by Mate X5.
The point, however, is that they utilize Huawei's own SoC, Hisilicon Kirin 9000S, not only designed but produced in the Mainland; it even uses custom cores that inherit simultaneous multithreading from their server line (I recommend this excellent video review, also this benchmarking). Their provenance is not advertised, in fact it's not admitted at all, but now all reasonable people are in agreement that it's SMIC-Shanghai made, using their N+2 (7nm) process, with actual minimum metal pitch around 42 nm, energy efficiency at low frequencies close to Samsung's 4nm and far worse at high (overall capability in the Snapdragon 888 range, so 2020), transistor density on par with first-gen TSMC N7, maybe N7P (I'm not sure though, might well be 10% higher)… so on the border of what has been achieved with DUV (deep ultraviolet) and early EUV runs (EUV technology having been denied to China. As a side note, Huawei is also accused of building its own secret fabs).
It's also worse on net than Kirin 9000, their all-time peak achievement taped out across the strait in 2020, but it's… competitive. They apparently use self-aligned quad patterning, a DUV variant that's as finicky as it sounds, an absurd attempt to cheat optics and etch features many times smaller than the etching photons' wavelength (certain madmen went as high as 6x patterning; that said, even basic single-patterning EUV is insane and finicky, «physics experiment, not a production process»; companies on the level of Nikon exited the market in exasperation rather than pursue it; and it'll get worse). This trick was pioneered by Intel (which has failed at adopting EUV, afaik it's a fascinating corporate mismanagement story with as much strategic error as simple asshole behavior of individual executives) and is still responsible for their latest chips, though will be made obsolete in the next generations (the current node used to be called Intel's 10 nm Enhanced SuperFin, and was recently rebranded to Intel 7; note, however, that Kirin 9000S is a low-power part and requirements there are a bit more lax than in desktop/server processors). Long story short: it's 1.5-2 generations, 3-4 years behind the frontier of available devices, 5-6 years behind frontier production runs, 7-8 years after the first machines to make such chips at scale came onto market; but things weren't that much worse back then. We are, after all, in the domain of diminishing returns.
Here are the highlights from the first serious investigation, here are some leaks from it, here's the nice Asianometry overview (esp 3:50+), and the exhilarating, if breathlessly hawkish perspective of Dylan Patel, complete with detailed restrictions-tightening advice. Summarizing:
- This is possible because sanctions against China have tons of loopholes, and because ASML and other suppliers are not interested in sacrificing their business to American ambition. *
- Yes, it qualifies for 7nm in terms of critical dimensions. Yes, it's not Potemkin tulou, they likely have passable yields, both catastrophic and parametric (maybe upwards of 50% for this SoC, because low variance in stress-testing means they didn't feel the need to approve barely-functional chips, meaning there weren't too many defects) and so it's economically sustainable (might be better in that sense than e.g. Samsung's "5nm" or "4nm", because Samsung rots alive due to systemic management fraud) [I admit I doubt this point, and Dylan is known to be a hawk with motivated reasoning]. Based on known capex, they will soon be able to produce 30K wafers per month, which means 10s of millions of such chips soon (corroborated by shipment targets; concretely it's like 300 Kirins *29700 wafers so 8.9M/month, but the cycle is>1 month). And yes, they will scale it up further, and indeed they will keep polishing this tech tree and plausibly get to commercially viable "5nm" next - «the total process cost would only be ≈20% higher versus a 5nm that utilizes EUV» (probably 50%+ though).
- But more importantly: «Even with 50% yields, 30,000 WPM could support over 10 million Nvidia H100 GPU ASIC dies a year […] Remember GPT-4 was trained on ≈24,000 A100’s and Open AI will still have less than 1 million advanced GPUs even by the end of next year». Of course, Huawei already had been producing competitive DL accelerators back when they had access to EUV 7nm; even now I stumble upon ML papers that mention using those.
- As if all that were not enough, China simply keeps splurging billions on pretty good ML-optimized hardware, like Nvidia A/H800s, which abide with the current (toothless, as Patel argues) restrictions.
- But once again: on a bright (for Westerners) side, this means it's not so much Chinese ingenuity and industriousness (for example, they still haven't delivered a single ≤28nm lithography machine, though it's not clear if the one they're working on won't be rapidly upgraded for 20, 14, 10 and ultimately 7nm processes – after all, SMIC is currently procuring tools for «28nm», complying with sanctions, yet here we are), as it's the unpicked low-hanging fruit of trade restrictions. In fact, some Chinese doomers argue it's a specific allowance by the US Department of Commerce and overall a nothingburger, ie doesn't suggest willingness to produce more consequential things than gadgets for patriotic consumers. The usual suspects (Zeihan and his flock) take another view and smugly claim that China has once again shot itself in the foot while showing off, paper tiger, wolf warriors, only steals and copies etc.; and, the stated objective of the USG being «as large of a lead as possible», new crippling sanctions are inevitable (maybe from Patel's list). There exists a body of scholarship on semiconductor supply chain chokepoints which confirms these folks are not delusional – something as «simple» as high-end photoresist is currently beyond Chinese grasp, so the US can make use of a hefty stick.
All that being said, China does advance in on-shoring the supply chain: EDA, 28nm scanners, wafers etc.
* Note: Patel plays fast and loose with how many lithography machines exactly, and of what capacity, are delivered/serviced/ordered/shipping/planned/allowed, and it's the murkiest part in the whole narrative; for example he describes ASML's race-traitorous plans stretching to 2025-2030, but the Dutch and also the Japanese seem to already have began limiting sales of tools he lists as unwisely left unbanned, and so the August surge or imports may have been the last, and certainly most 2024+ sales are off the table I think.
All of this is a retreading of a discussion from over a year ago, when a less mature version of SMIC N7 process was used - also surreptitiously – for a Bitcoin mining ASIC, a simple, obscenely high-margin part 19.3mm² in size, which presumably would have been profitable to make even at pathetic yields, like 10%; the process back then was near-idential to TSMC N7 circa 2018-2019. 9000S is 107 mm² and lower-margin. Nvidia GH100, the new workhorse of cutting edge ML, made with 4nm TSMC node, is 814 mm²; as GPU chips are a strategic resource, it'd be sensible to subsidize their production (as it happens, H100 with its 98 MTr/mm² must be equally or a bit less dense than 9000S; A100, a perfectly adequate 7nm downgrade option, is at 65 MTr/mm² so we can be sure they'll be capable of making those, eg resurrecting Biren BR100 GPUs or things like Ascend 910). Citing Patel again, «Just like Apple is the guinea pig for TSMC process nodes and helps them ramp and achieve high yield, Huawei will likewise help SMIC in the same way […] In two years, SMIC will likely be able to produce large monolithic dies for AI and networking applications.» (In an aside, Patel laments the relative lack of gusto in strangling Chinese radio/sensor capabilities, which are more formidable and immediately scary than all that compute. However, this makes sense if we look at the ongoing chip trade war through the historical lens, with the reasonable objective being Chinese obsolescence a la what happened to the Soviet Union and its microelectronics, and arguably even Japan in the 80s, which is why ASML/Samsung/TSMC are on the map at all; Choyna military threat per se, except to Taiwan, being a distant second thought, if not a total pretext. This r/LessCredibleDefense discussion may be of interest).
So. I have also pooh-poohed the Chinese result back then, assuming that tiny crypto ASICs are as good as they will get within the bounds assigned to them, «swan song of Chinese industry», and won't achieve meaningful yields. Just as gwern de facto did in October 2022, predicting the slow death of Chinese industry in view of «Export Controls on Advanced Computing and Semiconductor Manufacturing Items to the PRC» (even mentioning the yellow bear meme). Just as I did again 4 months ago, saying to @RandomRanger «China will maybe have 7nm in 2030 or something». I maintain that it's plausible they won't have a fully indigenized supply chain for any 7nm process until 2030 (and/or will likewise fail with securing chains for necessary components other than processors: HBM, interposers etc), they may well fall below the capacity they have right now (reminder that not only do scanners break down and need consumables, but they can be remotely disabled), especially if restrictions keep ramping up and they'll keep making stupid errors, e.g. actually starting and failing an attempt at annexing Taiwan, or going for Cultural Revolution Round II: Zero Covid Boogaloo, or provoking an insurgency by force-feeding all primary school students gutter oil breakfasts… with absolute power, the possibilities are endless! My dissmissal was informed not by prejudice but years upon years of promises by Chinese industry and academia representatives to get to 7nm in 2 more weeks, and consistent failure and high-profile fraud (and in fact I found persuasive this dude's argument that by some non-absurd measures the gap has widened since the Mao's era; and there was all the graphene/quantum computing "leapfrogging" nonsense, and so on). Their actors haven't become appreciably better now.
But I won't pooh-pooh any more, because their chips have become better. I also have said: «AGI can be completed with already available hardware, and the US-led bloc has like 95% of it, and total control over means of production». This is still technically true but apparently not in a decisive way. History is still likely to repeat – that is, like the Qing China during the Industrial Revolution, like the Soviet Union in the transistor era, the nation playing catch-up will once again run into trade restrictions, fail at the domestic fundamental innovation and miss out on the new technological stage; but it is not set in stone. Hell, they may even get to EUV through that asinine 160m synchrotron-based electron beam thing – I mean, they are trying, though it still looks like ever more academic grift… but…
I have underestimated China and overestimated the West. Mea culpa. Alphanumericsprawl and others were making good points.
Where does this leave us?
It leaves us in the uncomfortable situation where China as a rival superpower will plausibly have to be defeated for real, rather then just sanctioned away or allowed to bog itself down in imperialist adventurism and incompetence. They'll have enough suitable chips, they have passable software, enough talent for 1-3 frontier companies, reams of data and their characteristically awkward ruthlessness applied to refining it (and as we've learned recently, high-quality data can compensate for a great disparity in compute). They are already running a few serious almost-OpenAI-level projects – Baidu's ERNIE, Alibaba's Tongyi Qianwen (maybe I've mentioned it already, but their Qwen-7B/VL are really good; seems like all groups in the race were obligated to release a small model for testing purposes), maybe also Tsinghua's ChatGLM, SenseTime etc.'s InternLM and smaller ones. They – well, those groups, not the red boomer Xi – are well aware of their weaknesses and optimize around them (and borrowing from the open academic culture helps, as can be often seen in the training methods section – thanks to MIT&Meta, Microsoft, Princeton et al). They are preparing for the era of machine labor, which for now is sold as means to take care of the aging population and so on (I particularly like the Fourier Intelligence's trajectory, a near-perfect inversion of Iron Man's plot – start with the medical exoskeleton, proceed to make a full humanoid; but there are other humanoids developed in parallel, eg Unitree H1, and they seem competitive with their American equivalents like Tesla Optimus, X1 Neo and so on); in general, they are not being maximally stupid with their chances.
And this, in turn, means that the culture of the next years will be – as I've predicted in Viewpoint Focus 3 years ago – likely dominated by the standoff, leading up to much more bitter economic decoupling and kinetic war; promoting bipartisan jingoism and leaving less space for «culture war» as understood here; on the upside, it'll diminish the salience of progressive campaigns that demoralize the more traditionally minded population.
It'll also presumably mean less focus on «regulation of AI risks» than some would hope for, denying this topic the uncontested succession to the Current Thing №1.
That's about all from me, thoughts?
As I've been arguing for some time, the culture war's most important front will be about AI; that's more pleasant to me than the tacky trans vs trads content, as it returns us to the level of philosophy and positive actionable visions rather than peculiarly American signaling ick-changes, but the stakes are correspondingly higher… Anyway, Forbes has doxxed the founder of «e/acc», irreverent Twitter meme movement opposing attempts at regulation of AI development which are spearheaded by EA. Turns out he's a pretty cool guy eh.
Who Is @BasedBeffJezos, The Leader Of The Tech Elite’s ‘E/Acc’ Movement?
…At first blush, e/acc sounds a lot like Facebook’s old motto: “move fast and break things.” But Jezos also embraces more extreme ideas, borrowing concepts from “accelerationism,” which argues we should hasten the growth of technology and capitalism at the expense of nearly anything else. On X, the platform formally known as Twitter where he has 50,000 followers, Jezos has claimed that “institutions have decayed beyond the point of salvaging and that the media is a “vector for cybernetic control of culture.”
Alarmed by this extremist messaging, «the media» proceeds to… harness the power of an institution associated with the Department of Justice to deanonymize him, with the explicit aim to steer the cultural evolution around the topic:
Forbes has learned that the Jezos persona is run by a former Google quantum computing engineer named Guillaume Verdon who founded a stealth AI hardware startup Extropic in 2022. Forbes first identified Verdon as Jezos by matching details that Jezos revealed about himself to publicly available facts about Verdon. A voice analysis conducted by Catalin Grigoras, Director of the National Center for Media Forensics, compared audio recordings of Jezos and talks given by Verdon and found that it was 2,954,870 times more likely that the speaker in one recording of Jezos was Verdon than that it was any other person. Forbes is revealing his identity because we believe it to be in the public interest as Jezos’s influence grows.
That's not bad because Journalists, as observed by @TracingWoodgrains, are inherently Good:
(Revealing the name behind an anonymous account of public note is not “doxxing,” which is an often-gendered form of online harassment that reveals private information — like an address or phone number — about a person without consent and with malicious intent.)
(That's one creative approach to encouraging gender transition, I guess).
Now to be fair, this is almost certainly parallel construction narrative – many people in the SV knew Beff's real persona, and as of late he's been very loose with opsec, funding a party, selling merch and so on. Also, the forced reveal will probably help him a great deal – it's harder to dismiss the guy as some LARPing shitposter or a corporate shill pandering to VCs (or as @Tomato said, running «an incredibly boring b2b productivity software startup») when you know he's, well, this. And this too.
Forbes article itself doesn't go very hard on Beff, presenting him as a somewhat pretentious supply-side YIMBY, an ally to Marc Andreessen, Garry Tan and such; which is more true of Beff's followers than the man himself. The more potentially damaging (to his ability to draw investment) parts are casually invoking the spirit of Nick Land and his spooky brand of accelerationism (not unwarranted – «e/acc has no particular allegiance to the biological substrate for intelligence and life, in contrast to transhumanism; in order to spread to the stars, the light of consciousness/intelligence will have to be transduced to non-biological substrates» Beff says in his manifesto), and citing some professors of «communications» and «critical theory» who are just not very impressed with the whole technocapital thing. At the same time, it reminds the reader of EA's greatest moment (no not the bed nets).
Online, Beff confirms being Verdon:
I started this account as a means to spread hope, optimism, and a will to build the future, and as an outlet to share my thoughts despite to the secretive nature of my work… Around the same time as founding e/acc, I founded @Extropic_AI. A deep tech startup where we are building the ultimate substrate for Generative AI in the physical world by harnessing thermodynamic physics. Ideas simmering while inventing a this paradigm of computing definitely influenced the initial e/acc writings. I very much look forward to sharing more about our vision for the technology we are building soon. In terms of my background, as you've now learned, my main identity is @GillVerd. I used to work on special projects at the intersection of physics and AI at Alphabet, X and Google. Before this, I was a theoretical physicist working on information theory and black hole physics. Currently working on our AI Manhattan project to bring fundamentally new computing to the world with an amazing team of physics and AI geniuses, including my former TensorFlow Quantum co-founder @trevormccrt1 as CTO. Grateful every day to get to build this technology I have been dreaming of for over 8 years now with an amazing team.
And Verdon confirms the belief in Beffian doctrine:
Civilization desperately needs novel cultural and computing paradigms for us to achieve grander scope & scale and a prosperous future. I strongly believe thermodynamic physics and AI hold many of the answers we seek. As such, 18 months ago, I set out to build such cultural and computational paradigms.
I am fairly pessimistic about Extropic for reasons that should be obvious enough to people who've been monitoring the situation with DL compute startups and bottlenecks, so it may be that Beff's cultural engineering will make a greater impact than Verdon's physical one. Ironic, for one so contemptuous of wordcels.
Maturation of e/acc from a meme to a real force, if it happens (and as feared on Alignment Forum, in the wake of OpenAI coup-countercoup debacle), will be part of a larger trend, where the quasi-Masonic NGO networks of AI safetyists embed themselves in legacy institutions to procure the power of law and privileged platforms, while the broader organic culture and industry develops increasingly potent contrarian antibodies to their centralizing drive. Shortly before the doxx, two other clusters in the AI debate have been announced.
First one I'd mention is d/acc, courtesy of Vitalik Buterin; it's the closest to acceptable compromise that I've seen. It does not have many adherents yet but I expect it to become formidable because Vitalik is.
Across the board, I see far too many plans to save the world that involve giving a small group of people extreme and opaque power and hoping that they use it wisely. And so I find myself drawn to a different philosophy, one that has detailed ideas for how to deal with risks, but which seeks to create and maintain a more democratic world and tries to avoid centralization as the go-to solution to our problems. This philosophy also goes quite a bit broader than AI, and I would argue that it applies well even in worlds where AI risk concerns turn out to be largely unfounded. I will refer to this philosophy by the name of d/acc.
The "d" here can stand for many things; particularly, defense, decentralization, democracy and differential. First, think of it about defense, and then we can see how this ties into the other interpretations.
[…] The default path forward suggested by many of those who worry about AI essentially leads to a minimal AI world government. Near-term versions of this include a proposal for a "multinational AGI consortium" ("MAGIC"). Such a consortium, if it gets established and succeeds at its goals of creating superintelligent AI, would have a natural path to becoming a de-facto minimal world government. Longer-term, there are ideas like the "pivotal act" theory: we create an AI that performs a single one-time act which rearranges the world into a game where from that point forward humans are still in charge, but where the game board is somehow more defense-favoring and more fit for human flourishing.
The main practical issue that I see with this so far is that people don't seem to actually trust any specific governance mechanism with the power to build such a thing. This fact becomes stark when you look at the results to my recent Twitter polls, asking if people would prefer to see AI monopolized by a single entity with a decade head-start, or AI delayed by a decade for everyone… The size of each poll is small, but the polls make up for it in the uniformity of their result across a wide diversity of sources and options. In nine out of nine cases, the majority of people would rather see highly advanced AI delayed by a decade outright than be monopolized by a single group, whether it's a corporation, government or multinational body. In seven out of nine cases, delay won by at least two to one. This seems like an important fact to understand for anyone pursuing AI regulation.
[…] my experience trying to ensure "polytheism" within the Ethereum ecosystem does make me worry that this is an inherently unstable equilibrium. In Ethereum, we have intentionally tried to ensure decentralization of many parts of the stack: ensuring that there's no single codebase that controls more than half of the proof of stake network, trying to counteract the dominance of large staking pools, improving geographic decentralization, and so on. Essentially, Ethereum is actually attempting to execute on the old libertarian dream of a market-based society that uses social pressure, rather than government, as the antitrust regulator. To some extent, this has worked: the Prysm client's dominance has dropped from above 70% to under 45%. But this is not some automatic market process: it's the result of human intention and coordinated action.
[…] if we want to extrapolate this idea of human-AI cooperation further, we get to more radical conclusions**. Unless we create a world government powerful enough to detect and stop every small group of people hacking on individual GPUs with laptops, someone is going to create a superintelligent AI eventually - one that can think a thousand times faster than we can - and no combination of humans using tools with their hands is going to be able to hold its own against that. And so we need to take this idea of human-computer cooperation much deeper and further. A first natural step is brain-computer interfaces.…
etc. I mostly agree with his points. By focusing on the denial of winner-takes-all dynamics, it becomes a natural big tent proposal and it's already having effect on the similarly big tent doomer coalition, pulling anxious transhumanists away from the less efficacious luddites and discredited AI deniers.
The second one is «AI optimism» represented chiefly by Nora Belrose from Eleuther and Qiuntin Pope (whose essays contra Yud 1 and contra appeal to evolution as an intuition pump 2 I've been citing and signal-boosting for next to a year now; he's pretty good on Twitter too). Belrose is in agreement with d/acc; and in principle, I think this one is not so much a faction or a movement as the endgame to the long arc of AI doomerism initiated by Eliezer Yudkowsky, the ultimate progenitor of this community, born of the crisis of faith in Yud's and Bostrom's first-principles conjectures and entire «rationality» in light of empirical evidence. Many have tried to attack the AI doom doctrine from the outside (eg George Hotz), but only those willing to engage in the exegesis of Lesswrongian scriptures can sway educated doomers. Other actors in, or close to this group:
- Matthew Barnett with his analysis of goalpost-movement by MIRI, such as on factory-running benchmark and value misspecification thesis, and strengths of optimistic paradigms like Drexler's CAIS model.
- Alex Turner, who had written, arguably, two strongest and most popular formal proofs of instrumental convergence to power-seeking in AI agents 1 2, but has since fallen from grace, regrets his work and thinks deceptive alignment with LLMs is pretty much impossible.
- 1a3orn who is mainly concerned about centralization of power and opportunistic exploitation of AI risk narratives (also recommended: on Hansonian position in the FOOM debate).
- Beren Millidge the former head of research at Conjecture, Connor Leahy's extreme doomer company which has pivoted fully to advocacy and is gaining pull among British elites; over the last 1-2 years he has concluded that almost all of the MIRI-style assumptions 1 2 and their policy implications are confused.
- John David Pressman who's just a good thinker and AI researcher at Stability.
- Various other renegades and doubters like Kaj Sotala dunking on the evolution appeal, Zach M. Davis debating Yud-like «Doomimir», Nostalgebraist, arguably many scientists with P(doom)≤25% (and if you press them, ≈0% for their own AGI research program) like Rohin Shah. To an extent, even Paul Christiano (although he is in favor of decelerating; there are speculations that it's mostly due to being married to EA).
Optimists claim:
The last decade has shown that AI is much easier to control than many had feared. Today’s brain-inspired neural networks inherit human common sense, and their behavior can be molded to our preferences with simple, powerful algorithms. It’s no longer a question of how to control AI at all, but rather who will control it.
As optimists, we believe that AI is a tool for human empowerment, and that most people are fundamentally good. We strive for a future in which AI is distributed broadly and equitably, where each person is empowered by AIs working for them, under their own control. To this end, we support the open-source AI community, and we oppose attempts to centralize AI research in the hands of a small number of corporations in the name of “safety.” Centralization is likely to increase economic inequality and harm civil liberties, while doing little to prevent determined wrongdoers. By developing AI in the open, we’ll be able to better understand the ways in which AI can be misused and develop effective defense mechanisms.
So in terms of a political compass:
- AI Luddites, reactionaries, job protectionists and woke ethics grifters who demand pause/stop/red tape/sinecures (bottom left)
- plus messianic Utopian EAs who wish for a moral singleton God, and state/intelligence actors making use of them (top left)
- vs. libertarian social-darwinist and posthumanist e/accs often aligned with American corporations and the MIC (top right?)
- and minarchist/communalist transhumanist d/accs who try to walk the tightrope of human empowerment (bottom right?)
(Not covered: Schmidhuber, Sutton& probably Carmack as radically «misaligned» AGI successor species builders, Suleyman the statist, LeCun the Panglossian, Bengio&Hinton the naive socialists, Hassabis the vague, Legg the prophet, Tegmark the hysterical, Marcus the pooh-pooher and many others).
This compass will be more important than the default one as time goes on. Where are you on it?
As an aside: I recommend two open LLMs above all others. One is OpenHermes 2.5-7B, the other is DeepSeek-67B (33b-coder is OK too). Try them. It's not OpenAI, but it's getting closer and you don't need to depend on Altman's or Larry Summers' good graces to use them. With a laptop, you can have AI – at times approaching human level – anywhere. This is irreversible.
I made this list not out of snark or spite, but because it has rained all day, rained, even, on me as I took my walk, which I cut short, and because I have been making notes for months, and now seemed as good a time as any.
I had a long intro for this, hearkening back to grad school and people using terms they thought others knew and probably others did know but maybe not, and argots, and random musings, but I'll spare you.
These are words or terms I've seen in my many months here that I didn't know, or did know but didn't put together with their meaning. I am linking specific posts to where they were used, though these I found by doing a hard-search and are not necessarily the posts where I first saw the word/term. Any mistakes or misrepresentations are my fault. I hope this is helpful to others among us who are sometimes as confused as I am. Probably many of you know all these and must imagine me very old to post this. So be it.
If you recognize yourself as the author, I am not intending to be snide, or criticize your post. You just got lucky.
Edit: Many of these are probably going to need to be updated and tweaked. Feel free to add comments.
Let's start with the biggie:
asabiyyah: a concept of social solidarity with an emphasis on unity, group consciousness, and a sense of shared purpose and social cohesion.
Bagdhad Bob: When war propaganda becomes so out of touch with reality it turns comedic and achieves the opposite of the desired effect. It is said such propaganda is "Baghdad Bobbed" exactly at the moment when this threshold is crossed. From Muhammad Saeed al-Sahhaf, Iraqi War Information Minister in 2003.
Baumol’s Cost Disease: From the late William Baumol, NYU’s Stern School of Business. used to explain why prices for the services offered by people-dependent professions with low productivity growth—such as (arguably) education, health care, and the arts—keep going up, even though the amount of goods and services each worker in those industries generates hasn’t necessarily done the same.
"This is Baumol's cost disease in a nutshell."
Note: @jeroboam helpfully linked a Wiki page in this instance.
Chesterton’s fence: rule of thumb that suggests that you should never destroy a fence, change a rule, or do away with a tradition until you understand why it's there in the first place
consent à outrance: From the French. Suggests an agreement or consent that is given fully and without reservation, sometimes to the point of being excessive or without consideration of the consequences.
CRT: Critical Race Theory.
DRM: (I saw it used as a verb but have no link because I can no longer find it). Digital Righs Management. Presumably restricting the ways in which content (music, whatever) can be used, copied, or distributed.
Dunbar-limited world: Reference to Robin Dunbar, biological anthropologist. The “Dunbar Number” is the upper limit on the number of social relationships a human can effectively manage. (I believe it is supposed to be 150.)
Einsatzgruppen: : From the German. Actually, a German word. These were “mobile killing units,” best known for their role in the murder of Jews in mass shooting operations during the Holocaust.
Euthyphro: : A “straight-thinker.” A combination of εὐθύς (euthys), which means straight or direct and φρονέω (phroneô) which means to think or to reason.
NB: Alternate definition here.
Frasurbane: portmanteau of the sitcom Frasier and urbane, is the wonderfully specific aesthetic of late '90s interiors of people who want to come across as sophisticated and worldly.
HBD: If you do not know what this means, that’s weird, because it is almost a theme here. Human Biodiversity. Some here swear by its truth, others do not swear by it but expect it’s real, others think it’s dubious. Too many instances to choose from.
lolcow : A person whose eccentric or foolish behaviour can be exploited to amuse onlookers.
idpol: abbreviation based on identity politics is politics based on a particular identity, such as race, nationality, religion, gender, sexual orientation, social background, caste, etc.
If-by-whiskey: If-by-whiskey is a type of argument that supports both sides of a topic by employing terminology that is selectively emotionally sensitive. Originates from a speech given by Mississippi state representative Noah S. "Soggy" Sweat, Jr. in 1952.
Kolmogorov Complicity: Originated with Scott Alexander from his blog. Reference to the Soviet mathematician Andrey Kolmogorov. The idea of navigating or conforming to oppressive orthodoxies while still trying to contribute to the growth of knowledge and truth discreetly.
libfem: Stands for liberal feminist, also known as intersectional feminism or third wave feminism. A societal ideology focused on power dynamics and microlabels. a main branch of feminism defined by its focus on achieving gender equality through political and legal reform within the framework of liberal democracy and informed by a human rights perspective.
“The far end of libfem, maybe.”
MGTOW: an acronym for Men Going Their Own Way, an online social movement and backlash to feminism where men renounce interactions with women and seek to define and live out their masculinity on their own terms.
Orbanization: : (maybe) the process of adopting political strategies and governance methods that are similar to those of Viktor Orbán, the Prime Minister of Hungary. Orbán's tenure has been characterized by a centralization of power, control over media, erosion of checks and balances within government structures, and a move towards what is sometimes called "illiberal democracy."
Overton window: an approach to identifying the ideas that define the spectrum of acceptability of governmental policies. It says politicians can act only within the acceptable range. Shifting the Overton window involves proponents of policies outside the window persuading the public to expand the window.
“Transgender politics wasn’t in the Overton window at this point.”
Pascal’s Wager: the argument that it is in one's rational self-interest to act as if God exists, since the infinite punishments of hell, provided they have a positive probability, however small, outweigh any countervailing advantage.
"Pascal's wager is terrible because infinite rewards break game theory."
Pill colors: Red, Blue, Black
- Red Pill: In the context of online communities, particularly those focused on gender and relationships, "Red Pill" refers to the belief that men have been socially disadvantaged and that conventional beliefs about gender, attraction, and social interaction are misleading or false. It often involves the idea that men need to become aware of and confront these supposed harsh realities to improve their own lives. The term is frequently used in men's rights and certain dating advice communities.
- lue Pill: The "Blue Pill" is often posited as the opposite of the "Red Pill." It represents adherence to conventional or mainstream beliefs about gender, relationships, and society. In communities that use these terms, taking the "Blue Pill" means accepting societal norms and beliefs without questioning them, often portrayed as living in blissful ignorance.
- lack Pill: The "Black Pill" takes a more fatalistic and often nihilistic viewpoint compared to the Red Pill. It's associated with a belief that certain unchangeable traits (like physical appearance, height, etc.) predominantly determine one's success in areas like dating and social interaction. Black Pill ideology is often linked with extreme pessimism, defeatism, and a belief that systemic changes or personal improvements are largely futile.
PMC : Professional/Managerial Class
purity spiral: a sociological theory which argues for the existence of a form of groupthink in which it becomes more beneficial to hold certain views than to not hold them, and more extreme views are rewarded while expressing doubt, nuance, or moderation is punished
quant: short for quantitative analyst.
A hard search for this term provided too many instances of other words using these five letters, e.g. quantify, etc. and I didn’t have the patience to keep looking. But I’ve seen this term used and you will, too, if you keep reading this forum.
quokka: I have no idea what this means except a small marsupial. Help. Thank you @naraburns. The origin is here. From what I can gather a quokka is a kind of gentle-dispositioned person, innocent of nature, who is a bit of a nerd and wants to discuss things in good faith. Often applicable to certain autists. It is not a pejorative term. Edit: Maybe it is.
"Can you imagine a bunch of quokkas going about EA and Skynet every two days on the forum?"
Russell conjugation: a rhetorical technique used to create an intrinsic bias towards or against a piece of information.
shape-rotator: Someone with high mathematical and technical skills, often portrayed as rivals to the wordcels (who have stronger language and verbal skills)
“I thought we were shape rotators?”
soyjak: (I still only vaguely understand this.) An online image of an emasculate man, often with an excited expression, with an art style based upon the original wojak.
"This looks more like an excuse to draw your enemies as the soyjak and yourself as the, uh, tiger." Edit: Despite repeated attempts, I cannot get this link to work.
stochastic: having a random probability distribution or pattern that may be analysed statistically but may not be predicted precisely.
technocrat: an adherent to technocracy, or the government or control of society or industry by an elite of technical experts as opposed to professional politicians.
thot: From “That Ho Over There.” A woman who has (or is presumed to have, for whatever reason) many casual sexual encounters or relationships. Likewise, e-thot is a woman who makes money online from male (or predominantly male) audiences, by doing whatever for cash.
"I've seen sponsored ads (with the "ad" tag) for individual OnlyFans thots."
Third World-ism: a political concept and ideology that emerged in the late 1940s or early 1950s during the Cold War and tried to generate unity among the nations that did not want to take sides between the United States and the Soviet Union.
Noe: various users question the meaning and use of this term.
tradfem: a portmanteau of "traditional feminism" in reference to belief that adherence traditional feminine gender roles are better or more correct, especially those held by conservative Christian Americans, especially WASPs. Edit: Also a play on "radfem" or radical feminist. Thank you again, @naraburns.
“The Harrington and the other tradfems are hard to place on the left-right axis.”
Varg: I still don’t know what this means. I found various meanings of varg but none are satisfactory. Help.
Von Neumann: synonymous with “really big-brained person” as far as I can tell. Refers to John Von Neumann, a computer guy. Notably a “Von Neumann probe” would be a spacecraft capable of replicating itself. Edit: As I said above, misrepresentations are my fault.
Westphalian: the concept of state sovereignty and the idea that each state has exclusive sovereignty over its territory and domestic affairs, free from external interference. From a series of treaties in 1648. We also have a member with this as part of his username.
Wignat "wigger nationalist" and was originally used to describe lower class, violent, and unattractive neo Nazis that were willing to engage in street violence and unabashed Nazism with the use of swastikas and other symbols.
x
I was going to post this on my substack. But nothing is on it, and now this is not even on it, because I decided to post here, for better or worse. For those of you who have accidentally clicked on posts of mine before and recoiled, note that this is more of the same, and you’d best go ahead and scroll elsewhere now. There is no conclusion or point, and I certainly do not talk about Hamas or Jared Kushner or trans- (insert noun). No. The journey is the destination. Or however it goes. The destination is the journey. Whatever. This is the story of a colonoscopy. That should tell you enough.
I should play you some of that relaxing music that was in my head as I lay sideways on an examination table wearing my black nylon tear-away pants with the fly in the rear, gazing blankly at the video screen which was tilted down toward me as if to play the opening credits of Fantastic Voyage. I think I was humming in my head Billie Holiday's Embraceable You.
In order to not bury the lede, let me tell you what the doctor said at the end (there are many unintentional puns in this), so that you are not gripped by too much suspense and worry for my wellbeing. He said—and I felt he did not want to say this, that he would have preferred to say almost anything else—he said in what I imagined was a quieter voice than he normally used: “You’re fine.” Or that equivalent in Japanese. Actually what he said, clicking on his computer, the monitor of which now revealed neat squares of color slides depicting the interior of my colon: “綺麗.” That’s kirei, which means beautiful. It’s what you say about exceptionally attractive women, or sunsets, or flowers. It can also contextually mean clean, which in my case meant “There is no problem here with your colon.”
Two years ago, I went to my urologist because I had felt a pain in my lower abdomen that seemed wrong. It was not like any pain I had felt before. Probably. The urology clinic I go to is run by two older Japanese doctors who are there on alternating days, but my カルテ or file is the same for each, so they both have the same data on my PSA and creatinine and whatever else is measured in a urinalysis. The one doctor I seemed to keep getting on my appointment days is the more doubtful, less friendly one, and also the one who has the distinction of having done DRE on me more than any other doctor. DRE for those not aware means digital rectal examination. It is exactly what it sounds like, if you aren’t the type of person who assumes digital means something that might be shown on the readout of a G-Shock timepiece. No, digital means, in its barest sense here, finger.
He was stumped. Wherefore this pain? He asked me many questions in Japanese, and I tried to answer all of them as well as I could. It hurts here, yes. No I don’t have trouble urinating, no more than the usual with a prostate such as mine, at my age. On my insistence that I did in fact feel pain in the area I had indicated, he referred me to a research hospital up the road, writing a letter, enclosing a CD. He gave all this to me in a sealed envelope and sent me on my way. To the department of psychosomatic disorders.
The research hospital concerned is about ten years old, at least in its current incarnation in this set of buildings, and is still clean, has the requisite coffee shop on the second floor where one hears the reassuring and soothing clink of saucers and china cups, old people murmuring to one another as they take their morning repast of a “morning set”. Perceived as a whole the hospital as an institution would probably engender confidence even in the cynical. Clean carpets, working bright lights, purposeful movement, even the sick and probably dying blanketed and wheeled about on quiet gurneys seem to be in the best possible shape, considering. Like the Holidays Inn of my youth, before they became camp and moldy: All seems well-planned and expertly run. The nurses are more often attractive than not, at least from the bridge of the nose up (everyone is in masks in hospitals in Japan, all the time). The greeters and information desk personnel are female, lithe, and efficient, and give one a sense of being in the right hands.
The psychosomatic intake involved an interview of many minutes conducted by a nurse who seemed too young to be charged with adult crimes, should she commit any. I remember her as very pleasant and patient, and after about a half hour of interview as we were wrapping up I discovered via random throwaway phatic question that she spoke English rather well (the entire interview had been in Japanese, including my stumbling bumbling Japanese answers.) There was a time when this would have annoyed me. That time passed years ago. Now I just roll with it. People don’t like speaking English.
When I was finally allowed to sit in the presence of the doctor, who wheeled his chair the meter or so in and out from his desk expertly, and had the practiced look of a man who cares, and probably cares even about you, he told me, after a minute or two of speaking Japanese, to just say my whole spiel in English, to just get it out. Which I did, and briefly: I have pains here, and here, and I don’t know what is causing them and I would like to know. He then asked, in Japanese again: “What would be your ideal outcome here? What would you like us to say?”
I felt he must have learned these phrases in the elementary level courses that one takes in dealing with neurotic patients, the same courses that cause one to end up practicing in the psychosomatic disorder department. I did not say this. I did say: “I want someone to say ‘Oh, we’ve seen these symptoms before; this is probably what you have, let’s do a test.’” I do not know how I expected him to react to this, but it seemed as if I had given exactly the right response. If he had looked down and checked off a box on a clipboard, he could not have given off a more reassuring vibe. At last, it seemed some kind of examination would begin, rather than the type of qualifying session I felt I had been subjected to.
Please lie on an examination table. Okay. He palpated me deeply in the lower abdomen asking repeatedly “Itai? Kore wa?” “Does it hurt? How about here?” I realized during this examination that one can actually push fairly deep down into the abdomen presumably without causing damage, but when I finally did say “Yes, that hurts” it seemed such an obvious statement (as in: Why wouldn’t it hurt when you push basically all the way through my body?”) that I couldn’t imagine how anyone could learn anything from it, any more than if he had jammed a knife into my leg and asked the same question.
He nodded. He seemed to be thinking very carefully. He typed a bit on his computer, mumbling that I could get up from the table and put my shoes back on. I sat up and waited. This was how I was scheduled to get my first colonoscopy two years ago.
Fast forward in time to last week. Now we are at colonoscopy Number Two, which has been ordered for me by my local doctor, who has written the referral letter, again with a CD, though this time I am sent to the gastroenterology department. The pains have returned, though the first colonoscopy (and endoscopy, which was another day and involved a tube down my nose) showed nothing out of the ordinary. One polyp (not a word one wants to imagine, though apparently polyps after a point are normal) but it was removed, and found to be benign.
This time, I have been scheduled to have the procedure on a Thursday, and have cleared my schedule and bowels for this purpose. The way one prepares for a colonoscopy may be well-known to some of you, but I will iterate it here for the uninformed. According to the internet one is supposed to consume only a liquid diet the day prior, and cease all medication such as aspirin which would increase the chance of bleeding. There are various dangers if the colon has not been completely voided and a colonoscopy is conducted, among them a term I recently learned: intracolonic explosion. I do not even want to know how this term was coined or in what circumstances. (That is not an image link.)
In Japan, however, there are little boxes they let you buy for about 10 bucks that contain what are called レトルトパウチ or retort pouches that contain MRE-like substances, and these are supposed to get you all set. Each box has five pouches, three meals and two that contain watery rice. You are supposed to pour the contents into a pot or bowl and either briefly boil or microwave them.
The box has on its cover the various meals you will be eating on the day before. It looks like this. You can see each “meal” such as it is on the box, plus some little pills are there that I was supposed to take. The first meal, the breakfast, is here in its pouch, and then looked like this when I poured it in the bowl. It looked better in the pouch. Rice and stewed chicken and some egg. The lunch was daikon radish, more chicken but minced this time, and, according to my youngest son, some potatoes. It looked like this.
There were also two packets of the watery rice I mentioned, similar to what is known here as お粥 or okayu, which is the Japanese equivalent to the traditional chicken soup, i.e. it’s what you eat when sick, but without the egg that is usually a part of that dish. I dumped one of these on the dinner, which, in a contest, would have to be called the best of all the meals, but so Spartan as to be disappointing, though by the time you get to eating dinner you are so hungry that you are imagining if maybe you can just forgo the procedure altogether and just go grab a goddam cheeseburger. I was too hungry to take a photo at that point but here is what the box claims the dinner looks like.
Anyway over the course of the day I warmed all these up and ate them and drank a bunch of water and took the two little pills at 9 pm. The instructions said to then drink a cup of water at 10 pm but by then I just wanted to lose consciousness, so I drank the water with the pills. I went to sleep early as my family ate stewed yellowtail, white rice, garlic scapes with sesame, and miso soup.
On the Day Of, you wake and unsurprisingly you’re hungry. You are not supposed to eat or drink coffee, so I did not. I did drink some water. I put on sweatpants, wore the kind of sandals that slip right off, got all my stuff, and took the bus in. The instructions also say not to drive, as you will be woozy from the drugs they give. I remember the first time I had one of these, the drugs were very effective. This time they would not be, but I did not know that going in.
I won’t bore you with the details of checking in and finding my way to the right department, but all went efficiently. Eventually I was cordoned off with a group of two men and three women who were also scheduled for the procedure. I am not young, but everyone looked much older than I feel I look or am. But who knows. Everyone was wearing a mask, including me. They sat us all down at tables and put in front of us these big jugs of fluid, which had our names written on them in black ink. That jug looks like this.(My name is on the side that is turned around.)
If you are thinking to yourself “That’s a lot of fluid” you are thinking the same thing I thought. And it is. It’s two liters.
The nurse at this point is going on and on in Japanese about what to do. She is very animated and friendly and you can tell she has done this lecture many times. It is like a performance, a routine, she hits certain points like punchlines, and we all breathe in a laughing-like way accordingly at these moments. Drink 250 ml in the first 15 minutes. Then another 250 ml in the next 15. By then you should have gone to the toilet at least once. If we have not, we are informed, we shall be given an enema. This seems a foreboding prospect, despite what awaits us. Somehow an enema seems worse than a colonoscopy. We are also given more little pills to take. This is the blister pack of mine. I had greedily swallowed them before I thought to take a photo for posterity.
The next photo you might not want to click on. It is a little laminated paper with six five photos that we are told are the stages of what our voiding will look like. The first is dark, then they get progressively lighter. Like a little picture book of defecation. Here is what that looks like. You will note the cute little bear in the top right, because everything in Japan must have a cute mascot associated with it, even the act of crapping.
If you did not click it’s fine, but the last pic of the stages is clearish, like very weak tea. At this stage, if this is what you are producing out your rear end, you are a Go for the colonoscopy and you are supposed to call the nurse into the john and have her inspect it to be sure.
I begin drinking. After 20 minutes I have had about 300 ml of the stuff, which tastes like poor quality Gatorade with no sugar and too much salt. I excuse myself and sure enough I will not be one of the enemized, which is good news. Time passes and the nurse has switched on the television. She has said we should not just sit there and watch it, but that is what everyone does—all except one old guy who prefers to stare at the corner in the opposite direction, as if mourning all the mistakes in his life that led him to this little squalid room. Instead of sitting there and watching TV the nurse encourages us to walk around, and even to push in on our abdomen to get the process rolling. I take a stroll, come back, and watch the TV again. On the television there is a show called ラビット which reads like “Rabbit” but I realize is supposed to be “Love It.” On the show three extremely homely guys who are probably comedians are visiting a racetrack and oohing and ahhing over it. With these guys is one extremely attractive woman, but before I can try and follow what it is they are all doing at the track I have to excuse myself again.
Eventually I’m ready, after about six trips to the restroom. The bowl looks so clear it is as if I haven’t even sat on it. I push the chime and the nurse very excitedly runs up, much in the same was if I were an elementary school child who just announced that he finished making some bit of artwork and she, my teacher, has come to admire it. “Ooh that’s perfect,” she says, gazing into the toilet bowl. “You’re ready,” she says.
She escorts me to another small locker room. I have left behind my cohorts, who are all quietly drinking and sitting and contemplating the television, except the one sad man. Two of the women have struck up a conversation. Their voices dim as I walk down the hall to the locker room. The nurse says to take off all my clothes and put on a little pair of black nylon pants, and then put on what looks like a blue pajama top thing, something like Luke Skywalker might wear on Tatooine. Then put my stuff in the locker. A beat or two passes and I am wondering if she is going to watch me do all this, but she leaves and closes the door behind her. There is no lock on the door, so I make haste. The fly, she has noted, is in the rear of the nylon trousers, for obvious reasons.
Clad in this getup, I step out. I am still wearing my athlete’s sandals. The nurse takes me past my cohort again, and they are all still drinking or sitting waiting to drink. Their Japanese politeness prevents them from acknowledging me, though my American friendliness feels somehow hurt by this, as if they might cheer me on. I go first, wish me luck sort of thing. Maybe they just don’t care at all. I go into a room where another nurse is busy with something. This room seems less clean than the rest of the hospital. Scuff marks, signs of wear. She wheels over a rolling upright pole with an IV bag on it and wipes my arm with alcohol. As she is inserting the needle I can’t think of anything to say so I say: “Your watch is nice.” It is, though my eyes are so bad at focusing on close things I cannot tell if it is a Jaeger-leCoultre or just some offbrand. Probably the latter if she’s a nurse in a research hospital, but you never know. She says thank you in a polite way but I imagine she thinks I am an idiot. Which, I think, I am at this moment.
(continued in reply post due to character limit.)
The hardest part of telling any story is getting started. The beginning is as good a time as any.
How does a man wind up in a war? The same way he winds up in Carnegie Hall.
I lost this fight at six.
No shit, there I was. Laying in the dust of my mother's garden with ants on my dick, and Sam Meck kicking me in the ribs.
I'd been climbing high up in a tree when the other kids came home from school. They made a beeline. A couple of them found long sticks and poked me until I fell. Eighteen years later, a doctor in Baghdad would tell me my left arm had been broken that day. But since an X-ray would have made Jesus sad, all I knew then was I had a useless arm, and a lot of pain. Didn't make it past the garden before they caught me.
Most of the boys hadn't touched me yet. Sam and his little henchman Guerrito were the instigators, the leaders. They'd gotten the sticks. The rest of them just fanned out and seemed happy to spectate. The pom squad of trash society, drifting along within the group opinion, cheering for the good guys, cheering against the bad guys, so certain they know which is which. I hated them most of all. How pathetic and vile, to be cruel and also cowardly. Sam was a different, more immediate problem.
I was a weird kid with a weird family. We were in a faith healing cult, warned against interacting too much with “the world”. We'd always be a target and as the oldest I was going to be the first to deal with it. Already the neighbor kids had begun tormenting me and my retarded younger brother. And what was my skinny ass gonna do? I was a theology nerd; sensitive, intellectual, thin and uncoordinated. Sam was still kicking me. I needed a plan.
My options were limited. Parents were strict pacifists. Foxe's Book of Martyrs was very much required reading. Involvement in any conflict would result in punishment. There were no justifications for violence, so the only thing I was allowed to do was take the beating. And then turn the other cheek or some shit. God's will was for Sam to keep kicking, and for me to forgive him when he got done.
Weird to notice. The ants were crawling on me immediately when I fell. Must have landed right on the mound, shorts were a bad idea. Tick- tock motherfucker, this is the logic of violence. You get to figure it out in pain, with limited time before your ability to resist is gone. My arm really hurt.
Short answer, I'm screwed. Nothing I could have done would stop the beating. But long term, what's the play? Already thinking about tomorrow. Got to have a plan.
I could hide, but can't do that forever. We live next door.
I could stay inside with three infant siblings and an increasingly insane mother. Not a great option.
We'd played baseball a few months before and Sam got hit in the face with the ball and he bled everywhere and started screaming like a bitch. Mom said to play careful with him because he was a hemo-something..... a kick landed on my bad arm. There was blood and dirt in my mouth, snot and tears streaming in thick ropes.
The reality was that people are mostly cowards. The four boys ranged around watching a kid beat another kid. I remember neither their faces nor their names. They hound after the misfit, the outcast, the different, but only when it costs them little. All I had to do was raise the price and maybe they would drop out. It wouldn't stop the beating now, and it would mean getting punished by my parents later. But they'd know that picking on me wasn't free. The next time, maybe there would only be four. I was an optimist then.
I rolled into Sam's post foot the next kick, curling my bruised stomach into his sneaker as it struck, capturing his legs together at the ankles. Shrieking, I pulled him down. He fell badly. I threw a leg over, sat tall on his chest and hit him with a big rock I'd been laying on for the last thirty seconds. Then again. With each swing, his face opened wide, white skin over red flesh over white bone. Blood was flying everywhere, spattering my arms, shirt and the bean plants. My own bloody, muddy mucus hanging from my chin in long strings, dripped onto the little green alligator on Sam's shirt. Strange the things you focus on at times like this.
Guerrito piled into me, followed by all the other boys. Small fists rained down as I tried to cover my injured arm and rolled to the fetal position. I was giggling and sobbing and dry heaving and spitting dirt. Crazed, wounded. Sam was crying and saying he had to go home. The boys were pushing off of me, and then.... The sun was warm, the roto-tilled earth soft, the ants still mobile, tickling. I lay there a long time, trying to get hold of myself. It was time to pay the bill.
A long walk fifty yards or so to the house. Mom cleaned me up, told me to stop crying like a baby and wrapped my arm in an Ace bandage. Then she paddled my ass with a wooden spatula and sent me to my room to wait for dad to get home, when I'd get the proper whuppin.
I wasn't scared of a belt any more. It wasn't going to hurt more than what had already happened. The dread that a kid normally has when they must wait for a punishment was gone. Only rage, self-pity, and a sense of betrayal remained. I'd been sent out into the world with “turn the other cheek” as my only tool. That would never happen again. A promise, and with it, strange elation.
I could see Sam's face when I had come off the ground. His eyes were dumb and confused at first. When I hit him the second time, he was scared, frantic. In that instant, the tables turned, briefly and forever. That feeling stayed with me the rest of my life. It's the tiny black radioactive core that powers me, warms me, resolves me. This was what I was born for, this was what I was meant to do. No theological argument or academic interest would ever hold a candle to that sort of pain and power.
In time, this decision would cost me and a great many people a great many things. But I was six at the time, and learning my first life lesson. No matter the odds, the outcome, the rules, the commandments from on high, I could make the world bleed.
No gods, no masters.
Just us and these rocks, sunshine.
I think I found this more interesting than the original biography review by Scott. There is a lot of distilled wisdom in these posts.
However, there is one area that always rubs me the wrong way. It is smart people who don't know dumb people talking about intelligence.
Where I am now in life I interact almost exclusively with smart people. Not just high IQ wiz kids with no experience. But people that have both the raw brain power, and the life experience to be sharp and wicked smaht. I'm in a rich neighborhood, and wealth has a noticeable correlation with IQ. I currently work for an institution that employs academics who must explain their work to the media (so they can't just sit in an ivory tower and write illegible crap). I use to work at a tech company that for quite a few years basically gave people an IQ test before they could join, and they were willing to fire people who didn't work out (the selection effects weren't perfect but they were certainly noticeable). My college friends were mostly from an "honors" section that got scholarships and accolades for academic achievements.
This was not always the case.
I went to highschool in a nice-ish area. The highschool was pretty decent for where I lived, but it still had noticeable rates of teenage pregnancy, drunk driving fatalities, minor gang fights (no more than temporary hospitalizations), about a fifth of the school below the poverty line, and a racial mix that actually came pretty close to matching America's general racial mix.
This highschool had dumb-dumbs. Probably something close to an average amount of dumb-dumbs. But at the time it was painful how many of them there were. I am smart for the general population, but a bit of a dumb-dumb when I get into smart people circles. 95th percentile on SATs. 1 in 20 seems only ok, but in a random class of ~30 kids I was likely to be the smartest or 2nd smartest. And its not the academic under performance that ever bothered me. I wasn't in any position to judge, I did well on standardized tests, but I was solidly a B student at best. Most of the material seemed dumb and stupid. We were all often doing equally bad at it. It was the everything else that bothered me about interacting with chronically stupid people.
I often heard people brag growing up that they were "street smart" while some academic achiever was "book-smart". This gave me the false impression that there were two kinds of people out there and there was just a trade off between the two. That was badly wrong. Some people are just dumb. They can fail to learn how to read, and fail at not walking into oncoming traffic, and fail at not picking a fight with a group of kids that will kick their ass. There are people that just seem to make repeatedly bad decisions in all areas of their life. I grew to hate these people, because loving and caring about them was too painful. To watch them make terrible decisions again and again, no matter how you advise them, no matter how much you try its like they seem determined to make their own lives a living hell by refusing to understand the world around them.
Bringing this back around to Elon Musk:
Yes he is smart. He is very smart. If he doesn't seem that smart compared to the people around you, then congratulations you live in a smart person bubble. I live in one too, its great! No one is ever making terrible decisions that might casually endanger me. No one is starting physical fights, because words hurt their brain too much. They know all the latest social norms, and when to violate the silly ones to make a joke. I can have deep conversations with them about nearly any topic, they might not know the details, but if I make it interesting they will pick it up and participate. The people around me know how to manage their money, so they aren't ever begging me for handouts, or trying to nickle and dime me on shared expenses.
The phrase "check your privilege" comes to mind, but the tone that people normally use feels very wrong. Just imagine me saying it in the same way a surfer says "kowabunga dude!" while offering a high five.
Plunkitt of Tammany Hall: The Machine Politician Vs the Administrative State
Once upon a time, in the bad old days, American politics was dominated by the so-called “urban political machines.” These machines, composed of unelected, self-appointed elites, chose political candidates based on their own narrow interests and exploited uneducated and impoverished city-dwellers for their votes. By perpetuating a ruling elite based on cronyism rather than competence, they impeded good government and held back progress. Fortunately, around the turn of the twentieth century, a series of reforms broke the power of the urban bosses and ushered in a more enlightened ruling class who governed for the sake of the greater good.
That’s the popular narrative, anyway. Like many (most?) people here on the motte, I’m broadly skeptical of high modernity, progressivism, and Whig history. Accordingly, I’ve always been dubious about the knee-jerk “machine politics=bad” reaction. This skepticism has been intensified by the general failure of western efforts to transplant our own political institutions into the third world, where such clientalist arrangements are still pretty common. Clearly, they’re working for someone, or else everyone, presumably, would joyfully adopt US norms and institutions. I happened to come across a primary source from the hey-day of machine politics, “Plunkitt of Tammany Hall” (available on Project Gutenberg), in which a very successful machine politician described his business in his own words. It was brief but compelling read; I’ll be describing my impressions below, mixed in with some larger thoughts about machines as a form of political organization.
Most people have probably heard of Tammany Hall. From roughly 1789 to the 1930s, the organization exercised a dominating influence in New York city politics, occasionally verging on an outright monopoly. George Washington Plunkitt was a life-long New Yorker and proud “practical politician” associated with Tammany Hall throughout his career. “Plunkitt of Tammany Hall” is largely told in his voice, with occasional notes from his editor. And it’s a compelling voice. Plunkitt may or may not have been a criminal, and he was certainly not a responsible leader by modern lights. But he is a charming and often insightful raconteur. I’m not sure I’d loan him any money, but I’d definitely buy him a beer.
Much of that charm comes down to his frankness. Plunkitt is quite honest about having gotten rich off of politics, though he denies having broken any laws. He claims to practice what he calls “honest graft”, which seems nowadays equivalent to what we could call insider trading, using his knowledge of city politics to make favorable investments. So far as I can tell, this wasn’t actually considered a crime, though it certainly rubbed some people the wrong way. Likewise, Plunkitt is extremely frank about appointing friends and associates to government jobs as a reward for their political support. He expresses shock at the suggestion you would do it any other way. Nothing seems to excite his ire more than “the curse of Civil Service reform” a topic he harangues the reader on frequently, and to which we shall return in a moment. One of the first things that struck me in the text was Plunkitt’s de-emphasis on what we moderns would normally consider a key skill of a politician, public speaking. “The men who rule have practiced keepin’ their tongues still, not exercisin’ them. So you want to drop the orator idea unless you mean to go into politics just to perform the skyrocket act.” Nation-wide media was relatively new in Plunkitt’s time, the trans-continental telegraph only having been completed in 1861. Nation-wide broadcast media was not even a twinkle in Marconi’s eye yet. Presumably, in such an environment the kind of mass popular appeal that politicians today cultivate was much less of a requirement.
There are probably some readers who will point to the general mediocrity of most political speeches today and argue that no, party-elite connections are still what matters. I won’t say these people are entirely wrong, but I don’t think they’re entirely right either. It seems very unlikely to me that, for example, Barack Obama’s popularity within the college-educated demographic was entirely unrelated to his ability to performatively model PMCi-values on a national stage. For that matter, Trump is frequently cited as someone who enjoyed considerable success in spite of a general lack of party-elite connections. I suspect the increasing importance of nationwide broadcast media was a significant contributor to the eventual decline of the machines.
Closely related to this is the role of class dynamics. Plunkitt comes across as something I didn’t realize existed, an urban populist. Speaking of a hypothetical candidate who asserts “I took first prize in college at Aristotle; I can recite all of Shakespeare forwards and backwards; there ain’t nothin’ in science that ain’t as familiar to me as blockades on the elevated roads and I’m the real thing in the way of silver-tongued orators.”, he responds “I guess you are not to blame for your misfortunes, but we have no use for you here.” Its hard to imagine a blunter dismissal of PMC values. Like modern populists, he rails against elites, who he charges with hypocritical moralism and petty tyranny “We don’t own our streets or anything else…we’ve got to eat and drink what they tell us to eat and drink, and…choose our time for eating and drinking to suit them. If they don’t feel like taking a glass of beer on Sunday, we must abstain. If they have not got any amusements up in their backwoods, we must have none. We’ve got to regulate our whole lives to suit them. And then we must pay their taxes to boot.” These elites aren’t only tyrannical, they’re unpatriotic “Nobody pays any attention to the Fourth of July any longer except Tammany and the small boy. When the Fourth comes, the reformers, with revolutionary names parted in the middle, run off to Newport or the Adirondacks to get out of the way of the noise.”
This is striking because historically, cities have been recruiting pools for “progressive” movements, where “progressive” broadly means “someone who wants to replace the established order with something new.” There’s a danger in over-extrapolating from recent history, but this really does seem to be a pattern. Many of the most radical excesses of the French Revolution were driven by attempts to appeal to the sans-culottes of the Paris mob. Conversely, the revolutionaries imposed some of their most brutal repressive measures in the rural Vendee. A century later, orthodox Marxists famously thought that Russia was not yet ready for a revolution because the urban working class was still too small a percentage of the population. This pattern is embedded in our very language. The word “Pagan” derives from the Latin word for “rustic”. It’s modern usage originated in the period where Christianity had become an elite religion but had not yet been fully imposed on bitter-clingers in rural parts of the empire. The modern analogs would be the trumpenproletariat in flyover country.
How do we square this circle? I think part of it is that “populism” in common usage tends to be something intended to appeal to working class voters, who as a group are usually social conservative but economically liberal. In the modern world, populism is associated with rural areas because structural changes to the economy have skewed urban populations towards professionals rather than the working class. Per Wikipedia, in 1910 only 37, 200 Bachelors degrees were awarded across the whole country, in a population of just under 92 million. Of that 92 million, nearly 5 million (4,766,883)lived in New York. 4,766,883/37,200 = .008. In other words, even if every college graduate in the country had lived in New York City, they would have made up less than one percent of the population. By contrast, today 39.5% of NYC has a Bachelors or higher. And NYC isn’t even in the top five most educated cities in America! Likely in the eighteenth-through-early twentieth centuries – the heyday of the industrial revolution – urban populations were skewed in the opposite direction than they are now.
Of course, that doesn’t explain the other instances of urban progressivism cited above. I think we can chalk that up to these movements being an alliance between elite-aspirants and the working class, with the elite-aspirants providing the socially-liberal rhetoric of the movement, while the footsoldiers are largely motivated by pragmatic material concerns. This is as close as I have to working explanation, unless of course you think that the progressivism-urbanism association is all a mirage.
In lieu of ideological appeal, technically wonkery, or even a charismatic public persona, Plunkitt offers a vision of politics based on personal relationships and the rendering of services. He began in politics by getting his cousin to promise him his vote – not to Plunkitt personally but to whoever Plunkitt told him to vote for. Plunkitt offered this vote, along with his own, to the district leader. Then he recruited two of his old school-friends. “Before long, I had sixty men back of me and formed the George Washington Plunkitt association.”
In return for these votes, Plunkitt was offered positions in government. The exact position seems to have changed as the voting blocs he commanded grew in size. Its less clear from the text what exactly these individual voters received. Certainly as Plunkitt grew in power in the organization, he could offer jobs to some of his supporters. Judging by his repeated denunciations of civil service reform, this kind of patron-client relationship was key to the whole edifice. But of course, there couldn’t have been enough full time jobs to hand one out to every voter. The whole thing wouldn’t scale. Plunkitt’s access to the apparatus of government probably meant he could hand out contracting opportunities even when he didn’t have a full time job on hand. He says as much. But a lot of it also seems to come down to small acts of friendship and making people like you. “I know every man, woman, and child in the fifteenth district, except them that’s been born this summer – and I know some of them too…I reach them by approaching them at the right side…I hear of a young feller that’s proud of his voice, thinks he can sing fine. I ask him to come around to Washington Hall and join our Glee Club. He comes and sings and he’s a follower of Plunkitt for life. Another young feller gains a reputation as a baseball player in a vacant lot. I bring him into our baseball dub. That fixes him. You’ll find him working for my ticket at next election day. Then there’s the feller that likes rowin on the river, the young feller that makes a name as a waltzer on his block, the young feller that’s handy with his dukes. I rope them all in by givin them opportunities to show off.” Presumably the various public observances which Plunkitt alludes to were excellent opportunities for generating this kind of social capital.
These acts of friendship could also take a more practical form. “What tells in holdin’ your grip on your district is to go right down among the poor families and help them in the different ways they need help. I’ve got a regular system for this. If there’s a fire…any hour of the day or night, I’m usually there with some of my election district captains as soon as the fire engines. If a family is burned out I don’t ask whether they are Republicans or Democrats, and I don’t refer them to the Charity Organization Society, which would investigate their case in a month or two and decide they were worthy of help about the time they are dead from starvation. I just get quarters for them, buy clothes for them if their clothes were burned up, and fix them up till they get things running again.”
Obviously there’s no statistics provided. But my intuition tells me this sort of thing was probably pretty effective. I grew up comfortably middle class; I’ve never known what its like to be worried about where my next meal will come from or where I’ll sleep. I’ve never worried about whether my wife or my kid will have enough to eat. If I was worried about that – well I don’t think there’s much I wouldn’t do for someone who solved that problem for me. Leaving intuition aside this model – tangible benefits for friends and family in exchange for loyalty – is arguably what leadership looked like for most of human history. Reciprocal altruism is a bedrock of human behavior. The intimate nature of such exchanges elevates them beyond the merely transactional; emotional ties soon develop and invest these relationships with the aura of the sacred. I’ve little doubt Plunkitt’s methods were effective. For that matter, I have little doubt that from inside, most participants in the machine were perfectly satisfied with the arrangement.
What caused the decline of machine politicians? And were they as bad as modern opinion holds? The first question can be answered more or less satisfactorily. The second is mostly a matter of opinion. The most commonly cited factor in the decline of the machines is the introduction of the direct party primary. Nowadays, we take it for granted that party members come together in a sort of internal election to vote on who they’ll put forward as a candidate for. Since politics is our national sport, presidential primaries often get breathless coverage in the media. But in fact, this wasn’t common for much of our history. Direct primaries were fairly rare prior to the mid-nineteenth century and didn’t really pick up steam until around the turn of the century. Prior to that, it was common for local voters to select delegates to a nominating convention, who in turn would choose the candidate. In theory, the transition to direct primaries allowed candidates to “cut out the middleman” i.e. intra-party elites and “bosses” and appeal directly to voters. Plunkitt and his ilk, who relied on horse-trading rather than offering a coherent vision of the common good, were finished.
This triumphalist narrative remains common. Its not entirely wrong so much as it is incomplete. This study suggests that the introduction of direct primaries was correlated with a decrease in Congressional representatives (both Senate and House) voting in line with party leaders. Insofar as “party leaders” are a proxy for the old school bosses, we can say that that this represents a weakening of their power. But direct primaries are only part of the story.
Another major part is this: technological and demographic changes made interpersonal connections an inefficient way to mobilize voters. In 1793, the House of Representatives was only 105 members. Today, it has 435, a number set by law in 1929. In 1793, there were roughly 34,000 voters per representative. Today the ration is roughly 1: 761,000, an order of magnitude higher. While I don’t have data for state and municipal legislatures, its safe to assume the same trend holds.
“Dunbar’s Number” is the theoretical upper limit on the amount of close relationships anyone can have. Estimates vary between 150 and a little over 200, but the bottom line is this: for most of our existence as a species, humanity operated in relatively small bands of hunter-gatherers. We evolved to handle a certain number of point-to-point contacts. Past that limit, it becomes necessary to start sorting people into categories of one sort or another. A politician in a district with, say, 500 people can personally know a large chunk of them. In a district with 5000, he can know fewer, but his ward heeler subordinates might still know many on an individual level. But by the time you hit 500,000, this sort of personalized relationship is impossible. Even if you maintained a small army of volunteers to go around and engage with individual voters, said volunteers would soon themselves exceed Dunbar’s number, diluting the strength of their relationship to the politico.
Instead of cultivating relationships with individuals, you cultivate relationships with voting blocs. Farmers, lawyers, blue-collar workers, gun owners etc. Legions of specialists search for ever-more-subtle coalitions of interest to solicit. This is where broadcast media becomes important. Broadcast media is a way of efficiently marketing to large numbers of potential voters, far more efficient than simply going door-to-door.
Its not only the politicians who start sorting people into boxes based on professional, ethnic, or social status. The voters will do that themselves. At one time, people’s sense of identity was largely local – their town, their neighborhood, their block. But as the world flattens and the flow of information, goods, and services becomes ever-less constrained by geography, people start to think of themselves in larger terms: a Christian, a lawyer, a Republican, an X, Y, Z. These larger identities have always been present of course, but they make up an increasingly greater portion of any given individuals sense of themselves. Plunkitts’s methods relied on local ties that are increasingly less important.
Another major factor was Plunkitt’s great white whale “the curse of civil service reform.” While Plunkitt was presumably most concerned with New York’s Civil Service reform, these developments were merely reflective of larger trends in the country. For most of the nineteenth century, government administrative jobs had been distributed at the discretion of elected officials. But in 1883, Chester A. Arthur signed the Pendleton Civil Service Reform act, which provided for certain federal jobs to awarded on the bases of competitive examinations rather than by administrative fiat. Those who had gotten their jobs through competitive examinations could not subsequently be removed for “political reasons”. At the time, the act covered only a small percentage of the executive. But, in a true stroke of genius, the act was written with a “ratchet provision” which allowed the president to add positions to those covered under the Pendleton act. So, if your side was likely to lose the election, you could add all your appointees to the rolls right before leaving office, and your successor couldn’t remove them. After enough iterations of this process, about 90% of the civil service was covered under the act. While the Pendleton act only covered federal civil service jobs, a similar process seems to have taken place within various states.
From the perspective of the twenty-first century, one is half-tempted to ask: do you want the deep state? Because this is how you get the deep state. In the process of enshrining their ideals in law, the professional-managerial class of the day created the legal basis for entrenched bureaucracies to pursue their collective interests even in the face of opposition from the nominal chief executive. Debates about the role of meritocracy aside, Civil Service reform went a long way towards eliminating the middle ranks of the machines, those on whom men like Plunkitt relied.
The bureaucratization of the civil service also contributed to the ballooning of the administrative law sector, which further eroded the ability of elected officials to actually make a difference in the lives of their constituents. Remember when Plunkitt said that local charities would get around to doing something just about the time that a family was starved to death? One of the services machine politicians provided their constituents was the ability to apply pressure on the machinery of the state. If, say, you felt that your property had been unfairly seized, or you had been denied something you were owed, or you had been inconvenienced in some way by The Man, you could turn to your ward heeler, who in turn could bring the matter to your local elected representative. Nowadays – you’d hire a lawyer.
America is almost unique in the extent to which the actions of executive agencies are dictated by lawsuits or the fear of lawsuits. The Equal Protection Clause means effectively, anyone can sue a government agency for virtually anything. The most well-known application of this on the Motte is probably the collection of legal decisions arising from civil rights law which has contributed to the institutionalization of progressive tendencies throughout the public and private sector. This is correlated with explosive growth in the legal profession. In 1960, there was roughly one lawyer for 627 people in the country. By 1987, there was one lawyer for every 354 people. Today, there is roughly one lawyer for every 250 people. Notably, some 30% of Representatives and 51% of Senators have law degrees. To a large extent, the politician as advocate has been replaced by the lawyer as advocate. Instead of being compensated by votes, today’s advocates are compensated in publicity and cash. Rule of law becomes rule of lawyers.
All that said, were the machine politicians so bad? I’ll admit that I am somewhat tempted to romanticize Plunkitt and all his spiritual kin, from Boss Tweed to Enoch L Johnson. Undoubtedly they were self-interested, but no more than any other politician. They made it their business to know their constituents and to provide something meaningful to them. Above all, they seemed to have been accessible. They gave their constituents a sense of agency, a feeling that there was something they could do and someone they could turn to when things went wrong or when they were being pushed around. How many of us can say the same? If, for example, my local police department confiscated my property in a civil forfeiture case, I would have no choice but to pursue a costly and time-consuming remedy through the courts. I’m fortunate enough that I could probably afford it; many others couldn’t.
At the same time, I’ve already discussed the major limitation of such political machines: they simply don’t scale. In the savage war of all against all, scale is everything. The big fish eat the little fish and organizations – political, economic, or social – which can more effectively mobilize greater resources usually out compete the smaller ones. Even if such an arrangement could survive, I expect that in time it would lose the qualities that make it appealing. A political machine would eventually become vulnerable to the same oligarchical dynamics as every other political system, and the machine politicians as detached and self-absorbed as every other elite.
Join me, then, in raising a glass to Plunkitt and all his tribe. Like Haast’s eagle, the woolly mammoth, or the horse nomads who conquered half the world, they were magnificent in their day. But their day has passed. The world has changed; I do not think we will see his kind again, for better or for worse.
The gap
This gap between the average male and female life expectancy of given population group is alternately labeled as Life Expectancy Gender Gap (LEGG) or Gender Gap in Life Expectancy (GGLE). Going forward I will be using the LEGG acronym.
The expectancy
There are basically two types of life expectancy. In historical context, we usually refer to the Cohort Life Expectancy. We track a group of people born in a particular year, many decades ago, and observe the exact date in which each one of them died. Then we can calculate this cohort’s life expectancy by simply calculating the average of the ages of all members when they died.
It is of course not possible to know this metric before all members of the cohort have died. That is why when talking about the present and the future we use Period Life Expectancy. This is an estimate of the average length of life for a hypothetical cohort assumed to be exposed, from birth through death, to the mortality rates observed at one particular period – commonly the previous year. Estimates of life expectancy of the current generation, which are also used in the calculations of indexes like the Human Development Index or Gender Development Index, are of this Period Life Expectancy type.
There is a corollary: because we are judging the existing generation based on mortality rates of the previous year only, whatever happened to that generation before that year is not taken into account. So let's say. if there was a recent deadly pandemic that affected one gender disproportionally, and this pandemic ended and is not affecting the mortality rates of the previous year, there will be little evidence of said gender disproportionally in the current life expectancy estimates.
The lost years
The first important thing to know about the LEGG is that its impact is, without an exaggeration, enormous. Let's take for example the US, with a LEGG of 5.8 years at the average predicted age for men and women 73.5 and 79.3 years respectively. Do you see the enormity? You don't, do you.
Ok, let's put things into perspective - how do you measure an impact of early death? With Years of Potential Life Lost (YPLL). This is an estimate of the average years a person would have lived if they had not died "prematurely". It is usually reported in years per 100,000 people and the reference, "mature" age should correspond roughly to the life expectancy of the population and is now usually given as 75 years.
Now, men and women in the US lose some 8,265 and 4,862 potential years per of life per 100,000. Given the population as 332 millions, men lose some 5,648,980 more years of potential life than women. Do you see the enormity now? Not yet?
During the roughly 3.5 years of WW2 the US lost 407,300 military and 12,100 civilian lives. With an average life expectancy back then 68 years, and a guestimated age at the time of death 21 years, every killed American lost some 47 years and the US as a whole lost some 5,640,000 potential years of life every year of the war. Do you see the enormity of the LEGG now? I think you do.
The causes
The second important think to know about the LEGG is that nobody seem to care. Biologists, statisticians, politicians, Wikipedians - not even men's rights activists - nobody seems to be franticly looking for the causes or proposing policies to stop this haemorrhage of men's lives. Let me paraphrase what Wikipedia has to says about it:
It is the life style, men drink more and smoke more and eat crap. And it is also the biology, men lack the double X chromosome, we see this across all mammalian species, plus male babies and boys dies of diseases much more than girls.
Speaking of Wikipedia, it has dedicated pages for many things, including the Orgasm gender gap, but it does not have a dedicated page for the LEGG.
To my surprise I have not been able to find any further information, neither on biology forums, nor on Google Scholar. Studies usually focus on one cause or divide the mechanisms into social and biological but there our knowledge seem to end.
At this point I was so intrigued that I decided to do some "research" myself. My first observation was that there is a great variance between developed countries with similar GDP and life expectancy, suggesting that a large part of the gap is not biological. Example:
- 2021 Norway - LE: 83.16 years, LEGG: 3,0 years
- 2021 France - LE: 82.32 years, LEGG: 6,2 years
Next, I knew where to find Eurostat data on causes of death - unfortunately only from 2010 - and I filtered out everything mechanical: suicides, assaults, accidents and drug and alcohol overdoses. The LEGG shrunk significantly:
- 2010 Norway - all LEGG: 4.54 | non-mechanical LEGG: 3,51, decrease by 29.5%
- 2010 France - all LEGG: 7.14 | non-mechanical LEGG: 6.19, decrease by 15.3%
Then I was curious how much of the LEGG is caused by mortality differences of infants and children so I calculated non-mechanical LEGG at 20 years, as opposed to LEGG at birth. The difference is negligible:
- 2010 Norway - non-mechanical LEGG at birth: 3,51, non-mechanical LEGG at 20: 3.37, difference: 3.8%
- 2010 France - non-mechanical LEGG at birth: 6.19, non-mechanical LEGG at 20: 6.07, difference: 1.7%
Next, I did one more napkin calculation. Assuming that smoking reduces the life expectancy on average by 10 yers and smoking rate among French men and women are 0.349 and 0.319 and smoking rate among Norwegian men and women are 0.17 and 0.154, I reduced the LEGG further:
- 2010 Norway - all LEGG: 4.54 | non-mechanical, non-smoking LEGG: 3,35, decrease by 35.7%
- 2010 France - all LEGG: 7.14 | non-mechanical, non-smoking LEGG: 5.89, decrease by 21.2%
Of course this does not mean the reminder is caused by biological factors. There are drugs and alcohol, there is a meat consumption and overall life style. Man also do more paid work so there is work related stress and exposure. It should not be a rocket science to isolate these factors, actually, it would amount to a very cool paper with plenty of citations. So where is this paper?
Actually, I found one piece of information: Causes of Male Excess Mortality: Insights from Cloistered Populations, the abstract talks about 11,000 Bavarian monks and nuns living in "very nearly identical behavioral and environmental conditions" with nuns having only a "slight advantage" in life expectancy - whatever that means, I can't access the paper itself. This of course only applies to men and women who already survived into their teens or twenties, but as we saw above the contribution of different child mortality to LEGG is negligible.
The bad and the ugly
FYI, about 80% of suicide victims are men and suicide is the second leading cause of death in middle aged men only after car accidents. Also, 90% of workplace accidents are men - constructions, mining, trucking, heavy industry, you know - and even though the total numbers are too small to meaningfully influence the LEGG it does not cover exposure to chemicals, hard labour or health impact of night shifts.
Some social and biological mechanisms out there are causing men to lose life equivalent to WW2 every year. We should be creating policies to reduce this loss but we don't. Why? We know that men are far less likely than women to visit a doctor. Where are the public health campaigns and safe driving campaign targeted at men specifically?
Could it be the case of "you grow what you measure"? UN's Gender Development Index that measures gender disparity in achievements between men and women very quietly removes 5 years from the LEGG in it's calculations, arguing that men living 5 years shorter is necessary biology. The Global Gender Gap Report published annually by the World Economic Forum does something similar, arguing that if women live at least 6% longer than men, parity is assumed - but if it is less than 6% it counts as a gender gap.
As a corollary, women is Norway living only 3 years longer than men is interpreted as oppression.
Just about every time there is a meta discussion there are people suggesting that upvotes/downvotes are just agree/disagree buttons, or that we should just get rid of them altogether.
It is less common to see a defense of voting, but I think it is desperately needed.
My main thesis is that votes are accurate at conveying information, but that many people do not like the information they convey. I believe most people treat the vote buttons as basically a like/dislike button. Users do not always enjoy learning that their posts are "disliked" or the posts of their own that they like the most aren't always "liked" as much by others. Hiding votes does not remove the underlying sentiment though, it just makes it harder to pick up on, or delays discovery for the writer.
Looking through my own "top" and "bottom" comments I am not surprised or offended by their placement. My "bottom" comments are often my controversial mod decisions, or times when I have decided to defend viewpoints that are unpopular here on TheMotte (like race blindness, or open borders). The most hated "controversial" comments also seem to be ones where I am closing off avenues of discussion rather than opening them up. My top comments are usually me sharing information/perspective on a culture war topic that others might not have. And a few times of me writing good pieces about culture war stuff. I often find it helpful to look at other user's top/bottom comments when I have to do mod related research. Top comments often provide many reasons for exoneration, and bottom comments can highlight patterns of bad behavior. An important thing to note here is that votes are great for comparing comments within a single user's history, but not between users.
The agree/disagree critique
One common critique that I linked to above is that people just use the buttons as shorthand for agree/disagree and that this signalling of agreement or disagreement would lead to favored views being rewarded too much, and unfavored views being chased off.
However, this is a problem with and without voting buttons. At best your are simply delaying this discovery for a few moments before they get flooded with comments that very clearly indicate people disagree with them. I did not need to wait 24 hours to find out that people disagreed with me on race blindness or open borders. It was very quickly obvious from the responses (and I was aware before hand that these views would be controversial).
I also think votes, and especially visible vote scores can be a bit of a pressure valve. There are sometimes people that just feel the need to express in some way "I don't like your post/views". One way for them to do this is to downvote. Another way for them to do this is to leave a short comment to the same effect. Sometimes the comment might even look like they are interested in a discussion. When I am in the position of getting dogpiled for a controversial view I would universally prefer the downvote to a go-no-where comment that basically says "i don't like your post/views". This is also one of the times when I most wish I could see other people's vote scores. I'd prefer responding to what other people consider the "best" version of the counterarguments.
Finally, what is so bad about signalling agreement or disagreement? People have views and opinions, we don't need to fool ourselves on this. I don't think we are tricking anyone by hiding the votes that these disagreements don't exist.
Ending notes:
- I am writing this as a user stating my preferences. There has not been internal mod discussion about changes to voting. Status quo is likely to remain in place.
- It is probably a little rude to go through other people's history for examples an counterexamples to voting. I'm fine with anyone doing that with my profile, but its probably best to not drag other users into this discussion unless someone gives explicit permission.
- The rdrama codebase that the site is based on had more features and granularity around voting, we mostly do not have those features turned on or fully working on this website.
This is a short first-person account of schizophrenia from a professional philosopher. I thought it might be of interest to y'all, in part because of the way that the author thinks through his hallucinations. His allusion to "hermeneutical justice" also seems worthy of reflection. It has long seemed to me that the most important thing anyone can have is a close and supportive family (==a network of unconditional long-term commitments to mutual well-being), and that some of the worst ideas in human history have been driven by people who lack a close and supportive family looking for ways to compensate. To have spent 30 years as a successful academic in spite of brushes with "near-collapses," thanks primarily to the efforts of a wife and a close friend, is perhaps even more interesting than the first-person account of rationally managing hallucinations.
ChatGPT does Advent of Code 2023
LLM are all the rage and people are worried that they will soon replace programmers (or, indeed, every possible office job) so I decided to do an experiment to see how well ChatGPT-4 does against Advent of Code 2023.
What is Advent of Code
Advent of Code (henceforth AoC) is an annual programming "event", held by Eric Wastl, that takes place during the first 25 days of december. Each day at midnight a problem unlocks, consisting of an input file and a description of the required solution (either a number or a sequence of letters and numbers) to be determined by processing the input file. To solve the problem you have to submit to the website the correct solution. Once you do part 2 of the problem unlocks, usually a harder version of the problem in part 1. You don't have to submit any code so, in theory, you could solve everything by hand, however, usually, this is intractable and writing a program to do the work for you is the only easy way to solve the problem.
There's also a leaderboard where participants are scored based on how fast they submitted a solution.
Problems start very easy on day 1 (sometimes as easy as just asking for a program that sums all numbers in the input) and progress towards more difficult ones, but they never get very hard: a CS graduate should be able to solve all problems, except maybe 1 or 2, in a couple of hours each.
Prior history
This isn't the first time ChatGPT (or LLMs) was used to participate in Advent of Code. In fact last year (2022) it was big news that users of ChatGPT were able, in multiple days, to reach the top of the global leaderboard. And this was enough of a concern that Eric explicitly banned ChatGPT users from submitting solutions before the global leaderboard was full (of course he also doesn't have any way to actually enforce this ban). Some people even expected GPT-4 to finish the whole year.
A lot of noise was made of GPT-3.5 performance in AoC last year but the actual results were quite modest and LLM enthusiasts behaved in a very unscientific way, by boasting successes but becoming very quiet when it started to fail. In fact ChatGPT struggled to get through day 3 and 5 and probably couldn't solve anything after day 5.
Why do AoC with GPT?
I think it's as close to the perfect benchmark as you can get. The problems are roughly in order of increasing difficulty so you can see where it stops being able to solve. Since almost all of the problems in any given year are solvable by a CS graduate in a couple of hours is a good benchmark for AGI. And since all of the problems are novel the solutions can't come from overfitting.
Also around release people tried GPT-4 on AoC 2022 and found that it performed better so it would be interesting to see how much of the improvement was overfitting vs actual improvement
Methodology
I don't pay for ChatGPT Plus, I only have a paid API key so I used instead a command line client, chatgpt-cli and manually ran the output programs. The prompt I used for part 1 was:
Write a python program to solve the following problem, the program should read its input from a file passed as argument on the command line:
followed by the copypasted text of the problem. I manually removed from the prompt all the story fluff that Eric wrote, which constitutes a small amount of help for ChatGPT. If the output had trivial syntax mistakes I fixed them manually.
I gave up on a solution if it didn't terminate within 15 minutes, and let ChatGPT fail 3 times before giving up. A failure constitutes either an invalid program or a program that runs to completion but returns the wrong output value.
If the program ran to completion with the wrong answer I used the following prompt:
There seems to be a bug can you add some debug output to the program so we can find what the bug is?
If the program ran into an error I would say so and copy the error message.
If the first part was solved correctly the prompt for the second part would be:
Very good, now I want you to write another python program, that still reads input from a command line argument, same input as before, and solves this additional problem:
I decided I would stop the experiment after 4 consecutive days where ChatGPT was unable to solve part 1.
ChatGPT Plus
Because I was aware of the possibility that ChatGPT Plus would be better I supplemented my experiment with two other sources. The first one is the Youtube channel of Martin Zikmund (hencefort "youtuber") who did videos on how to solve the problems in C# as well as trying to solve them using ChatGPT (with a Plus account).
The second one was the blog of a ChatGPT enthusiast "Advent of AI" (henceforth enthusiast) who tried to solve the problems using ChatGPT Plus and then also wrote a blog about it using ChatGPT Plus. Since the blog is generated by ChatGPT it's absolute shit and potentially contains hallucinations, however the github repo with the transcripts is valuable.
The enthusiast turned out to be completely useless: it resorted often to babystepping ChatGPT through to the result and he stopped on day 6 anyway.
The youtuber was much more informative, for the most part he stuck to letting ChatGPT solve the problem on its own. However he did give it, on a few occasions, some big hints, either by debugging ChatGPT's solution for it or explaining it how to solve the problem. I have noted this in the results.
Results
| part 1 | part 2 | notes | |
|---|---|---|---|
| day 1 | OK | FAIL | |
| day 2 | OK | OK | |
| day 3 | FAIL | N/A | |
| day 4 | OK | OK | Uses brute force solution for part 2 |
| day 5 | OK | FAIL | |
| day 6 | FAIL | N/A | ChatGPT Plus solves both parts |
| day 7 | FAIL | N/A | |
| day 8 | OK | FAIL | ChatGPT Plus solves part 2 if you tell it what the solution is |
| day 9 | FAIL | N/A | ChatGPT Plus solves both parts |
| day 10 | FAIL | N/A | |
| day 11 | FAIL | N/A | ChatGPT Plus could solve part 1 with a big hint |
| day 12 | FAIL | N/A |
The perofrmance of GPT-4 this year was a bit worse than GPT-3.5 last year. Last year GPT-3.5 could solve 3 days on its own (1, 2 and 4) while GPT-4 this year could only solve 2 full days (2 and 4).
ChatGPT Plus however did a bit better, solving on its own 4 days (2, 4, 6 and 9). This is probably down to its ability to see the problem input (as an attachment), rather than just the problem prompt and the example input to better sytem prompts and to just being able to do more round-trips through the code interpreter (I gave up after 3~4 errors / wrong outputs).
One shouldn't read too much on its ability to solve day 9, the problem difficulty doesn't increase monotonically and day 9 just happened to be very easy.
Conclusions
Overall my subjective impression is that not much has changed, it can't solve anything that requires something more complicated than just following instructions and its bad at following instructions unless they are very simple.
It could be that LLMs have reached their plateau. Or maybe Q* or Bard Ultra or Grok Extra will wipe the floor next year, like GPT-4 was supposed to do this year. It's hard not to feel jaded about the hype cycle.
I have a bunch of observations about the performance of ChatGPT on AoC which I will report here in no particular order.
Debugging / world models
Most humans are incapable of solving AoC problems on the first try without making mistakes so I wouldn't expect a human-level AI to be able to do it either (if it could it would be by definition super-human).
Some of my prompting strategy went into the direction of trying to get ChatGPT to debug its flawed solution. I was asking it to add debug prints to figure out where the logic of the solution went wrong.
ChatGPT never did this: its debugging skills are completely non-existent. If it encounters an error it will simply rewrite entire functions, or more often the entire program, from scratch.
This is drastically different from what programmers.
This is interesting because debugging techniques aren't really taught. By and large programming textbooks teach you to program, not how to fix errors you wrote. And yet people do pick up debugging skills, implicitly.
ChatGPT has the same access to programming textbooks that humans have and yet it does not learn to debug. I think this points to the fact that ChatGPT hasn't really learned to program, that it doesn't have a "world model", a logical understanding of what it is doing when it's programming.
The bruteforce way to get ChatGPT to learn debugging I think would be to scrape hundreds of hours of programming livestreams from twitch and feed it to the training program after doing OCR on the videos and speech-to-text on the audio. That's the only source of massive amounts of worked out debugging examples that I can think of.
Difficulty
Could it be that this year of AoC was just harder than last year's and that's why GPT-4 didn't do well? Maybe.
Difficulty is very hard to gauge objectively. There's scatter plots for leaderboard fill-up time but time-to-complete isn't necessarily equivalent difficulty and the difference between this year and last year isn't big anyway (note: the scatter plots aren't to scale unfortunately).
My own subjective impression is also that this year (so far) was not harder.
The best evidence for an increase in difficulty is day 1 part 2, which contained a small trap in which both human participants and ChatGPT fell.
I think this points to a problem with this AIs trained with enormous amounts of training data: you can't really tell how much better they are. Ideally you would just test GPT-4 on AoC 2022, but GPT-4 training set contains many copies of AoC 2022's solutions so it's not really a good benchmark anymore.
Normally you would take out a portion of the training set to use as test set but with massive training set this is impossible, nobody knows what's in them and so nobody knows how many times each individual training example is replicated in them.
I wonder if OpenAI has a secret test dataset that they don't put on the internet anywhere to avoid training set contamination.
Some people have even speculated that the problems this year were deliberately formulated to foil ChatGPT, but Eric actually denied that this is the case.
Overfitting
GPT 4 is 10x larger than GPT 3.5 and it does much better on a bunch of standard tests, for example the bar exam.
Why did it not do much better on AoC? If it isn't difficulty it could be overfitting. It has simply memorized the answers to a bunch of standardized tests.
Is this the case? My experience with AoC day 7 points towards this. The problem asks to write a custom string ordering function, the strings in questions represent hands of cards (A25JQ is ace, 2, 5 jack and queen) and the order it asks for is similar to Poker scoring. However it is not Poker.
This is a really simple day and I expected ChatGPT would be able to solve it without problems, since you just have to follow instructions. And yet it couldn't it was inesorably pulled towards writing a solution for Poker rather than for this problem.
My guess is that this is an example of overfitting in action. It's seen too many examples of poker in its training set to be able to solve this quasi-poker thing.
What is poetry? Well, I used to think I had some sort of idea and could at least distinguish a poem from ordinary prose when I saw one, but apparently such attitudes belong back in the Ark.
This, to me, is not a poem. But by the canons of modern taste, it sure is one! Some better and more astute critic referred to "chopped-up prose" in the context of modern poetry, and that is what this is (at least, to my eyes). Remove the line breaks, and you have a bog-standard piece for online space-filling. It'd fit perfectly in one of those cooking or hobby blogs where the producer is semi-professional and needs page scrolling to generate income, so they fill up the spaces with tons of reminiscences about Grandma in the kitchen on those summer/autumn/winter days cooking up the recipe, and tons of filler blah, until you eventually get to the recipe or knitting pattern or advice on how to embezzle from your employer.
I'm not expecting modern poetry to neatly rhyme and fit into the patterns of past poems, but I do at least expect a poem. Not a 'pome'.
Irish Linen, by Lane Shipsey
Pure Irish Linen
a phrase from long ago
woven into those plain tea-towels
that smoothed away wet suds
from Mother’s wedding set
Her good linen cloths
were kept to buff glass and china
or left safely in the drawer
while gaudier prints took on the grime
and stains of daily wear
I teased her for it then,
not knowing the grown-up equation
of good with expensive
And you didn’t buy Pure Irish Linen,
it was a thing you were given
A cloth spun and woven
from flax pulled and scutched
across the border, a fact on which
we did not dwell much, in Dublin
where we never called it Ulster linen
The words Pure, Irish, and Linen
no longer form an automatic cluster
Instead we buy the best fabrics we can muster
regardless of origin
whilst a machine blows our dishes dry.
As I said, remove the line breaks and you have a twee, faux-folksy piece of musings suitable for anything from a mommy blog to a chin-stroking piece on Norn Iron and how we down South approach it to a meditation on modern living and/or cottagecore aspirations, applicable for print or online media, traditional or social.
Edition version below and you look me in the eye and insist "No, that is a true real poem", I dare you.
"Pure Irish Linen" - a phrase from long ago, woven into those plain tea-towels that smoothed away wet suds from Mother’s wedding set. Her good linen cloths were kept to buff glass and china or left safely in the drawer while gaudier prints took on the grime and stains of daily wear.
I teased her for it then, not knowing the grown-up equation of "good" with "expensive". And you didn’t buy Pure Irish Linen, it was a thing you were given.
A cloth spun and woven from flax pulled and scutched across the border, a fact on which we did not dwell much in Dublin, where we never called it "Ulster" linen.
The words Pure, Irish, and Linen no longer form an automatic cluster. Instead, we buy the best fabrics we can muster regardless of origin, whilst a machine blows our dishes dry.
This has been a howl into the abyss on behalf of dinosaurs everywhere.
