site banner
PaperclipPerfector  5d ago (text post)   1684 thread views

Small-Scale Question Sunday for January 4, 2026

Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?

This is your opportunity to ask questions. No question too simple or too silly.

Culture war topics are accepted, and proposals for a better intro post are appreciated.

1
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

What does everyone think of Eliezer Yudkowsky?

I just realized this website was born from /r/TheMotte, which came from /r/slatestarcodex, which came from Scott Alexander, who came from LessWrong, which was created by Eliezer himself. He's technically a grandfather of this community.

I for one think he's more influential than he's given credit for, and I consider myself lucky to have come across his writings in my younger days.

I'd consider him a reasonably smart person and a great analytic writer (a lot of the Sequences really do what they set out to, which is to crystallize a particular useful reasoning principle into a well-delineated maxim that you can actually remind yourself of in a situation where it is helpful), unfortunately held back by high levels of narcissism, which firstly make him fail to apply the same critical reasoning he champions to core areas that pertain to his conception of his own status and secondly turn him, on a personal level, into a snake.

He may or may not be the "greatest living philosopher", as @Quantumfreakonomics said below, but this is strictly an indictment of living philosophers who are willing to own the label. I have heard better philosophy from my math olympiad buddies at age 16 during addled seminar camp all-nighters.

Timed the AI & EA stuff well but also pretty classic autist who perennially nerdsnipes himself. The first 10 or so chapters of MOR are great when it's more of a collection of skits than a clear narrative, too. Silicon Valley Nerd Culture just seems primed to randomly pump guys like him every so often who have the right idea at about the right time, but I don't think he's some great thinker and a lot of the AI alarmism hasn't aged well.

I'm not getting what the hype is about. Yes, he's (very) smart and (very) talented. There are a few smart people and a few talented people, he is one of them. But various superlatives directed at him is something that I am confused by. Then again, I am confused by great many things, so nothing really special here.

The hype mostly comes from his cultists. For reasons that are unclear to me because I wasn't there, he managed to get a large number of the readers of his Harry Potter fanfic to move to Berkeley and join his weird sex cult, which described itself as a "rationalist community" and devoted to "systematised winning" but was significantly less rational and winning than a randomly selected group of Greater SF techies. A few smart people who were attracted by his rationalist blogging were also involved.

To the extent that he matters, it is because he was the tech-elite certified wunderkind who had been banging on about then-hypothetical AI safety issues for years at the point where they suddenly became relevant, so when other tech elites suddenly realised that there were non-hypothetical AI safety issues they needed to start worrying about, some of them treated him as an expert.

[context and geneology]

He's... hard to talk about.

The critique has long echoed the old Samuel Johnson quote about being "both good and original; but the part that is good is not original, and the part that is original is not good" -- and the man has had a hatedom before 2012, so it's been echoing for a while. Most of the man's more accessible scientific writing is 'just' presenting well-established popsci into a more accessible form (sometimes without sufficient citations), while a lot of his predictive or even cutting-edge scientific analysis has to get suffixed with 'from a certain point of view' at best and 'ah yes but' at worst. If anything, that's only become more true over time: both The Golden Age Sequences and HPMoR have long relied on some of the sociology research that's be found the most wanting under the replication crisis.

Yudkowsky's been moderately open about some of this stuff, and his pro-AI, AI-is-easy, AI-is-hard, anti-AI changes have been a part of his whole story. I like that more than the people insisting they've always been right. It's still not something everyone likes, or that he can do consistently. There's never been a good retrospective on how MIRI's output was so absolutely bad on both the academic paper and popular-reader sides for so long, or the time they had an employee embezzle (tbf, not an unusual thing for new non-profits to have hit them), or yada yada.

But that's a bit of a victim of own success thing. Yudkowsky can't claim the whole replication movement anymore than he can claim the whole effective altruism one. He's at least been in the general vicinity too early to have jumped in front of the parade post-hoc, though. "Map is not the territory" and "fake answers" might have been well-known and obvious before 2008, but it wasn't until after that anyone put them together to actually poke at the core tools we thought we were using to find deep truths about reality. And these movements have been a large part of why so many of the older posts have aged so poorly, though not the only part.

((Although he's also a weird writer to have as big an impact as it seems he's had? The Sequences, fine, if good blog should change people's minds, it's a good enough blog. Why is HpMoR a more effective AI Safety program than Friendship is Optimal? Why is the Sword of Good so much more effective than a lot of more recent attempts at its take?))

... but all that's kinda side stories, at this point. Today, if you care about him, it's the AI safety stuff, not whether he guessed correctly on Kahneman vs Elisabeth Bik, or even on neural networks versus agentic AI research.

Which gets messy, because like Reading Philosophy Backwards, today, all of his demonstrated successful predictions are incredibly obvious, his failed ones ludicrous-sounding, and only the ones we can't evaluate yet relevant. Why would anyone care about the AI Box experiment when corporations or even complete randos are giving LLMs a credit card and saying have fun? (Because some extremely well-credentialed people were sure that these sort of AI would be perfectly harmless if not allowed access to the outside world, even months after the LLMs were given credit card info.) Why would anyone be surprised that an AI might disclose private or dangerous information, if not told otherwise, when we now know LLMs can and do readily do those things? (Because 'the machine will only do what we program it to do' was a serious claim for over a decade.) Who could possibly believe that an LLM couldn't improve code performance? (Uh, except all the people talking about stochaistic parrots today, and convinced that it was philosophically impossible for years before then.)

And the big unsolved questions are very important.

But in turn, that doesn't make his proposed answers better or useful. Say what you will for the ethos of singularitarity races, but at least they have something more credible than the 'you can't just tell people not to do something' guy telling people not to do something, and ultimately that's all that policies like an AI pause boil down to. The various attempts to solve morality have made some progress, despite my own expectations. It might seem like the difference between timeless decision theory and functional decision theory is just huffing fumes, but it does have some genuine benefits... and we have no way to implement them, and no way to validate or even seriously consider whether we're even looking at the most important measures. We don't know what the system they'd need to be implemented on looks like, and it's speculative (though increasingly likely) there will even be a system, and it's not clear the people building that system will be interested or even aware of the general AI safety issues.

So there's big unsolved questions that have been largely left unasked.

But in turn, that doesn't make his proposed answers better or useful.

but it does have some genuine benefits... and we have no way to implement them, and no way to validate or even seriously consider whether we're even looking at the most important measures.

Note that these are useful if you share the Yudkowskian view of neural nets. Specifically, the view that it is impossible to align a neural net smarter than you; "a technique, inventable before the Singularity, that will allow us to make neural-net ASI and not die" is a contradiction in terms. There are thus no "useful" answers, if you define "useful" as "works on neural nets".

In this paradigm, 100% of surviving worlds follow this two-point plan:

  1. Neural nets are totally and permanently abandoned until after the Singularity; they are banned in all countries (convincing everyone is hard; easier is convincing enough nuclear powers, hard enough, that the holdout countries are either occupied or obliterated).

  2. Non-doomed versions of AI research (e.g. GOFAI, uploads) continue.

The reason you need #1 is that #2 is going to take at least 50 years to hit the Singularity. The reason you need #2 is that #1 is only metastable, not actually stable; sooner or later, in a hundred years or a million, the Butlerian Jihad will break down, at which point everybody dies unless we've hit the Singularity in the meantime.

And hence, work on how to make non-neural-net AI work is necessary (if less urgent than stopping neural nets, on which point Yudkowsky is indeed currently focusing).

He is Earth's greatest living philosopher.

Edit: I challenge any of the downboaters to name a better one.

Edit: I challenge any of the downboaters to name a better one.

I don't even think that Yudkowsky was the best thinker on LessWrong. Both David Friedman and Scott Alexander (when he was on) surpass him easily IMO.

Heck I can also think of a handful of regular commentors from those days that I thought were at least in his league if not the same ball park.

I don't even think that Yudkowsky was the best thinker on LessWrong. Both David Friedman and Scott Alexander (when he was on) surpass him easily IMO.

This is trivia, not science, but for kicks I decided to see how many LessWrong quotes from each user I've found worth saving over the years: Yudkowsky wins with 18 (plus probably a couple more; I didn't bother making the bash one-liner here robust), Yvain (Scott) takes second with 10, and while I have dozens of Friedman quotes from his books and from other websites, I can't find a one from LessWrong that I saved. (was Friedman was just a lurker on LessWrong?)

On the other hand, surely "best" shouldn't just mean "most prolific", even after a (grossly-stochastic) filter for the top zero-point-whatever percent. Scott is a more careful thinker, and David more careful still, and prudence ought to count for something too ... especially by Yudkowsky's own lights! We praise Newton for calculus and physics and downplay the alchemy and the Bible Code stuff, but at worst Newton's mistakes were merely silly, just wastes of his time. Eliezer Yudkowsky's most important belief is his conclusion that human extinction is an extremely likely consequence of the direction of progress currently being pursued by modern AI researchers, who frequently describe themselves as having been inspired by the writings of: Eliezer Yudkowsky. I'm not sure how that could have been avoided, since the proposition of existential AGI risks has the proposition of transformative AGI capabilities as a prerequisite and there were naturally going to be people who took the latter more seriously than the former, but it still looks superficially like it could be the Ultimate Self-defeat in human history, in both senses of that adjective.

While I agree that EY is a better philosopher than the vast majority of people currently teaching in university philosophy departments, a rock with "Touch grass daily, call your mother weekly" written on it would be even better.

Analytic Philosophy as a discipline is the discipline of thinking deeply about things we don't understand well enough to have an actual discipline to think about in an informed way. "Natural philosophy" has been replaced by physics. "Philosophy of mind" should have been replaced by psychology and neuroscience. "Moral philosophy" would have been replaced by sociology and social anthropology if those disciplines functioned properly. Continental Philosophy is what bullshit looks like if you try to make it look like it was translated from French badly. Both are by definition unlikely to produce actionable insights.

He's brilliant, but also incredibly goofy.

What does everyone think of Eliezer Yudkowsky?

Answering this question would break the rules on fedposting.

I can't possibly imagine what this guy has done to make you say this

He advocates for killing me, as a part of killing an unbounded amount of people, for the goal of preventing an unapproved by him AGI. Therefore...

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats