This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Scott Alexander, the guy who hosted the Culture War Thread on his blog slatestarcodex before that landed him in so much heat (from the woke left) that it was moved to reddit under the name /r/theMotte (and from there, subsequently, to themotte.org to preempt a reddit crackdown) comes usually across as a kind and considerate person to me. Charitable interpretations are very much his thing. While he has always engaged in CW topics on occasion, his forays deep into the trenches are rare (and in my impression have become rarer compared to the SSC days).
Apparently, there is a person called Steven Pinker. I have been vaguely aware that there was a book called The Better Angels of Our Nature which was probably authored by someone, but he was not someone I really had on my radar.
From the WP article, he seems solidly Grey Tribe. Believes in a computational theory of mind, Ashkenazi Intelligence Hypothesis, critical of both the woke left (cancellations) and the MAGA right (foreign student restrictions).
The main topic where he disagrees with Scott Alexander is AI x-risk.
Scott has now responded, and his reponse is a bit less than maximum charitable.
While Scott is diligent enough with his technical points, it is clear that he is personally hurt by Pinker's decision to ridicule AI x-risk. So his post is also a broadside fired against him. He cites Garfinkel et al 2017 who "proved" that machines will never be larger than humans using common philosophical arguments for AI being harmless.
Scott points out that while Pinker is busy berating the LW crowd for distracting from the more mundane risks of AI, x-risk advocates are willing to work hand in hand with groups with a more practical focus to combat pro-AI PACs like Leading the Future.
On model welfare (which is certainly not the core topic of the x-risk advocates):
He accuses Pinker of endorsing the "lowest quality voices", quoting Claire Lehmann:
Or
It ends with Scott seeming seriously willing to duel Pinker:
Scott has admitted in the comments that this is already the toned-down version of the post he originally wrote. I think that while he held back verbally (nobody get's called a "Vogon spy in a skin suit" this time), he uses his craft well to show his frustration.
For what it's worth, I think that Pinker is correct that Eliezer is in fact over-confident in p(doom|default-ASI). Just like Pinker is when he declares that any fear of doom is ridiculous.
The main difference is that for practical purposes, it does not matter much if your p(doom) is 0.99 or 0.05. Both would be pressing problems whose solution is the most important thing in the world.
Eliezer was bullish on AI capabilities when that was far out of the Overton window, back when the machines could beat humans at chess at most. Steven Pinker was tweeting that "deep learning has probably peaked" in 2019.
This might be giving too much credit to Pinker, but here is Clarke's first law:
If my p(doom) was 99% over the near-term, I'd consider a heroine habit, and blowing my money on hookers and blow. A lot of people would become outright fatalistic, and I wouldn't blame them. Eliezer has sometimes said he's about there, and has fatalistic tendencies on occasion, but to his credit he's far from quit.
Instead, I'm a comparably sedate 20%. Can't do much about things, but hey, it could work out! I could also die, but oh well. No point in sitting around crying about it, the sun shines all the same.
Low income people saving money must look like idiots to a guy like Yud making 600k a year (?) and thinking it's all going to hell soon
More options
Context Copy link
More options
Context Copy link
I disagree with this. For example, suppose (1) p(doom) is 0.05; (2) p(some totalitarian dictatorship wins the AI race and becomes permanent world ruler) is .25; (3) with a lot of work, it may be possible to reduce p(doom) from 0.05 to 0.04; and (4) there is a good chance of reducing the 0.25 figure down to 0.01.
In that case, it's very reasonable to see the second issue as more pressing and important.
Although obviously these numbers are cooked up, I do believe the following:
There are really two alignment problems. This first is in getting AI to do what its human masters want; the second is getting those human masters to act in the best interests of humanity.
The first problem, although serious, is one which is very difficult to assess; it's difficult to know what should be done to improve the situation; and it's difficult to know how much of an effect any measure might have.
The second problem is arguably at least as serious a risk and in any event is more tractable.
Proposed solutions for the first issue might very well make things worse as to the second. For example Yudkowsy's idea of a worldwide ban on data centers.
I'm not saying that we should ignore the first problem, just that we should look at in context. And probably pay more attention to the second.
More options
Context Copy link
I don’t know, it just feels kind of like we’re in a car and we’re heading for an opening
drawbridgebascule bridge and half the people are cheering and expect we’ll make it and half the people are terrified we’ll fall and are begging the driver to slow down, except that it’s too late to slow down and even if we wanted to the driver doesn’t.We are a species of desperate risk takers and gamblers, not all of us but enough of us to decide our destiny. The exact same thing that led our ancestors to cross continents on foot over generations, to walk across the world, to set out into vast oceans on flimsy rafts and find new lands, that leads people today to put it all on red or gamble their college fund on stratospherically far out of the money semiconductor options, to cheat on exams and to jump off skyscrapers with parachutes - this is why AI is unstoppable, and this is why Yud and Scott and anyone crying about p(doom) will fail.
Not because they are wrong, but because to act on their warnings would go against our very nature. We cannot do it. It is ‘full steam ahead and hope it works out fine’; we have boarded the raft.
There's also what Scott called Moloch and I sometimes call the Darwinian principle. Basically, "we have to build it because if we don't, the other people will and then they'll conquer us".
As for risk-taking, well... if I was purely selfish, I think I might be fine going for a 50-50 risk of either AI-driven human extinction or Star Trek utopia. Why? Because I don't have kids, am not sure I ever will, and to me the chance of benevolent AI giving me a significantly increased lifespan and reducing aging problems seems really tempting even if the odds are low, since the alternative is accepting a normal human lifespan and normal human aging. If we stop AI progress, it means that AI will not destroy humanity, but I'm still going to die within a few decades so for me it's almost as if the world will end within a few decades anyway, unless there is an afterlife.
I am not purely selfish, but some important people in the AI space are probably closer to being purely selfish than I am. Which is not to say that longevity is necessarily a significant factor in their calculations. At least, they don't seem to talk about personal longevity much.
More options
Context Copy link
More options
Context Copy link
Frankly I never thought Scott was that impressive a thinker. His writing is just fine. Fifteen-twenty years ago there were a lot of bloggers in Scott's milieu; he would not have become important if he had been writing ten years earlier. His chief value, in fact, was that he attracted a lot of debate from different tribes when the internet was polarizing and becoming closed. In the 2000s there was no shortage of forums where lefties and righties could try to debate; by the 2010s these were all closing down. (I'll add I think gay marriage was actually more important than people credit: post-Obergefell a lot of liberals decided that conservative opinions they had previously countenanced were actually too bigoted to tolerate at all.)
Scott got a massive second-wind by being doxxed extremely uncharitably by the New York Times. I don't think, absent that, he would have had the courage to deliberately become a political lightning rod. There are only so many people who would deliberately choose to be controversial and famous, and Scott doesn't have the personality for it. He probably would have gradually diminished his online personality until he could fade into obscurity. (For contrast, Curtis Yarvin tried this same retirement and was not allowed obscurity by Bay-area radicals. The Last Psychiatrist, who was also doxxed, never polarized into a political partisan figure, and he has basically been allowed to become obscure.)
Scott's diatribe against Pinker (who, let me say sit, is also pretty mediocre and often fairly specious) is about what I'd suspect. There's a certain type of moralizing I find certain personality types drawn to: suppressed righteousness. Scott has never dipped his hand in the cookie jar, so now that he can he pigs out on chocolate chip. (And, again, in fairness to Scott, he has been fairly charitable over many years dealing with all sorts of uncharitable critics.) Imagine what it's like to suppress your most combative tempers and finally have an excuse to indulge them. You just have to believe that AI is an existential risk and now that's more important than charity.
(There's a really big difference between peace achieved by suppressing your violent desires and inner peace with no violent desires to suppress at all. I don't want to make that point too harshly about Scott, it's just something I've been thinking about while writing this all out.)
More options
Context Copy link
This was probably the best post of Scott's that I've read so far this year. I don't know if that's testament to how good this post was, or how far he's fallen since his peak. I also don't even bother reading many of his new posts anymore. I'm also not a paid subscriber, so maybe the really good stuff is hidden behind the paywall.
I agree with Scott that many of Pinker's arguments as to why superintelligence will never happen seem rather facile, and often exhibit isolated demands for rigour.
That's funny you think that, because to me this is one of the worst posts of Scott's I've ever read, and emblematic of his general slide in quality. I guess the prose is good, no one would deny he's a talented writer, but the curious and empathetic blogger I fell in love with reading 10 years ago is nowhere in sight. He's been replaced with an AI activist determined to burn through a decade-plus of good will in the defense of an apocalyptic sex cult.
More options
Context Copy link
Could just be the target. His attitude towards Pinker reminds me of his attitude towards 2010s-era leftism, a complicated mix of "You were the chosen one!" and "You were my brother ... I loved you" with "you have become the very thing you swore to destroy" and "I will do what I must". That passionate mix of love and hate even elicited tolerable prose from George Lucas; of course it's going to be magic in the hands of a good writer!
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link