This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Scott Alexander, the guy who hosted the Culture War Thread on his blog slatestarcodex before that landed him in so much heat (from the woke left) that it was moved to reddit under the name /r/theMotte (and from there, subsequently, to themotte.org to preempt a reddit crackdown) comes usually across as a kind and considerate person to me. Charitable interpretations are very much his thing. While he has always engaged in CW topics on occasion, his forays deep into the trenches are rare (and in my impression have become rarer compared to the SSC days).
Apparently, there is a person called Steven Pinker. I have been vaguely aware that there was a book called The Better Angels of Our Nature which was probably authored by someone, but he was not someone I really had on my radar.
From the WP article, he seems solidly Grey Tribe. Believes in a computational theory of mind, Ashkenazi Intelligence Hypothesis, critical of both the woke left (cancellations) and the MAGA right (foreign student restrictions).
The main topic where he disagrees with Scott Alexander is AI x-risk.
Scott has now responded, and his reponse is a bit less than maximum charitable.
While Scott is diligent enough with his technical points, it is clear that he is personally hurt by Pinker's decision to ridicule AI x-risk. So his post is also a broadside fired against him. He cites Garfinkel et al 2017 who "proved" that machines will never be larger than humans using common philosophical arguments for AI being harmless.
Scott points out that while Pinker is busy berating the LW crowd for distracting from the more mundane risks of AI, x-risk advocates are willing to work hand in hand with groups with a more practical focus to combat pro-AI PACs like Leading the Future.
On model welfare (which is certainly not the core topic of the x-risk advocates):
He accuses Pinker of endorsing the "lowest quality voices", quoting Claire Lehmann:
Or
It ends with Scott seeming seriously willing to duel Pinker:
Scott has admitted in the comments that this is already the toned-down version of the post he originally wrote. I think that while he held back verbally (nobody get's called a "Vogon spy in a skin suit" this time), he uses his craft well to show his frustration.
For what it's worth, I think that Pinker is correct that Eliezer is in fact over-confident in p(doom|default-ASI). Just like Pinker is when he declares that any fear of doom is ridiculous.
The main difference is that for practical purposes, it does not matter much if your p(doom) is 0.99 or 0.05. Both would be pressing problems whose solution is the most important thing in the world.
Eliezer was bullish on AI capabilities when that was far out of the Overton window, back when the machines could beat humans at chess at most. Steven Pinker was tweeting that "deep learning has probably peaked" in 2019.
This might be giving too much credit to Pinker, but here is Clarke's first law:
I'd say that I lost some respect for both of them through this argument, but to be honest I don't think I ever had much respect for Pinker. The ignorant and strawmannish Enlightenment Now really ended any sympathy I had for him as a thinker. Nonetheless, Pinker's letter to Scott was courteous and concise; I think you can tell that Pinker is a better or at least more professional writer.
Scott's response... to be honest, it reminded me of point six here. The world Scott is in is small and homogenous - there's an awful lot of citing Omohundro and Hinton, and after a while one starts to think that that's because he's only got a handful of people, and is shocked and appalled that anybody disagrees with them. Scott does land some real hits on Pinker - the RAND study mistake is an embarrassing mis-step by Pinker - but also avoids some of Pinker's stronger arguments, and sets up his own straw men.
I'm not particularly interested in nitpicking many of the specific disagreements, especially because plenty of them leave ample room for both to be, in a sense, right. Pinker is correct that 'intelligence' can't be modelled as a single variable; Scott is correct that nonetheless it is reasonable to talk about 'intelligence' in a general sense.
What stood out to me, personally, was the way that the mask came down a bit, and setting aside all disputes about facts, Scott is dedicated to normative commitments that I think are repulsive, or frankly insane. Consider:
I'm sorry, but this is lunatic. We need to invest in language models or human civilisation will be permanently stunted? What? He's got some sort of science fiction eschatology in mind that seems both absurd on the factual merits and, perhaps more importantly, morally blinkered. While I did always know that Scott had some of those bizarre transhumanist beliefs (and I'm certainly not a convert to Phil-Torres-like conspiracism), it is, at least, a useful reminder that he approaches the whole issue of technology from a bizarre starting point that I cannot sympathise with.
There are surely some technologies or techniques that human minds cannot comprehend but AI can.
No realistic number of people doing calculations by hand could match a modern CPU. There could be no online banking, CGI rendering, internet without computers.
The same principle applies for AI. We are already seeing this in mathematics, human efforts are increasingly being left in the dust. It follows that restricting AI forever will result in us not attaining the most sophisticated technologies that require immense feats of comprehension to discover and deploy.
There's nothing bizarre about this either, most people seem to agree that it's possible to make AIs that are qualitatively superior to human intelligence, certainly most AI scientists.
There's a missed third option of "have a singularity, but not with neural nets". GOFAI, uploads, and real-human intelligence enhancement are all options there.
Scott said "smarter-than-human AI", which does include GOFAI, but in practice GOFAI would likely not be hit and could plausibly be excepted later even if it were.
Good point, I think it'd be certainly wiser to focus on uploads and intelligence augmentation. But also we suck at this, for the last 80 years eugenics has been a no-no word, authorities aggressively misunderstand what intelligence even is or its biological foundation. The current version of 'real human intelligence enhancement' we do is just sending more people to university, which doesn't work now and isn't going to start working anytime soon despite costing trillions. We are so far away from real intelligence enhancement it's surreal.
Also GOFAI seems to be in 'sounds good, doesn't work' territory, you say it's out in the neverlands of 30-300 years like how AGI used to be before transformers. Per my understanding, it's also a literal stochastic parrot in that it can only do what you program it to do, only obey rules that you install into it, or just clobber problems with search. Unless search techniques become super general-purpose, GOFAI cannot lead to AGI or ASI? And if it were somehow possible to use search so powerfully, that would probably give us a classical-Yudkowsky AI monster that obeys instructions super literally.
Meanwhile LLM intelligence is demonstrably working. Halting work on LLMs in their moment of triumph and switching to human brain emulation, uploads, human augmentation seems extremely difficult, nigh-impossible even.
It seems we shall have to pay the price for decades and decades of neglect.
I never said GOFAI is risk-free; it's not. It's just not only-Davros-would-do-this-deliberately lunacy like neural nets.
Regarding capability to hit AGI: it's certainly tricky, but it seems possible for a team of humans working over a long period to build something that none of them can fully simulate in his head, so I don't think it hits impossibility.
Normal people don't like AI. Really don't like it. That helps a lot.
Also, I would keep in mind that the geopolitical calculus could change rather drastically. Xi clearly wants a Taiwan invasion button for next year; who knows if he'll press it. A nuclear exchange would knock out the US/Chinese power grids and cripple China for the next century.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link