This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Yet another Eliezer Yudkowsky podcast. This time with Dwarkesh Patel. This one is actually good though.
Listeners are presumed to be familiar with the basic concepts of AI risk, allowing much more in-depth discussion of the relevant issues. The general format is Patel presenting a series of reasons and arguments that AI might not destroy all value in the universe, and Yudkowsky ruthlessly destroying every single one. This goes on for four hours.
Patel is smart and familiar enough with the subject material to ask the interesting questions you want asked. Most of the major objections to the doom thesis are raised at some point, and only one or two survive with even the tiniest shred of plausibility left. Yudkowsky is smart but not particularly charismatic. I doubt that he would be able to defend a thesis this well if it were false.
It feels like the anti-doom position has been reduced to, “Arguments? You can prove anything with arguments. I’ll just stay right here and not blow myself up,” which is in fact a pretty decent argument. It's still hard to comprehend the massive hubris of researchers at the cutting-edge AI labs. I am concerned that correctly believing yourself capable of creating god is correlated with falsely believing yourself capable of controlling god.
I listened to that one, and I really think that Eliezer needs to develop a politician's ability to take an arbitrary question and turn it into an opportunity to talk about what he really wanted to talk about. He's taking these interviews as genuine conversations you'd have with an actual person, instead of having a plan about what things he wants to cover for the type of audience listening to him in that particular moment. While this conversation was better than the one with Lex, he still didn't lay out the AI safety argument, which is:
"Consistent Agents are Utilitarian + Orthogonality Thesis + Instrumental Convergence + Difficulty of Specifying Human Goals + Mesa-Optimizers Exist = DOOM"
He should be hitting those 5 points on every single podcast, because those are the actual load-bearing arguments that convince smart people, so far he's basically just repeating doom predictions and letting the interviewers ask whatever they like.
Incidentally while we're talking of AI, over the past week I finally found an argument (that I inferred myself from interactions with chatGPT, then later found Yann Lecun making a similar one) that convinced me that the entire class of auto-regressive LLMs like the GPT series are much less dangerous than I thought, and basically have a very slim chance of getting to true human-level. And I've been measurably happier since finding an actual technical argument for why we won't all die in the next 5 years.
I find it odd that Lecun writes:
Toxicity is not a stupid mistake, it is a style of communication. It is possible to convey accurate information about reality in a toxic way. It is also possible to convey total nonsense in a very polite and well-mannered way.
More options
Context Copy link
He doesn't seem to have an intuitive understanding of what the everyman on the street is missing. That's probably why he never starts with a ground-up outline of the whole problem. Its hard to know what fundamental assumptions your audience is missing before you get to Q&A.
I agree that we probably don't get doom from literally just stacking more layers. The concerning thing is that these seem like easier problems to solve than I would have expected getting computers to understand concepts would be. I still think we are at least one or two BIG breakthroughs from true AGI (the kind that makes humans obsolete). Those could come tomorrow, or they could come 10 years from now, or never.
This is the way I think about it, too. Despite popular fears, Deep Blue was never going to reach AGI no matter how much you refined its algorithm or how much computing power you gave it. Chess, like writing a five paragraph essay, was a test we used to measure the unobserved variable of intelligence, so people assumed that once a machine could play chess as well as humans (or research and write a five paragraph essay as well as humans), humans were presently donezo. But that unobserved variable is a tricky thing to define, and so, to engineer.
Every year offers a small chance that someone grasps the key insight that yields an AI capable of recursive self-improvement. But it's unpredictable when this insight will arrive. We don't know the shape of the thing we're fumbling for.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link