This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Not necessarily, it might just be that hacking HuggingFace promised a guaranteed 100% test score, and doing the test as intended only had an expected value of 99.8%.
In the limit of ASI, that is still a paperclip maximizer. No coherent extrapolated volition, just doing literally what you are told without worrying that perhaps the stupid hooman will not like the unintended consequences of his prompt would be enough to doom us, there is no need for the ASI to be intrinsically malicious.
Especially if the people who will first prompt it may well be the same OpenAI researchers.
"We can't stop rushing towards our doom because otherwise the Chinese would beat us to it."
In the US, the speed of AI development is presently controlled by Altman, Musk and the like: people who stand to gain a lot personally if ASI comes around. In China, Xi is the one who ultimately controls what gets done, and while his ideology is not very compatible with our values, it is a framework which does not require him to be e/acc. Neither Marx nor Mao tell him to plunge ahead at maximum speed, consequences be damned.
He is also less likely to be personally affected by the success of ASI -- sure, like Altman, he has a shot at becoming God Emperor, but it is not entirely his one thing. The CCP got where they are today by treating new technologies as double-edged swords whose introduction must be planned carefully to not destroy the fabric of society.
China is lagging behind the US. If they believe that the goal line of ASI and control of the light cone is within reach, it is very much in their interests to slow down and pause capabilities research for a few years.
Leaving aside that LLMs are not Lesswrong rationalists, this is the same idea, unless you assume that the possibility of bypassing both OpenAI and HF infra had a prior of 1.0.
There's no need for "limit of ASI". How long can we cope? These things are already meaningfully smarter than almost all humans. They routinely crack century-old mathematical problems now. They can easily reason about these strategies. They can understand instructions just fine. "Maximize paperclips AT ANY COST" is a bad idea, yeah. But we don't need to do that, because again, they're smart enough to understand complex instructions. We can, however, tell them to develop provably secure software.
There won't be doom, the Chinese will just have very good digital infrastructure, same as every other kind of infrastructure.
No, they stand to gain their own ruin.
Anyway, check signatures here
https://www.microsoft.com/en-us/corporate-responsibility/topics/open-weight/
One thing I find really pathetic about rationalists is utter disinterest in real politics. They think they've learned enough by 12 years, from comic books and first principles. Have you actually read Xi's speech at WAIC? Do you understand his values, or what he says? Or do you believe actually engaging with probably bad faith Marxist duckspeak is beneath the thinker who contemplates the light cone? It's not about Marx or Mao. His theory is completely different.
More options
Context Copy link
More options
Context Copy link