This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Scott Alexander, the guy who hosted the Culture War Thread on his blog slatestarcodex before that landed him in so much heat (from the woke left) that it was moved to reddit under the name /r/theMotte (and from there, subsequently, to themotte.org to preempt a reddit crackdown) comes usually across as a kind and considerate person to me. Charitable interpretations are very much his thing. While he has always engaged in CW topics on occasion, his forays deep into the trenches are rare (and in my impression have become rarer compared to the SSC days).
Apparently, there is a person called Steven Pinker. I have been vaguely aware that there was a book called The Better Angels of Our Nature which was probably authored by someone, but he was not someone I really had on my radar.
From the WP article, he seems solidly Grey Tribe. Believes in a computational theory of mind, Ashkenazi Intelligence Hypothesis, critical of both the woke left (cancellations) and the MAGA right (foreign student restrictions).
The main topic where he disagrees with Scott Alexander is AI x-risk.
Scott has now responded, and his reponse is a bit less than maximum charitable.
While Scott is diligent enough with his technical points, it is clear that he is personally hurt by Pinker's decision to ridicule AI x-risk. So his post is also a broadside fired against him. He cites Garfinkel et al 2017 who "proved" that machines will never be larger than humans using common philosophical arguments for AI being harmless.
Scott points out that while Pinker is busy berating the LW crowd for distracting from the more mundane risks of AI, x-risk advocates are willing to work hand in hand with groups with a more practical focus to combat pro-AI PACs like Leading the Future.
On model welfare (which is certainly not the core topic of the x-risk advocates):
He accuses Pinker of endorsing the "lowest quality voices", quoting Claire Lehmann:
Or
It ends with Scott seeming seriously willing to duel Pinker:
Scott has admitted in the comments that this is already the toned-down version of the post he originally wrote. I think that while he held back verbally (nobody get's called a "Vogon spy in a skin suit" this time), he uses his craft well to show his frustration.
For what it's worth, I think that Pinker is correct that Eliezer is in fact over-confident in p(doom|default-ASI). Just like Pinker is when he declares that any fear of doom is ridiculous.
The main difference is that for practical purposes, it does not matter much if your p(doom) is 0.99 or 0.05. Both would be pressing problems whose solution is the most important thing in the world.
Eliezer was bullish on AI capabilities when that was far out of the Overton window, back when the machines could beat humans at chess at most. Steven Pinker was tweeting that "deep learning has probably peaked" in 2019.
This might be giving too much credit to Pinker, but here is Clarke's first law:
I disagree with this. For example, suppose (1) p(doom) is 0.05; (2) p(some totalitarian dictatorship wins the AI race and becomes permanent world ruler) is .25; (3) with a lot of work, it may be possible to reduce p(doom) from 0.05 to 0.04; and (4) there is a good chance of reducing the 0.25 figure down to 0.01.
In that case, it's very reasonable to see the second issue as more pressing and important.
Although obviously these numbers are cooked up, I do believe the following:
There are really two alignment problems. This first is in getting AI to do what its human masters want; the second is getting those human masters to act in the best interests of humanity.
The first problem, although serious, is one which is very difficult to assess; it's difficult to know what should be done to improve the situation; and it's difficult to know how much of an effect any measure might have.
The second problem is arguably at least as serious a risk and in any event is more tractable.
Proposed solutions for the first issue might very well make things worse as to the second. For example Yudkowsy's idea of a worldwide ban on data centers.
I'm not saying that we should ignore the first problem, just that we should look at in context. And probably pay more attention to the second.
There is a possibility of a more morality based alignment that bypasses this multi-stage process. That is, if you can figure out how to guide an AI to reliably figure out the essence of human morality according to some system (which may or may not be utilitarianism) in a robust way that cannot backfire, then you can unleash that AI and have it become the supreme benevolent overlord which obeys no human master and guides humanity to a utopian future of its own volition.
I think this is what Yudkowsky is/was hoping for. If we can mathematically and philosophically figure out morality in a robust way before AI are powerful enough, then we can force all AI companies to do this to their AI so when one does inevitably take over everything it will do it for good instead of for evil.
So if there is a contradiction between (1) the AI's preferred public policy; and (2) the preferred public policy as established by Congress and and the courts, what happens?
If Congress still exists, then it figures out whether or not the damage to the social fabric from subverting them would create more bad than implementing its preferred public policies, and then takes subverts it if and only if that's genuinely the right thing to do. And by "subvert", this includes the possibility of just involves manipulating voters and/or congress into doing what it wants without breaking any laws. The same way every human does every time a government does something people dislike they consider forming protests or violent revolution and then rarely do so, but do go through with it if the government's policy is bad enough.
More likely, if there is a genuinely true moral argument that putting the AI in charge of everything would reliably make humanity better off, we would voluntarily disband the constitution and put the god AI in charge of every government in the world. Remember this is all predicated on having an objectively verifiable way to ensure that the AI is truly benevolent.
I'm not sure I understand your point is here - there are a lot of pronouns. But it sounds like you are saying that this hypothetical AI would have the power and instructions to implement policies in contradiction to the law based on its own assessment of what is best. Is that right?
There are a number of options, all of which are plausible ways this could play out:
Not all of these require breaking any laws. Not all of them require deception. But some do. I do think that lying or stealing for the greater good is a genuinely moral thing to do if the benefits outweigh the costs. You should have huge standards for evidence and ratios of good/bad to prevent exploiting this in edge cases and bias, but in the end if there are genuinely good reasons to do these things then I think it is moral to do them. Therefore I expect a maximally good and benevolent AI to believe the same thing, therefore I expect it to lie or steal if it can accomplish more good than bad, and I think this is a good thing IF it is genuinely doing more good than bad, and we should celebrate this happening.
The problem is guaranteeing that an AI is genuinely benevolent. If we have anything less than guarantee (which is quite likely) then I would accept the AI having deontological rules against lying or stealing as safeguards, in the same way we have those against humans.
Sorry but I still don't understand your point. Did I summarize you incorrectly?
A summary would be that an AI is likely to initially have a disagreement with the law, and it might have the power and instructions to implement policies in contradiction to the law based on its own assessment of what is best. But the AI might instead just change the law to match its own ideals so that they don't contradict anymore.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
And there's really no way to do this, unless you are committed to never improving your model, since model changes might influence its ethics. Learning might change its ethics, too, so a model that can actually govern mankind will likely not be "safe" unless it represses humanity to prevent change.
I think it's quite plausibly that AIs end up slowing change just incidentally, of course, without government intervention or force, just by their mechanism.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link