This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
You said earlier
This, to me, contradicts unbounded development to maintain edge over rival nations.
Correct.
Now I am not saying that unbounded development is impossible and I (would argue that I) take alignment fairly "seriously" but if you look at other weapons systems we don't see unbounded development there. So our expectations should be that future weapons development will continue along similar lines. Not that that is the only course but that it is the most likely course.
You could counter-argue that over a long-enough timeline improbable events become likely, which is fair enough. But of course that is true of essentially all existential risks and does not imply that existential risk from AI is especially likely relative to other existential risks.
Does that make sense?
If you want to make the argument that AI development will hit a wall then that's certainly a position where intelligent people can disagree, it's just a different argument to the one where the wall is imposed by government interest in not going too far past what is needed for population pacification.
Comparing AI to other weapons systems in a maximally generic way seems silly. Bombs aren't developed unboundedly because increased explosion yield has a relatively low upper bound of being useful at all. Obliterating more than a city center just isn't strategically useful so development continues but in other areas like platform and delivery development which are themselves bounded by MAD doctrine. AI doesn't have this same yield scaling diminishing return. More intelligence is simply better and will be simply better scaled up to a point where alignment becomes existential.
And there's no reason to think that the government wouldn't also believe that after a certain point, intelligence would either be actively harmful or not worth the extra effort required to get it.
A position which is asserted commonly as if needing no defense despite the fact that this does not seem to be true of our own species, at least in the evolutionary sense.
You keep doing this thing where I talk about how more powerful AI is obviously more useful for existential inter-government rivalries and then you note that it might not be useful past a point for pacifying a population as if it's a counter argument. It's not, these things can obviously both be true at once.
It is trivially better for smiting your enemies and evolutionally brains have grown right up to the size that they barely fit through the mother's birth canal and it's seen fit to leave the offspring helpless for a long time so that they can develop even further. There are some lower kinds of intelligence with very low pressure environments where the extra calories aren't worth the extra compute but that isn't the environment we find ourselves in. Intelligence is the ability to manipulate the environment to your will, the environment is contested and more intelligence wins the competition.
I've also pointed out that in traditional weapons systems that are useful for inter-government rivalries aren't subject to unbounded growth either. You've replied by special pleading for "intelligence." Now, you define intelligence as the ability to manipulate the environment to your will. (We can set aside for the moment the fact that this is an atypical definition of intelligence - it suggests that a 200 IQ paraplegic is much much less intelligent than a newborn.)
Very well - various world governments have passed on massive intelligence advantages in the past for reasons as trivial as budgetary decisions or as silly as unfounded environmental concerns pushed by fringe ecological groups. Why will AI intelligence different from, to pick just one obvious example, the intelligence [ability to manipulate the environment to your will] provided by nuclear power?
If we go with your definition of intelligence, that is probably true. But by that definition, AI is very far from being meaningfully intelligent. On the one hand, I like your definition because it highlights one of the things I have been banging on about AI here at the Motte; on the other hand, I dislike it because it doesn't let me dissect the divergence from "good at tests" and "good at power" - two things which are often conflated and, in my opinion, should not be.
Like how a slippery slope is not a fallacy if you describe why the slope is slippery(otherwise informing someone of a slippery step they ought watch out for as they leave a building would be a fallacy), "special pleading" is not a fallacy when one describes why a specific case is different, which I have. I've described why other weapons systems do not have unbounded development, thus it is not special pleading.
If the 200 IQ paraplegic is able to communicate in any way I do expect they would be able to have a larger impact on the general environment. If you want a more explicit definition Yud comes up with cross domain optimization although applying it to our current conversation would take a couple back and forths.
Genuinely, if you have not read the sequences yet, I suggest you do. At least then you'll be arguing with the Yud contingent in their own terms and not your misunderstanding of them.
Sure, and perhaps we can agree to disagree on the extent to which your arguments about the distinctions were persuasive. I am open to believing that intelligence is qualitatively different than other weapons systems, but I think the idea that the selection pressures (if you will) that apply to other systems won't apply to it is silly. Hopefully that distinction makes sense.
I guess I don't really understand the general rule you're trying to make. Nuclear is the only real weapon system that has truly had a strongly bounded development ceiling. We're still making better missiles, airplanes, drones, boats, ect. The bounding on those is mostly just hard physical rules. Intelligence scales in a new vector direction. Not only is the intelligence directly usable in warfare through things like cyber where the offence/defense equilibrium seems to favor the attacker, but it also acts as a multiplier for all the other scaling. You get better airplanes, drones, boats and missiles faster because one of the most bottle necked inputs to improvements is intelligence. And then there are all the recursive elements, scaling intelligence scales how well we can scale intelligence, it also allows us to efficiently search design space for other weapons systems to scale. Intelligence scaling is the trick that let humans conquer the planet, increasing our access to it is a whole new game.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link