This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
If you want to make the argument that AI development will hit a wall then that's certainly a position where intelligent people can disagree, it's just a different argument to the one where the wall is imposed by government interest in not going too far past what is needed for population pacification.
Comparing AI to other weapons systems in a maximally generic way seems silly. Bombs aren't developed unboundedly because increased explosion yield has a relatively low upper bound of being useful at all. Obliterating more than a city center just isn't strategically useful so development continues but in other areas like platform and delivery development which are themselves bounded by MAD doctrine. AI doesn't have this same yield scaling diminishing return. More intelligence is simply better and will be simply better scaled up to a point where alignment becomes existential.
And there's no reason to think that the government wouldn't also believe that after a certain point, intelligence would either be actively harmful or not worth the extra effort required to get it.
A position which is asserted commonly as if needing no defense despite the fact that this does not seem to be true of our own species, at least in the evolutionary sense.
You keep doing this thing where I talk about how more powerful AI is obviously more useful for existential inter-government rivalries and then you note that it might not be useful past a point for pacifying a population as if it's a counter argument. It's not, these things can obviously both be true at once.
It is trivially better for smiting your enemies and evolutionally brains have grown right up to the size that they barely fit through the mother's birth canal and it's seen fit to leave the offspring helpless for a long time so that they can develop even further. There are some lower kinds of intelligence with very low pressure environments where the extra calories aren't worth the extra compute but that isn't the environment we find ourselves in. Intelligence is the ability to manipulate the environment to your will, the environment is contested and more intelligence wins the competition.
I've also pointed out that in traditional weapons systems that are useful for inter-government rivalries aren't subject to unbounded growth either. You've replied by special pleading for "intelligence." Now, you define intelligence as the ability to manipulate the environment to your will. (We can set aside for the moment the fact that this is an atypical definition of intelligence - it suggests that a 200 IQ paraplegic is much much less intelligent than a newborn.)
Very well - various world governments have passed on massive intelligence advantages in the past for reasons as trivial as budgetary decisions or as silly as unfounded environmental concerns pushed by fringe ecological groups. Why will AI intelligence different from, to pick just one obvious example, the intelligence [ability to manipulate the environment to your will] provided by nuclear power?
If we go with your definition of intelligence, that is probably true. But by that definition, AI is very far from being meaningfully intelligent. On the one hand, I like your definition because it highlights one of the things I have been banging on about AI here at the Motte; on the other hand, I dislike it because it doesn't let me dissect the divergence from "good at tests" and "good at power" - two things which are often conflated and, in my opinion, should not be.
Like how a slippery slope is not a fallacy if you describe why the slope is slippery(otherwise informing someone of a slippery step they ought watch out for as they leave a building would be a fallacy), "special pleading" is not a fallacy when one describes why a specific case is different, which I have. I've described why other weapons systems do not have unbounded development, thus it is not special pleading.
If the 200 IQ paraplegic is able to communicate in any way I do expect they would be able to have a larger impact on the general environment. If you want a more explicit definition Yud comes up with cross domain optimization although applying it to our current conversation would take a couple back and forths.
Genuinely, if you have not read the sequences yet, I suggest you do. At least then you'll be arguing with the Yud contingent in their own terms and not your misunderstanding of them.
Sure, and perhaps we can agree to disagree on the extent to which your arguments about the distinctions were persuasive. I am open to believing that intelligence is qualitatively different than other weapons systems, but I think the idea that the selection pressures (if you will) that apply to other systems won't apply to it is silly. Hopefully that distinction makes sense.
I guess I don't really understand the general rule you're trying to make. Nuclear is the only real weapon system that has truly had a strongly bounded development ceiling. We're still making better missiles, airplanes, drones, boats, ect. The bounding on those is mostly just hard physical rules. Intelligence scales in a new vector direction. Not only is the intelligence directly usable in warfare through things like cyber where the offence/defense equilibrium seems to favor the attacker, but it also acts as a multiplier for all the other scaling. You get better airplanes, drones, boats and missiles faster because one of the most bottle necked inputs to improvements is intelligence. And then there are all the recursive elements, scaling intelligence scales how well we can scale intelligence, it also allows us to efficiently search design space for other weapons systems to scale. Intelligence scaling is the trick that let humans conquer the planet, increasing our access to it is a whole new game.
The military does not actually just pursue boundless development. Budgets have to be justified, and if a threat is not assessed to be present then development will not be pursued. For this reason the government often foregoes or even loses capabilities simply for budgetary concerns. The F-22, for instance, faced budgetary scrutiny after the fall of the Cold War because the Soviet threat no longer existed, and when it was procured, it was without an IR sensor (which was arguably short-sighted: they are now integrating IR pods onto the Raptor). And even that procurement decision was made because the Raptor was assessed to be a less risky, more mature design (the F-23 was superior in many respects). Most of the ships the US has now were essentially the cheaper options: the Virginia class was a budget Seawolf, the Tico cruisers were meant to be the low-end of a high-low mix that never came about, the Burke class has been dramatically expanded in capability beyond what was originally intended, as I understand it, rather than spend more money to build a ground-up capability. The retirement of the F-14 left the Navy without a fleet interceptor and it's only been recently that the Rhino has been able to pick up the slack with improved AMRAAMs and the air-launched Standard, and even now the Rhino is likely dramatically less capable than an upgraded F-14 would have been (at least in payload, range, and sensor power, although the F/A-18 has a lower RCS).
Basically, the general rule in the military is not to procure stuff simply because it is good. You procure stuff to defeat a very specific threat. You won't be able to go to the military and say "intelligence is good, spend trillions" - you need to be able to justify the cost.
If the US government was willing to throw cash at the wall to improve intelligence across the board, they could raise service member's salaries dramatically. They don't. The smart kids all go to Wall Street or Silicon Valley, although the Navy's nuclear submarine program and the NSA are able to scrape a few off to fulfill a few niche roles.
That's not to say that I disagree with you (and obviously the military is very interested in and will use artificial intelligence) but it's to hammer home my point: military procurement is not about procuring the best systems, it is about finding a compromise between cost and effectiveness. If the US military was given a blank check every time the opportunity to completely dominate the world arose, we would have had no-kidding battleships in orbit for about half a century now (and unlike the question of AGI/superintelligence/etc. the question of putting a nuclear powered nuclear propelled battleship in orbit is primarily an engineering one; the math is all worked out and has been "solved" for decades).
Again, because I think this bears repeating: if you go to the US government and you say "I can protect and ensure US hegemony for the foreseeable future practically guaranteed, it will just cost half a trillion dollars" there's every reason based on historical performance to think they will instead spend half a billion on "good enough" systems that can kick the can down the road. While intelligence might be different from past weapons systems, it seems unlikely that that difference will change the approach of the government procurement process (or the approach of the free market, for that matter).
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link