This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I agree with pretty much everything you said here. But regarding this:
How can you have a vision of transhumanism that doesn't require a benevolent world-ruling cult?
Well you institute open benevolent tyranny. Duh. Long live Tovarisch Transtalin! Death to the genemod-wreckers and tradhulaks!
That's of course not my idea of transhumanism, I'm entertaining your uncharitable frame. Nevertheless it is important, I think, to emphasize that Yud's conspiracy is unusual in that it continues as such past the establishment of the just world order. In literal Protocols, the purported end state follows a Grand Reveal and entails a rigid, but in many ways fair, sound and prosperous regime of open ethnic supremacism and Messiah's kingship. It's close to Moldbug's reformalization of power and to most conspiracy theories that have an end state. Yud is worse than that. He doesn't have an end state; Dath Ilani Keepers keep snooping around and manipulating all matters scientific and political, guarding «infohazards», unburdened from even the flimsy accountability that a tyrant's legible sovereign standing can imply. Their business is to worry for others; and others are not sharp enough to evaluate whether their worry is justified or pointlessly restrictive. It's very much in the spirit that Rao investigated in The Priest In The Arena.
Now, I don't have a clear idea as to what you believe my «transhumanism» is about. So for contrast, here's young Yud again:
He changed accents somewhat, became more jaded, but I don't think he ever swore off this Great Task.
So his idea here is a totalizing eschatological project to transform the human species into data. As an aside: I think trads like @urquan who seem to buy his flirtation with themes of muh poor children threatened by AI would do well to consider their odds of survival in the flesh, should they help these guys monopolize oversight of AI development under the guise of preventing AI X-risk. World-optimizing Yuddites think «misaligned AI» is a problem only insofar as it closes off the path to their utopia; but so do commoditized AIs that can be used to defend from their attempt at a Singleton and continue our business of chaotic, unstable, adversarial-cooperative civilization that they call «Moloch». If that hurdle is overcome, hoo boy, Moloch dies and so do you. Or not, maybe you never die. Depends on how you see it.
Politically it's utter bullshit for reasons that, I hope, needn't be spelled out. I don't want a World Government, I hate the Antichrist, I find utilitarian doctrine stomach-turning, I do not see the point in aligning humans with some optimal image. I am not a moral busybody, I wasn't raised on «Tikkun Olam» spirituality, I don't want to see the the world «optimized», and mainly want it to allow beings like me freedom of form, expression and evolution – in the limits that respect welfare of other sentients and their legacies. I didn't ask to be born human, it took me years to start feeling like one, I just think it's reasonable if I'm allowed to try being something else instead – but that logic naturally works both ways. Yud thinks he's not a human but «an intelligence», I think he's a cringeworthy slob, a trad Christian thinks he's an embodied sovl that'll be resurrected in the flesh for Final Judgement. Whatever, to each his own.
I think the technical intuition behind Yud's transhumanism is not entirely bullshit, but it's suspect enough that I'd rather not strongly advise anyone to take part.
His metaphysics, too, is not really bullshit, but I'm not sure about metaphysics. Despite accusations of vulgar materialism, I'm closer to neutral monism (similar to Christof Koch or this guy), I dimly see how Jung may be right about death here, I understand that transhuman utopias are still finite so it's more of a quantitative upgrade and not a profound philosophical solution to inadequacy of this universe, and crucially I appreciate that metaphysical questions are unanswerable, probably not even those that seem to me to be easily explicable delusions or mere confusion in terms or logic.
We can reach high certainty about entities causally entangled with us, and it makes sense to accurately predict those as if nothing else matters, being for all practical purposes a materialist, but any factor beyond is a true unknown. Sure, psychedelics are a waste of life, but I've done enough to learn that people with spiritual and other «weird» beliefs are not being retarded, so I don't feel strongly justified in trampling on people's belief systems, debugging and rectifying them or otherwise denying them their apparently wrongheaded lives. Even when I feel clearly correct, smugly superior and in a role that justifies a measure of custody, I'm still in doubt about that measure. (Hilariously, my most daring effort to make anyone See Past The Cognitive Biases resulted in a Netflix-addicted urbanite becoming an Orthodox monk). Yud is perversely addicted to the childish sense of all other people being retarded in comparison to him (an addiction that Literature teacher had beat out of me publicly in 2rd grade – one of the benefits of not being an autodidact), so of course he feels entitled to usurp their agency and act on their behalf. And for that, he feels, a benevolent conspiracy is necessary.
But this doesn't follow from transhumanism in any manner.
I think this just about sums it up.
I wasn't trying to be uncharitable. It was a genuine question. You've thought much more deeply about these particular issues than I have, so my own line of questioning may seem crude and naive in comparison.
My initial thought would be something like: we regulate technologies that have the capacity to do a lot of harm, because if you give millions of people the ability to inflict mass casualties, then eventually someone is just going to go crazy and inflict mass casualties. At fairly regular intervals, someone will simply go off with a knife or a gun and do as much damage as they can. So we put restrictions on guns, we put restrictions on contagious pathogens, we put a ton of restrictions on nukes. Now imagine the kind of destruction that could be unleashed if a disgruntled individual had unfettered access to an ASI. Maybe the vast majority of transhumans are peace-loving intelligences who want nothing more than to quietly contemplate the secrets of the universe, but eventually you're going to get a transhuman Ted K who just gets fed up and wants to tear the whole system down. The only way for the transhuman collective to defend themselves against this inevitability is with strict regulation.
Material abundance alone is not sufficient to foreclose the possibility of conflict and destruction. America is a relatively prosperous and secure nation, if you keep your head down you can go through life comfortably and unmolested, but look at recent events like Charlottesville or January 6th - people still find reasons to murder each other over disputes that are essentially symbolic in nature. Maybe because they simply have nothing better to do.
I believe you've spoken in the past about the possibility of technologically modifying a mind's base instincts and desires. Maybe you want to sand down some of the rougher edges of human nature; maybe undergoing such modification will be a precondition of being granted access to the ASI or other transhuman enhancements (although at that point, the governing body is already starting to sound more cult-like). But I'm skeptical that you can do a pinpoint surgical removal of the capacity for aggression and destruction and still have the sort of complex and multifaceted psychological constitution that makes life worth living in the first place. There was an old adage in the hacker (in Paul Graham's sense) community that went something like, "you can't have the power to do brilliant things unless you also have the power to do stupid things". This was frequently said in response to someone who had just done an accidental "rm -rf /" and wiped their whole drive. I am inclined to offer this adage as a general principle of psychology, sort of like a Gödel's incompleteness theorem for the design space of minds.
I have no doubt that you're genuine in this desire, but I'm skeptical about the long-term stability of such an arrangement. Most humans feel no conscious malice towards animals, but they still end up doing great harm to animals anyway, either through sheer accident (oil spills or plastics finding their way into the ocean), or because animal suffering is viewed as a necessary price to pay in the pursuit of some other goal (animal testing for scientific research). Eventually, someone's galactic paperclip farm is going to have some rather unpleasant effects on the earth, even though their intentions were nothing of the sort.
I shouldn't whine too much about it though. Maybe it really is just the natural course of evolution.
I don't buy that it works like that. It's a Shin Sekai Yori world model; but intelligence per se is not psychic energy. It's only a superpower when it's in a class of its own, when you can intimidate philistines around you, grab actual energy resources from under their noses; go to college (which Yud did not) and see how well that works. A smart person can come up with a great plan but a number of dumber people can spoil it; an ASI can invent some shitty superhack but that only works if there are no sub-ASIs and awareness of potential ASI attackers. All of this fear relies on FOOM, nanobots that cannot be stopped, superviruses from kitchen dirt that cannot be filtered, and other pretty magical assumptions that really amount to there definitely emerging a Singleton who meditates on his navel, Yud-style, and gets a colossal head start. We don't seem to be in that timeline, and it's exactly the proliferation of AIs that keeps us away from it. Basically @Dean gets it right here.
Shin Sekai Yori world model again. Read Diamond Age for a more interesting alternative.
I'm pretty positive that general intelligence in some true sense will be fully commoditized in a few iterations of AI tech, LLMs to self-stabilizing exploratory agents to some safe AIXI approximation, with every peasant getting a powerful Oracle to use; and we'll run into the limits of economies of scale that make DIY production of computronium an insignificant factor.
To an extent that there remain disruptive enhancements in a mature transhuman world, they will be hard to reproduce locally, and market will figure this out early on, probably with simple tools like insurance costs conditional on assessments, I think.
I'm also pretty sure that we'll quickly develop strong and universal revulsion towards unbounded replicators and every trajectory that ends in self-satisfied replication or, more broadly, runaway optimization, to the point that it's comprehensively excised from our metaphorical gene pool. So even if that factory comes online, it will get blown to bits by relativistic bombardment. Yes, I'm talking airstrikes… but in Space!
This might seem like a tight rope to walk. But intelligence is all about finding narrow roads in the solution space. With all the raw compute and sample efficiency Singularity will give us, we'll find a better Pareto Frontier than is imaginable today.
Sounds like a benevolent world-ruling cult to me.
I'm not convinced that this idea of permanently eliminating the capacity for harm-doing at the physical level is even coherent. It's easy to imagine a galactic version of the trolley problem, something like, there was a sudden unforeseen power failure (even if you are Literally God, there is still only a finite amount of energy available to you at any given time), and we have to divert power from this server cluster that hosts one trillion artificial consciousnesses in order to save this other cluster that hosts five trillion artificial consciousnesses. You're harming the one trillion, but why are you allowed to do that exactly? You could program yourself to be a strict utilitarian, but, you've already indicated that you wouldn't be amenable to that. You could just use your own good and sound moral judgment, but then we're back at the original problem; eventually someone is going to use their own good and sound moral judgment to fuck shit up.
I would just sum up my position thusly: suffering and torture are eternal, they will persist for as long as consciousness persists. It is illusory to think that they could be transcended with technological means. You can have a personal aesthetic preference for whether you want your suffering on earth or in space, in carbon or in silicon, but suffering will continue regardless.
Yep. Appropriately, it's a dystopian world. I do not endorse this solution. Canthus users were arrested in their evolution, and became monsters through it. Squealer is the only truly human hero in the story.
There are two solutions here. One is luddism and indeed extreme antihumanism, so that we do not have the capabilities to do evil (or much of anything). The other is totalitarianism, so that our betters manage those capabilities on our behalf. The latter, I think, is straightforwardly bad because people who volunteer for that role deserve negative trust; the former is unworkable because these people aren't going anywhere. Thus there's no escape.
Except, you can say goodbye to your neuroses and enjoy watching humanity grow up. We will probably have bigger-scale fuckups from now on, but that's not guaranteed. It's more certain that we will have more fun.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link