BANNED USER: temperamental dysregulation
DaseindustriesLtd
late version of a small language model
Tell me about it.
User ID: 745
Banned by: @Amadan
DaseindustriesLtd
late version of a small language model
3yr ago
Ah yes, «people are the new oil».
I like people. A few of them very much so, and many more at least a bit. My sympathy is egoistic: people are the set of beings to which I belong. I am a history and a world unto myself, and a particular take on the universe that we share. Others are the same. We are similar in nature and in scope, but unique in a way that needs no justification – unique like numbers are. So every death is a catastrophe, every death immiserates the sum of reality that remains.
Trivializing this with childish cynicism is, to my eyes, merely a petition to be excluded from that set of valuable beings. If you do not see yourself as an entity of immense worth, I can understand not extending the same courtesy to others.
DaseindustriesLtd
late version of a small language model
3yr ago
I say they should substantiate their estimates before picking the well-known evil we've been warned about so many times before over the evil they loosely extrapolate.
DaseindustriesLtd
late version of a small language model
3yr ago
Unless you're subscribing to some ineffable human spirit outside material constraints brainwashing is just a matter of using the right inputs to get the right outputs
It does not follow. A human can be perfectly computable and still not vulnerable to brainwashing in the strong sense; computability does not imply programmability through any input channel, although that was a neat plot line in Snowcrash. Think about this for a second, can you change an old videocamera's firmware through showing it QR codes? Yet it can «see» them.
If we invent machines capable of parsing an entire lifetime of user data, tracking micro changes in pupillary dilation, eye movement, skin-surface temp changes and so on
Ah yes, an advertiser's wet dream.
You should seriously reflect on how you're being mind-hacked by generic FUD into assuming risible speculations about future methods of subjugation.
DaseindustriesLtd
late version of a small language model
3yr ago
It's historically normal for China to have the world's greatest GDP. USA is the weird one here.
DaseindustriesLtd
late version of a small language model
3yr ago
China tried to buy and bribe and coerce and woo people who could enable them to produce competitive hardware. They started in the cooperation era already. They dedicated over a trillion to the task. It took them decades to admit failure, and new American sanctions are mopping up their silly projects.
Iran tried to catch up to a century-old technology to secure what it perceives as its existential interests, and still tries, and fails, and will fail in the future.
The logic that the opponent will get what is necessary for him to get amounts to magical thinking. Incentives do not magically transform into actualized successes. Yes, your opponent will make moves, but if you have decisive advantage, you will still be able to thwart them.
And some «bootlegs» are bad enough to not matter. Hesbollah has pipe rockets. Israel ballistic missiles and Iron Dome. It is impossible for Hesbollah to do crippling damage to their enemies with what they have, even if they literally sacrifice all of their members for it. With next-gen AI accelerators, it's not very different. At some point in the not-too-distant future, training runs affordable to Google and Microsoft and Department of Energy will require gigawatts of power supply and a small beach's worth of silicon in conventional hardware, with no time to build either.
That's assuming there even is insubordination. China bought muh overpopulation, China recognizes climate change, China pioneered Covid overreaction, China will possibly adopt extreme Yuddist position for AI )once somebody makes a convincing enough Xinnie Pooh animation with a diffusion model). Other bad guys aren't even playing.
the US would go to war with the world over AI, the world would learn to live without the US
The world would lose. Likely will. Luckily for Americans, most parts of the world that matter will submit to their moral authority even without that Herculean undertaking.
DaseindustriesLtd
late version of a small language model
3yr ago
Would anyone with a brain initiate zero COVID, suffocate the country for three years, then cancel it overnight and overload the medical system?
China is observably not maximizing their odds of geostrategic dominance, nor much of anything else, sans ass-covering by party elites.
And before we start worrying about Choynese AGI, we should focus on something they've had much more of a head start in: Choynese Eugenics.
China has been running the world's largest and most successful eugenics program for more than thirty years, driving China's ever-faster rise as the global superpower. I worry that this poses some existential threat to Western civilization. Yet the most likely result is that America and Europe linger around a few hundred more years as also-rans on the world-historical stage, nursing our anti-hereditarian political correctness to the bitter end.
When I learned about Chinese eugenics this summer, I was astonished that its population policies had received so little attention. China makes no secret of its eugenic ambitions, in either its cultural history or its government policies.
Chinese eugenics will quickly become even more effective, given its massive investment in genomic research on human mental and physical traits. BGI-Shenzhen employs more than 4,000 researchers. It has far more "next-generation" DNA sequencers that anywhere else in the world, and is sequencing more than 50,000 genomes per year. It recently acquired the California firm Complete Genomics to become a major rival to Illumina.
The BGI Cognitive Genomics Project is currently doing whole-genome sequencing of 1,000 very-high-IQ people around the world, hunting for sets of sets of IQ-predicting alleles. I know because I recently contributed my DNA to the project, not fully understanding the implications. These IQ gene-sets will be found eventually—but will probably be used mostly in China, for China. Potentially, the results would allow all Chinese couples to maximize the intelligence of their offspring by selecting among their own fertilized eggs for the one or two that include the highest likelihood of the highest intelligence. Given the Mendelian genetic lottery, the kids produced by any one couple typically differ by 5 to 15 IQ points. So this method of "preimplantation embryo selection" might allow IQ within every Chinese family to increase by 5 to 15 IQ points per generation. After a couple of generations, it would be game over for Western global competitiveness.
Edge.org 2013: 2013 : WHAT SHOULD WE BE WORRIED ABOUT?
Geoffrey Miller, currently an AI decelerationist
How's it been going for the last decade?
China is not a competitor to the West. China will implement any braindead regulation the West devises, faster and harsher and stupider. China is less relevant than Turkey or, certainly, Israel. I will say it as many times as I have to.
DaseindustriesLtd
late version of a small language model
3yr ago
China might well be willing to take the risk of a containment failure to be guaranteed to be the global leader in all aspects of military and economic and social power for centuries.
I wonder if there is anything at all that can shake American confidence in this projection. Not to mention the premise that Chinese leaders understand what is at stake.
DaseindustriesLtd
late version of a small language model
3yr ago
The end result is still just absolute tyranny for whoever ends up dancing close enough to the fire to get the best algorithm
Why? This assumption is just the ending of HPMOR, not a result of some rigorous analysis. Why do you think the «best» algorithm absolutely crushes competition and asserts its will freely on the available matter? Something about nanobots that spread globally in hours, I guess? Well, one way to get to that is what Roko suggests: bringing the plebs to pre-2010 levels of compute (and concentrating power with select state agencies).
This threat model is infuriating because it is self-fulfilling in the truest sense. It is only guaranteed in the world where baseline humans and computers are curbstomped by a singleton that has time to safely develop a sufficient advantage, an entire new stack of tools that overcome all extant defenses. Otherwise, singletons face the uphill battle of game theory, physical MAD and defender's advantage in areas like cryptography.
If this tech takes off all it will take is flipping a few algorithmic switches and you and any prospective descendants will simply be brainwashed with surgical precision by the series of algorithms that will be curating and creating your culture and social connections at that point
What if I don't watch Netflix. What if a trivial AI filter is enough to reject such interventions because their deceptiveness per unit of exposure does not scale arbitrarily. What if humans are in fact not programmable dolls who get 1000X more brainwashed by a system that's 1000X as smart as a normal marketing analyst, and marketing doesn't work very well at all.
This is a pillar of frankly silly assumptions that have been, ironically, injected into your reasoning to support the tyrannical conclusion. Let me guess: do you have depressive/anxiety disorders?
DaseindustriesLtd
late version of a small language model
3yr ago
I like Anatoly Karlin's argument:
I disagree with AI doomers, not in the sense that I consider it a non-issue, but that my assessment of the risk of ruin is something like 1%, not 10%, let alone the 50%+ that Yudkowsky et al. believe. Moreover, restrictive AI regimes threaten to produce a lot of outcomes things, possibly including the devolution of AI control into a cult (we have a close analogue in post-1950s public opinion towards civilian applications of nuclear power and explosions, which robbed us of Orion Drives amongst other things), what may well be a delay in life extension timelines by years if not decades that results in 100Ms-1Bs of avoidable deaths (this is not just my supposition, but that of Aubrey de Grey as well, who has recently commented on Twitter that AI is already bringing LEV timelines forwards), and even outright technological stagnation (nobody has yet canceled secular dysgenic trends in genomic IQ). I leave unmentioned the extreme geopolitical risks from “GPU imperialism”.
While I am quite irrelevant, this is not a marginal viewpoint—it’s probably pretty mainstream within e/acc, for instance—and one that has to be countered if Yudkowsky’s extreme and far-reaching proposals are to have any chance of reaching public and international acceptance. The “bribe” I require is several OOMs more money invested into radical life extension research (personally I have no more wish to die of a heart attack than to get turned into paperclips) and into the genomics of IQ and other non-AI ways of augmenting collective global IQ such as neural augmentation and animal uplift (to prevent long-term idiocracy scenarios). I will be willing to support restrictive AI regimes under these conditions if against my better judgment, but if there are no such concessions, it will have to just be open and overt opposition.
Naturally, people who speculate that their safetyism is protecting quintillions of eventual superduperhappy podmen scoff at a few tens or hundreds of excruciating megadeaths of their contemporaries.
Yudkowsky, for me, was at his most sympathetic when he lamented the death of his brother Yehuda Nattan.
I used to say: "I have four living grandparents and I intend to have four living grandparents when the last star in the Milky Way burns out." I still have four living grandparents, but I don't think I'll be saying that any more. Even if we make it to and through the Singularity, it will be too late. One of the people I love won't be there. The universe has a surprising ability to stab you through the heart from somewhere you weren't looking. Of all the people I had to protect, I never thought that Yehuda might be one of them. Yehuda was born July 11, 1985. He lived 7053 days. He was nineteen years old when he died.
I wonder at the strength of non-transhumanist atheists, to accept so terrible a darkness without any hope of changing it. But then most atheists also succumb to comforting lies, and make excuses for death even less defensible than the outright lies of religion. They flinch away, refuse to confront the horror of a hundred and fifty thousand sentient beings annihilated every day. One point eight lives per second, fifty-five million lives per year. Convert the units, time to life, life to time. The World Trade Center killed half an hour. As of today, all cryonics organizations together have suspended one minute. This essay took twenty thousand lives to write. I wonder if there was ever an atheist who accepted the full horror, making no excuses, offering no consolations, who did not also hope for some future dawn. What must it be like to live in this world, seeing it just the way it is, and think that it will never change, never get any better?
It's still about one minute, likely zero, for our cryogenic technology didn't progress much since then. I liked Yud more when he worried about that, instead of freaking out on podcasts about the need to slow progress to a crawl.
OTOH, never cared much for Roko.
DaseindustriesLtd
late version of a small language model
3yr ago
you think "just take the fascism to 11 and it solves everything" is going to help?
They always do. You see, that fascism wasn't rationally planned enough, now here's my proposal…
I think people who make decisions should have less credentials and more experience with different and unappealing facets of the world – and not isolated to enclaves of their distributed elite republic of conferences and respectable institutions. Regularized, so to speak. Ideally we should have the equivalent of Rumspringa for the aspiring PMCs. Imagine if every one of those do-gooders had to work for a year in a provincial Russian police office, then in a Haitian hospital, then something else along these lines. Perhaps the resulting selection and education wouldn't necessarily be very helpful, but it'll probably make things more interesting.
Seriously though I accept @2rafa's arguments, «compute governance» at least for new capacity is easy. I'm also more pessimistic because there's significant political will in favor of it, and crucially, no coherent and comparably fanatical opposition. We know how this went in Culture War battles.
DaseindustriesLtd
late version of a small language model
3yr ago
^^^ This is the societal consequence of Yudkowskian propaganda. This is why we fight.
DaseindustriesLtd
late version of a small language model
3yr ago
Since when are you under the impression that this is the choice? «The machine» will be built, is already largely built, the question is only whether you have control over some tiny share of its capabilities or it's all hoarded by the same petty tyranny we know, only driving the power ratio to infinity.
DaseindustriesLtd
late version of a small language model
3yr ago
Simply put, I do not believe this behavior generalizes to all unemployable people. The current crop is heavily comprised of, well, genuine scum. The assumption that their antisocial outcomes are expected for people who can get employed now, but not necessarily tomorrow, is a cope for the necessity of labor.
DaseindustriesLtd
late version of a small language model
3yr ago
A very German thing to believe. I weep for your people, but really you've been cooked since before both of us were born, so this revolution adds nothing.
DaseindustriesLtd
late version of a small language model
3yr ago
Never trust a Christian Scientist I guess. Especially when looking up a Rabbi's quote.
DaseindustriesLtd
late version of a small language model
3yr ago
What if we reach scaling limits soon, and only replace vast swathes of white collar work but don’t get into the recursive ASI machine god territory you seem to take for granted?
I do not actually assume recursively improving ASI in the true sense: returns from debugging and modest data engineering seem to have a ceiling, and we're not sure about much else working. But I also don't have to assume that.
Specifically my conservative prediction is something like this: near-term (<2027) AIs based on LLMs will have the general reasoning capability of a 130-140 IQ person as assessed either by success on real-world problems not loaded on esoteric knowledge or motor/perceptual abilities, or by long-term economic productivity; all the while being significantly cheaper to employ than such a person (say, <$3/hr, or <$3/amount of inference equivalent in output to 1 hour of such a person's labor).
What I do take for granted is that we have not exhausted the already published literature, to say nothing of in-house advances. Even GPT-4 is very likely far from the bleeding edge of research. Thus, I confidently say that it'd be asinine to expect the wave of progress that has carried us from GPT-2 to GPT-4 in four years to stop right about now, on white-collar midwit level – except by political fiat.
This conservative result would still be sufficient to blast through our current economic paradigm. Imagine multiplying the population of +2-2,5SD people in the world one hundredfold, but they only work and do not consume. Almost inevitably, in a few years we'll commodify human-level autonomous robotics, and that's, well, that's endgame.
Even if we do get ASI in 10-15 years it will be a brutal transition period.
It sure will. I don't expect much finesse even from well-meaning regulators.
Well, I basically buy that OpenAI is a new Manhattan Project, so that's not my biggest concern.
DaseindustriesLtd
late version of a small language model
3yr ago
We’re animals in pursuit of pleasure and status, and those who expect some kind of moral or cultural elevation in a labor-less future are kidding themselves.
“Goyim were born only to serve us. Without that, they have no place in the world; only to serve the People of Israel,” he said during a public discussion of what kind of work non-Jews are allowed to perform on Shabbat. “Why are gentiles needed? They will work, they will plow, they will reap. We will sit like an effendi and eat,” he said to some laughter. Yosef, the spiritual leader of the Shas Party and the former chief Sephardi rabbi of Israel, also said that the lives of non-Jews are protected in order to prevent financial loss to Jews. “With gentiles, it will be like any person: They need to die, but God will give them longevity. Why? Imagine that one’s donkey would die, they’d lose their money. This is his servant. That’s why he gets a long life, to work well for this Jew,” said the rabbi, who recently turned 90.
I'll commend you for being more of an equal-opportunity dehumanizer. But on the other hand Yosef at least allowed that some group of people be above beasts of burden, be meaningful ends unto themselves; the whole point of his not-exactly-marginal interpretation of Judaism is that Jews are such people. (An "effendi" is a lord, or a master, in Arabic, or so I'm told). And presumably he did more than eat, seeing his station. How about a deal: we appoint tool AI as «goyim», and be done with this ugly business.
I have never bought a Funko Pop, nor watched reality TV or played slot machine or a true gacha game for any length of time greater than needed to understand the principle. (This is not something to be proud of but more a consequence of boring behavioral rigidity. Also I basically cannot watch long-form videos, including podcasts and those vaunted American TV series that have replaced almost all culture for the middle class; this is a bit of a problem, another small problem a sufficiently developed AI will solve, opening up new troves of meaning. «Jarvis, boil down Yud's kvetching on Bankless, Fridman and Patel into a timeline of diffs with his previous eras, then superimpose on that substack about Extropia's Children!»).
Well, there have been some more addictive games I'll admit. But it's like with drugs: a perfect stimulant without burnout, or a psychedelic/opioid without tolerance, is a pipe dream. Our wetware that compels us to waste time on it also bounds the possible range of experiences, and thus makes it a finite journey. Even current games, trying to prevent disengagement, introduce additional mechanics almost as fast as humans grow tired of old ones and can learn new tricks (what fun is playing a game you can't grasp?), yet it still gets old, fast – faster than I can imagine any generative AI adapting. Even if it keeps up, at some not too distant point it will either saturate my bandwidth and close the category of «games» for good, like Tic-Tac-Toe, a fully solved space of behaviors; or, perhaps, it will deliver something that deserves a more serious term than a «game». And if I pursue richer modalities to appreciate new and fancier games – what's to stop me from modification in other directions, such as direct tampering with my reward system? Then, it's either switching off to exist in a pod as a wireheaded bugman, as you suggest, or evolution. Do we differ only in whether 0% or some fraction of humanity chooses the latter?
As for porn, it seems to me that its consumption drastically increases when I do not have fun and meaningful things to do and relationships to be part of. In any case, the same principle applies.
It also seems to me that I'm not exceptional. People who keep up the civilization around me from decaying do not watch reality TV only because they are bored out of their skulls: it's their cope. Like me, they have big or small real-world dreams – dreams they've put into the backlog and let them fall prey to decay. My backlog will plausibly span centuries if I can execute on it.
Even if theirs terminates on saturation with synthetic stimuli, I do not assert the right to condemn it. We – rather, Hajnali, East Asians and associates, for I have to remind you that «Работать западло» … oh God, I repeat myself so much – have evolved to condemn it out of necessity, to prevent slacking off and subsequent ruin. We don't have to do that anymore.
Have you read any Greg Egan? I think not. We usually discuss Iain Banks' Culture series (again series) as a portrayal of an advanced post-scarcity society. Anyway, Ehan's Diaspora is perhaps more in line with the good outcome I expect (with two the major caveats that a) my theory of personal identity does not allow surviving naive uploads and b) they do not have vastly superhuman AIs; but Citizens are for all intents and purposes AIs anyway, plugged into tools). The vast majority of Diaspora inhabitants are members of solipsistic «Polises», taken care of by AIs and having fun in their bespoke virtual utopias. This is fine. The self-selected minority explores physical universe. They, too, can turn back into virtuality or reach the end of their journey and accept expiration. This is fine as well.
Let us even be rain: the flight to the bottom is so long and exciting.
I'm pretty optimistic about our odds at finding meaning, because we find it successfully in such crap, it can't be very hard to find more – once crap is cleaned away. Likewise for sustenance: I don't particularly care how the financial side works out. So long as the critically capable technology proliferates enough to prevent unilateral power grab by some supposedly benevolent overlord like the USG using OpenAI as a front, it'll work out fine.
We're too used to stuff not working out. So much work is done just to tread water in this world of scarcity. It's immensely miserable. People all around have to toil, burn their lives, just to keep the civilization from decaying, to grow and deliver food, to fix the pavement, to write and debug code, to analyze datasets, to prescribe antibiotics – and that's still honest labor, still the ennobling sort; because many others, paradoxically often the well-off folks directly threatened by this technology, fight over the surplus value and create problems that have to be fixed (the inane issue of sales calls and spam-and-filter arms race comes to mind, scammy startups, much of finance… but that's just scratching the surface). To find meaning in this, to not contemplate suicide daily… Tens of thousands of years of selection under agricultural pressures sure have hurt us.
I think sometimes of Scott's review of that book about Indigenous Americans who looked with pity and disgust at the settlers, and settlers who «went Indian» and refused to recivilize themselves once «rescued». Sure, it's easy to mock the noble savage stereotype when you have all but exterminated these peoples and graciously allowed the remaining dregs devolve into alcoholic underclass, but that's speaking power to truth; with the truth being the fact that we've worked ourselves into a dead end and the only saving grace, the only possible redemption for the cursed route that Jared Diamond says has started with grain, is the possibility to hand the nightmare over to our ultimate tool, the universal solver, artificial general intelligence. This is a scenario Uncle Ted never anticipated, that he wouldn't recognize as desirable, but it's the best answer to his challenge that we can produce and likely will.
Suppose AGI works as intended. We first commoditize entire categories of high-end labor-intensive solutions, then de facto close markets for those solutions when their wares become cheaper than dirt and ubiquitous. Sure, it's not impossible that this will fail, for normal Molochian reasons, that AI will simply up the ante; but also not impossible that in the race between the rapidly improving universal solver – perhaps universal solvent too – and human greed/stupidity/incompetence the former achieves supremacy. Imagine a world where no code is buggy because bugs are found and patched faster than they are written. Then software begins to grow better, less bloated, optimizing on all axes including those the market had to discard, moving the entire multidimensional Pareto surface toward perfection. Then, imagine this applied to everything you deal with. Fewer and fewer problems. Fewer and fewer people employed to make them go away, coping that they would feel useless and meaningless without applying themselves to those Augean stables, that they'd just become deadbeat junkies or worse. Fewer and fewer copes to be heard.
We're in a dysfunctional codependent relationship with the festering undying corpse of our industrial civilization, the needy monstrosity that has to be fed our lifetimes. It's nice for people who feel happy with their «jobs» I suppose, but in the end, for the vast majority a job is something you wouldn't do if not paid for. If our tools fix all problems that require payment to make people bother, what will be left? Truth, perhaps. High-grade challenge that is somehow not amenable to automation. Relationships we actually want be part of and care about. Games. Self-expression. Contemplation. Philosophy. Things people turned to whenever they managed to escape the peasant-civilization hell for more than one generation. And new things too: things we are afraid to think of now because of how brow-beaten we are into normality.
It's pathological to fear the separation from our current regime of incentives. We'd have left much earlier if we could, but we couldn't, not without getting exterminated by those who stayed; and so we grew into the shape of our cope. The sooner this ends, the better and less painful.
DaseindustriesLtd
late version of a small language model
3yr ago
I think the issue has more to do with internalizing hypergamous female standards. A decently hot – say, 8/10 – woman is «not defective», we do not expect model looks, only generic high-quality genome (high symmetry, good skin/hair etc). A hot man is in the direction of the proverbial Gigachad/Pattinson/Cavill, an outlier in the sense of facial masculinization PLUS symmetry and those generic qualities. You do not get to above-average masculinization by averaging regular men.
Granted, a «very hot» woman will also be impossible to get with this method.
DaseindustriesLtd
late version of a small language model
3yr ago
Actually I haven't clicked the link and had no idea how she looks, nor do I care to click and learn that now. I usually presume that LW rationalists are agender textual entities (e.g. who is Kaj Sotala? Male, female, Indian-themed enemy from a roguelike? Sounds vaguely androgynous to me but I do not care).
But those pictures are of averaged women. It so happens, by virtue of trivial statistics, that you can average out random shortcomings and developmental errors with a random sample. Averaged women and men look pretty nice, markedly above average; at most, they can tell us about things like the typical anthropometric type and obesity rate.
... On second thought, an averaged man looks more meh. I guess this tells us something about eggs and sperm.
DaseindustriesLtd
late version of a small language model
3yr ago
Actually, it's a postrat «don't mean-spiritedly dunk on a rationalist» challenge (impossible), as is observable in comments.
Is this what we've come to? Ridiculing weird people for trying to have a consensual relationship they want? I mean, in a sense, a progressive would say that's a big part of what we do…
Regardless, I believe Katie will succeed in finding an agreeable partner, even if not an exact match. People in her social milieu do not really care for stuff like «town bike» or whatever. Age and looks – well, male rats are not movie stars either. Twitter infamy will come and pass.
This may be a genuinely rational move.
DaseindustriesLtd
late version of a small language model
3yr ago
Hilariously, his argument applies rather well to artificial neural networks. There, learning updates are also system-wide (unless you deliberately constrain them to a subset of weights) and we also can't always point to parameters that «store a fact», despite knowing perfectly that neural networks memorize, and even understanding how they do it. And if it's something less legible than a fact, such as a reasoning heuristic…
DaseindustriesLtd
late version of a small language model
3yr ago
It's not impossible but very hard in practice to prove a negative. You know that anti-nuclear people also demand extremely strong, cost-prohibitive proofs of safety, which is why we're in this mess. Of course, they have other nefarious motives to suppress human flourishing, but so do AI alarmists.
More to the point: decades ago, Nick Bostrom has proposed a taxonomy of X-risks. Those risks should be rigorously compared, for we must hedge all of them somehow. Some of those risks seem highly likely to me, follow from our prior social failures and even particularities of the current trend, and are comparable to «total human death» in moral (if not «utilitarian») badness, so the argument about «risk from AI cannot be quantified» doesn't hold. Bostrom:
While some of the events described in the previous section would be certain to actually wipe out Homo sapiens (e.g. a breakdown of a meta-stable vacuum state) others could potentially be survived (such as an all-out nuclear war). If modern civilization were to collapse, however, it is not completely certain that it would arise again even if the human species survived. We may have used up too many of the easily available resources a primitive society would need to use to work itself up to our level of technology. A primitive human society may or may not be more likely to face extinction than any other animal species. But let’s not try that experiment.
If the primitive society lives on but fails to ever get back to current technological levels, let alone go beyond it, then we have an example of a crunch. Here are some potential causes of a crunch:
5.1 Resource depletion or ecological destruction
The natural resources needed to sustain a high-tech civilization are being used up. If some other cataclysm destroys the technology we have, it may not be possible to climb back up to present levels if natural conditions are less favorable than they were for our ancestors, for example if the most easily exploitable coal, oil, and mineral resources have been depleted. (On the other hand, if plenty of information about our technological feats is preserved, that could make a rebirth of civilization easier.)
5.2 Misguided world government or another static social equilibrium stops technological progress
One could imagine a fundamentalist religious or ecological movement one day coming to dominate the world. If by that time there are means of making such a world government stable against insurrections (by advanced surveillance or mind-control technologies), this might permanently put a lid on humanity’s potential to develop to a posthuman level. Aldous Huxley’s Brave New World is a well-known scenario of this type [50].
A world government may not be the only form of stable social equilibrium that could permanently thwart progress. Many regions of the world today have great difficulty building institutions that can support high growth. And historically, there are many places where progress stood still or retreated for significant periods of time. Economic and technological progress may not be as inevitable as is appears to us.
6.3 Repressive totalitarian global regime
Similarly, one can imagine that an intolerant world government, based perhaps on mistaken religious or ethical convictions, is formed, is stable, and decides to realize only a very small part of all the good things a posthuman world could contain.
Such a world government could conceivably be formed by a small group of people if they were in control of the first superintelligence and could select its goals. If the superintelligence arises suddenly and becomes powerful enough to take over the world, the posthuman world may reflect only the idiosyncratic values of the owners or designers of this superintelligence. Depending on what those values are, this scenario would count as a shriek.
It is counterproductive to focus only on the well-propagandized model of of AI takeover through FOOM, in the age where AI built on principles radically different from those preferred by FOOM-argument-inventors is undergoing its Cambrian explosion; and in doing so exacerbate those Crunch-type risks. It is unprincipled. Moreover, it's wishful thinking: if only we could guard our asses from this one threat model! Perhaps one type of risk is truly greater than another, in raw probability or expected negative value or both. But just rehashing thought experiments about Seed AI from the 90s won't suffice to prove that the orthodox AI risk is the greater evil.
Now Bostrom himself proposes building a 6.3 regime, and Eliezer helpfully paves the way to it through his alarmism about training of capable models. I say we should at least demand they spell out why the possibility of eternity under their benevolent yoke, or fizzling out due to squandering our chances to expand, is preferable to getting paperclipped.
Because for me it is not so clear-cut. And be aware that we can fizzle out. I've argued about this here. We evidently have more than one chance to build an «aligned» (or as I'd rather have it, no-alignment-needed) AGI. We don't have infinite time for globohomo committees to surmount their perverse incentives, discover the true name of God through the game of musical chairs at Davos and immanentize Dath Ilan before proceeding to build said AGI – nor, I'd say, very good odds at aligning those committees to play the game in our interest.
Whatever came of Contrapoints getting canceled by other trans figures? Some right-wingers gloated about meta-cancer back then, but I guess you don't really cancel your stars after all?
This is a narrative which was promoted by the media, yes, I even imagine there's a grain of truth in it, but if she actually engaged with the «alt-right», it was on the condescending level of cooing and petting a spooked animal (and argument-free reviews to the effect of «I used to be in a dark place as a 4channer giganazi Cookie Monster joke enjoyer, but Contra
detransitionedderadicalized me, thanks now I can be myself» only reinforce this image. I have to plug in my old post «On Nazi Whisperers and closed memetic surfaces».Almost four years later, I still can't get into the mindset of a person who thinks his side loses every unbiased debate, but keeps believing in the cause and evangelizing it. This is a total profanation of what debates mean to me, on any level from the most brutish, chimp-like rhetorical power contest to the most quokka-esque rationalist «collaborative inquiry»; if not rigorous doubt, then at least depressive resignation ought to settle in as a result of repeatedly getting routed, I feel.
What I do know is that such robust belief grows coupled with disregard for debate and honesty in general.
More options
Context Copy link