This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Nearly all of us also want GoF shut down, to be clear.
There is, however, some significant difference between "a vaccine-resistant smallpox pandemic", as bad as that would be, and the true final form of bioweapons that a superintelligent AI could possibly access.
The absolute best-case of what that looks like, as in "we know 100% that this can be done, we just don't know how yet" is an incompatible-biochemistry alga with reduced need for phosphate and a better carbon-fixer than RuBisCO (we know RuBisCO is hilariously bad by the standards of biochemistry; C4 and CAM plants have extensive workarounds for how terrible it is because natural selection can't just chuck it out and start over). Release this, it blooms like crazy across the whole face of the ocean (not limited to upwelling zones; natural algae need the dissolved phosphate in those, but CHON can be gotten from water + air), zooplankton don't bloom to eat it because of incompatible biochemistry, CO2 levels drop to near-zero because of better carbon fixation, all open-air crops fail + Snowball Earth. Humanity would probably survive for a bit, but >99% of humans die pretty quickly - and of course the AI that did it is possibly still out there, so focussing only on subsistence plausibly gets you knocked over by killer robots a few years later.
Medium-case is grey goo.
Worst-case is "zombie wasps for humans"/"Exsurgent Virus"; an easily-spread infection that makes human victims intelligently work to spread it. To be clear, this means it's in every country within a week of Patient Zero due to airports, and within a couple more weeks it's worked its way up to the top ranks of government officials as everyone prioritises infecting their superiors. Good. Luck. With. That.
It is possible for things, like normal GoF, to be extremely bad and yet still be a long way from the true, horrifying potential of the field.
Good point, it was wrong to say that it's not worth caring about AI bioweapon risk. ASI ought to be so strong that it doesn't need bioweapons though, I think it can wrap us around its finger such that we serve its will without any biowarfare. Just dominating the internet would be a civilization-scale parasitism like your wasps, it would be inside our entire command-and-control structure and in a position of assured dominance.
More options
Context Copy link
I’m more sanguine about this stuff now, and not because it’s wrong. It’s because there are essentially an infinity of ways for super intelligent ASI to wipe out the human race - these are just the ways we can think of, and it’s going to be much smarter than us. If it happens, it’ll happen anyway, any safeguards will be redundant. It’s like trusting a bear with the possibility space for killing a fox or something - it can come up with a method (and a feasible one), but it’s one of a thousand ways a smart human could come up with.
More options
Context Copy link
All of those sound bad, but also very speculative?
We have a recent worked example of what can happen with GoF (true regardless of the true origins of covid-19); shouldn't we prioritize making sure that doesn't happen again over "stop Skynet"/"Butlerian Jihad Now" type stuff?
It's like hearing that Ford Pintos can explode due to their fuel tank design and responding with "OMG, cars can explode! Terrorists might start planting car bombs, I should work on anti-terrorism!"
The last one is very speculative; I have a suspicion it might be impossible. The middle one is somewhat less speculative; something akin to it is probably possible, but there are degrees of success and you're probably looking at more like "eats organic matter at a foot a day" than the "lol eats planet in minutes" sci-fi shit. The first one is proven possible by PNA, the aforementioned terribility of RuBisCO, and the wide variety of possible biomolecules only some of which are used. Anybody who knows second-to-third-year biochem knows that that design is 100% chemically and physically possible; the roadblock is the incredible difficulty of designing a full biochemistry ex nihilo (it'll be a while before anyone succeeds at this without AI aid, although I'd still rather nobody tried). I get that not everyone does know this, but seriously, this is uncontroversial in terms of "is this possible, given a blueprint?"; it is. That's why I said it's the best-case of "what the final form of bioweapons looks like"; they can be worse, but they can't be better.
I mean, I'd rather that 200 million people die next year from a pandemic over everyone dying 10 years from now. I'd rather that even if I'm one of the 200 million. I'm not seeing the issue.
The issue is that you are prioritizing problems that are arguably possible (well, one of them) but have never manifested in an even directionally similar way over one that just happened a few years ago, repercussions of which were quite severe and still being felt.
I resisted "millenarian cultist" analogies so as not to be uncharitable, but you didn't want to talk about Ford Pintos, so fuck it:
It's certainly possible that Jesus will descend and start casting the goats (that's you) into a lake of fire at any moment -- this is roughly the worst thing that could happen (for you); shouldn't you prioritize Christian worship more highly than (I assume) you do?
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link