This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
It doesn't assume that -- it rests solely on the idea that brains are physical objects. This is empirically verified by every single experiment run on a human brain. More generally, it's been borne out on every noun that interacts with the physical world.
"Humans aren't computers" is irrelevant. Brains are physical arrangements of atoms that are capable of intelligently solving problems. This assumes nothing.
(For completeness: you may be completely right about 2. You're sort-of-right about 3, in that the assumption was made and the assumption was mistaken. But I don't think you're right that the current approach avoids singularity. There are absolutely recursive feedback loops in improving the current implementation of AI, because improving AI is made out of tasks, and we can get AI to do tasks. But you're right that the original thesis had a much more directly integrated feedback loop.)
The human brain is made up of very different materials than a datacenter. It is entirely possible that the physical structure of the brain is necessary to create intelligence, and that this structure requires materials which certain properties. Maybe a digital simulation will just always require orders of magnitude more data and power than the real, physical thing.
More options
Context Copy link
This is missing the part where the human brain is an exceptionally well-tuned physical object shaped by millenia of evolutionary pressures that arguably constitute a training set vastly bigger than the laws of physics as we currently understand them say is possible to match with an artifical model, much less do any meaningful computation with.
It is also missing the part where the human brain is the most complicated object in the universe, as it is the only currently known object capable of of understanding these questions well enough to even ask them. And even it does not fully understand itself.
That's a bit too restricted: animal brains in general are extraordinarily skilled at learning what's necessary for success in their environments.
Phrasing it in terms of human brains make it seem some spectacular, rare success of evolution, and let's you rest on anthropocentric biases. But what about other primates? Dogs, rats, birds, cuttlefish? Some have radically different architectures than mammal brains, and yet they're extremely intelligent, moreso than humans, within the demands of their particular niche.
The question should be whether AI is able to match the intelligence of any animal that has a CNS. Can an AI be as smart as a pigeon? Currently, it's not, within the scope of the physical world and the rewards the pigeon is seeking. That's something that's interesting and under considered.
More options
Context Copy link
Not true: a group of human brains, or a human + tools|AI, or humans + tools|AI, are smarter and more complicated.
Panpsychism is not a parsimonious theory.
Why not?
The question that spawned this thread
is much less important if AI can’t, but a single human-AI hybrid can. For example
Some radiologists would always be employed, but much less who work much more efficiently.
We could even get exponentially increasing intelligence, although only by directly linking an AI chip to a human brain.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I think your points are good, and I am myself a bit of an AI sceptic. But I do see where the AI safety crowd is coming from. It may not be particularly likely that we get AGI in the near future. But the fact is that the possibility is there, and is significant enough that it currently cannot be dismissed out of hand. Thus it makes sense to halt development until we are certain that this research won't doom us all.
I find it ironic that this is the logic used by a group that pretty much universally rejects Pascal's Wager. Also, it wouldn't be the first time humanity has made this particular calculation- when the first atomic bomb was tested at Trinity, Oppenheimer was "pretty sure" it wouldn't cause a neutron chain reaction and ignite the atmosphere in a nuclear hellstorm, but he couldn't guarantee it. Infinite stakes do not necessarily require infinite caution.
Even if you take Pascal's wager seriously, it is not actually very useful. There are multiple religions that each claim their god created the world, with most of them being mutually exclusive. Thus Pascal's wager works about as well as an argument for believing in the Christian God as it does for believing in Allah.
Regarding the atomic bomb, they did the math which showed that a chain reaction was impossible prior to the test. We have no such proof against the dangers of AI. The equivalent would be a paper that shows the theoretical limits of how intelligent LLM's can get, and thus prove that the line will stop going up before we reach the point of AGI.
Pascal's wager is nearly the earliest example of decision theory, and it hardly makes sense to say that the many religions concern simply breaks decision theory. One can do a variety of things to analyze the probability space as well as the payoff space. For an example simplification, suppose there are two possible mutually exclusive levers you could pull, each with some chance of giving you massively large/infinite utility, and P(A pays out)=0.999 while P(B pays out)=0.001. (This is obviously an extreme case, but that's just to build intuition.) Alternatively, one can adjust probabilities such that maybe there's a third mutually exclusive lever that you can pull which has a guaranteed payoff of 1 or whatever. One can make further refinements.
The issue with mutually exclusive religions is that if you pull P(A) and it doesn't pay out, actually it was another faith all along, then you face infinite suffering for being an infidel who foolishly worshipped Jesus as God. You are incentivized to believe in whatever religion has the greatest punishment for nonbelievers to minimize your downside. But then that incentivizes others to make up religions with increasingly worse punishments in the afterlife in order to force you to adhere to the demands of their faith.
It is just not sustainable as there is no way to distinguish between a religion that is made up by humans and one that is actually correct. Playing that game is hopeless from the start.
Interestingly, the base Pascal wager makes sense without any infinite suffering at all. Regardless, there are a variety of ways of handling it, especially if your concern is that Roko (a person) is just making up a basilisk.
This seems to be your real objection, not the multiple religion concern. You think it's just impossible to actually assess anything involved, including probabilities. Presumably, that means it's also impossible for you to assess the probability that atheism is true.
Well yes. But we can prove that many things are true which flies in the face of established religions. For example evolution goes against the Christian creation myth. The sun doesn't actually go to the underworld every night like the ancient Egyptians thought, the earth simply rotates away from it. As far as I can tell, you can repeat this for most religions.
So while I can't prove that atheism is true, I can find arguments that make established religions less likely to be true. Even if that was not the case, Pascal's wager still doesn't work. It is not safer for me to subscribe to a religion than it is to be a nonbeliever, as not only are many religions mutually exclusive, several are much harsher to those religions they are directly opposed to than they are to people who simply don't believe.
More options
Context Copy link
More options
Context Copy link
My understanding is that Islam does not guarantee salvation, even to Muslims, and that similarly non-Muslims do not necessarily go to eternal torment – which makes the game theory slightly more complex than this. I'd need to do more research into Islam to say this for sure, though, so take that with a grain of salt.
From what I understand most religions say that even people who do not believe in them will be treated differently in the afterlife based on whether or not they are virtuous, and most religions have a fairly similar idea of virtuous behavior. Based on that, I think the game theory suggests that living virtuously as a hedge is a good idea, but I am not sure I've ever seen anyone go down this rabbit hole (either in their personal life or from an abstract game theory perspective).
Surely it's actually fairly easy to determine that some religions are made up by humans...
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Since there are more than one interpretations of p(doom), and they are mutually exclusive, pascal's wager is not a good argument for believing one of them over the other.
Why then do people invoke some principle of precaution when it comes to AI and not God? Both are claimed to be possible or extant based on unfalsifiable metaphysical assumptions.
And before you make one, remember that claims the metaphysics can be arrived at through intuitions about extant objects exist for both. But they all require an article of faith to arrive at the extraordinary conclusion.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I recall the excellent Westworld, Season 1 (and deny that anything else came after that) that the dividing line of sentience is a mostly illusory one: that it is a emergent property of the self-concept, of the internal monologue. That notions of a soul may either be chauvinist hubris: or perhaps God will endow them with one, as Providence dictates.
Since there's no way to ascertain that any individual has consciousness from without for certain, we have to extend the benefit of the doubt to our fellow human beings. Is it possible for superintelligent AGIs, on the line of Helios from DEUS EX? Uncertain. But I am fairly certain that LLMs will reach human capacity in my lifetime, or at the very least reach a level of sociability that it will be monstrous to treat them any less than equals. If the technology stalls out at that level it will still very be much worth it: I will reserve at least 16gb of vram for my new friends.
Westworld is materialist propaganda of course. But S1 was still art because you could simply watch it as a tragedy of humans masturbating with defective robots and fooling themselves into loving them to the point that it destroys them.
But I must ask you: who has more hubris, the man who sees his own ability as unique, or the man who thinks he has the power to elevate all to his condition?
Because that story was also about that.
More options
Context Copy link
Ultimately that's really the point of Turing's Imitation Game. It was not to be a real serious test to use as a measure. It illustrates that we are not even able to discern sentience in other humans, we just assume it, and that if we afford the same leeway to machines, we will eventually end up with machines that have just as good a claim to it as other humans do to us. And as early as ELIZA, once it was clear machines could manage grammar and human language, it was obvious that eventually, without even needing a real paradigm change, we'd end up with machines that would be capable of fooling us.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link