This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

Jump in the discussion.
No email address required.
Notes -
Humans existing and being good at these problems shows that it is possible to create an intelligence that can solve these problems to at least the skill level of a highly intelligent and competent human, without needing impossibly huge training sets to do so. The question is if we can replicate this on a computer. The bull case is that this is just a question of finding the right algorithm, and once we do, we will achieve AGI.
Since current AI can clearly help researchers write code faster, it stands to reason that the better AI we have access to, the faster we can improve the algorithm, which leads to a loop where better models are developed faster and faster. Once the models start approaching human-level intelligence they will be able to iteratively improve themselves without researcher oversight. And like that, we have justified drawing lines on the graph.
It doesn't assume that -- it rests solely on the idea that brains are physical objects. This is empirically verified by every single experiment run on a human brain. More generally, it's been borne out on every noun that interacts with the physical world.
"Humans aren't computers" is irrelevant. Brains are physical arrangements of atoms that are capable of intelligently solving problems. This assumes nothing.
(For completeness: you may be completely right about 2. You're sort-of-right about 3, in that the assumption was made and the assumption was mistaken. But I don't think you're right that the current approach avoids singularity. There are absolutely recursive feedback loops in improving the current implementation of AI, because improving AI is made out of tasks, and we can get AI to do tasks. But you're right that the original thesis had a much more directly integrated feedback loop.)
The human brain is made up of very different materials than a datacenter. It is entirely possible that the physical structure of the brain is necessary to create intelligence, and that this structure requires materials which certain properties. Maybe a digital simulation will just always require orders of magnitude more data and power than the real, physical thing.
More options
Context Copy link
This is missing the part where the human brain is an exceptionally well-tuned physical object shaped by millenia of evolutionary pressures that arguably constitute a training set vastly bigger than the laws of physics as we currently understand them say is possible to match with an artifical model, much less do any meaningful computation with.
It is also missing the part where the human brain is the most complicated object in the universe, as it is the only currently known object capable of of understanding these questions well enough to even ask them. And even it does not fully understand itself.
Not true: a group of human brains, or a human + tools|AI, or humans + tools|AI, are smarter and more complicated.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I think your points are good, and I am myself a bit of an AI sceptic. But I do see where the AI safety crowd is coming from. It may not be particularly likely that we get AGI in the near future. But the fact is that the possibility is there, and is significant enough that it currently cannot be dismissed out of hand. Thus it makes sense to halt development until we are certain that this research won't doom us all.
I find it ironic that this is the logic used by a group that pretty much universally rejects Pascal's Wager. Also, it wouldn't be the first time humanity has made this particular calculation- when the first atomic bomb was tested at Trinity, Oppenheimer was "pretty sure" it wouldn't cause a neutron chain reaction and ignite the atmosphere in a nuclear hellstorm, but he couldn't guarantee it. Infinite stakes do not necessarily require infinite caution.
Even if you take Pascal's wager seriously, it is not actually very useful. There are multiple religions that each claim their god created the world, with most of them being mutually exclusive. Thus Pascal's wager works about as well as an argument for believing in the Christian God as it does for believing in Allah.
Regarding the atomic bomb, they did the math which showed that a chain reaction was impossible prior to the test. We have no such proof against the dangers of AI. The equivalent would be a paper that shows the theoretical limits of how intelligent LLM's can get, and thus prove that the line will stop going up before we reach the point of AGI.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I recall the excellent Westworld, Season 1 (and deny that anything else came after that) that the dividing line of sentience is a mostly illusory one: that it is a emergent property of the self-concept, of the internal monologue. That notions of a soul may either be chauvinist hubris: or perhaps God will endow them with one, as Providence dictates.
Since there's no way to ascertain that any individual has consciousness from without for certain, we have to extend the benefit of the doubt to our fellow human beings. Is it possible for superintelligent AGIs, on the line of Helios from DEUS EX? Uncertain. But I am fairly certain that LLMs will reach human capacity in my lifetime, or at the very least reach a level of sociability that it will be monstrous to treat them any less than equals. If the technology stalls out at that level it will still very be much worth it: I will reserve at least 16gb of vram for my new friends.
Ultimately that's really the point of Turing's Imitation Game. It was not to be a real serious test to use as a measure. It illustrates that we are not even able to discern sentience in other humans, we just assume it, and that if we afford the same leeway to machines, we will eventually end up with machines that have just as good a claim to it as other humans do to us. And as early as ELIZA, once it was clear machines could manage grammar and human language, it was obvious that eventually, without even needing a real paradigm change, we'd end up with machines that would be capable of fooling us.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link