This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Neuroscience still has a lot of ground to cover, but we already know the brain isn't a binary computer. It seems to me that one very easily could be a materialist and think that the brain is not a computer and I've always been a bit puzzled by the consistent tendency to equivocate them.
The claim isn't that the brain is a "binary computer", it's that it's that however the brain works, it does not have computational capabilities that go beyond what is expressible by a Turing machine. So far we haven't been able to come up with a physical system of whatever sort that everyone agrees is able to come up with results that something like a digital computer can not even in principle. Roger Penrose does think that the human brain is one of those, and some mathematical insights humans can have are literally examples of super-Turing computation, but most everyone else thinks he's being a crank about this.
Your link says
It then goes on to explain that, arguably, "everything is computer."
Perhaps the human mind is a computer in the sense that everything is, but there doesn't seem to be good evidence that it is a computer in the sense that the metaphor is helpful to understanding the human mind. The human brain does not create representations of stimuli, store them, manipulate them, and retrieve them later upon demand according to a series of algorithmic rules.
Perhaps the human mind can't perform any mathematical calculations that cannot be performed by a Turing machine, but that doesn't mean that saying it is a computer is a helpful analogy. A digital tape recorder can record any song that a record can, but it's not helpful to call a record player a computer either - the mode of operation is different.
While I am sure that "not everyone agrees" my understanding is that it seems pretty clear that the universe, itself, is not simulable.
That's the thing. People didn't decide a priori that "everything is a computer". People just went looking for things that can't be mapped into computers all over nature and never found one.
This is pretty much what the debate comes down to though, remember the original argument was about whether we should expect AIs to surpass humans in everything humans can do. People keep trying to claim that humans have some magical domains of competence that will remain out of reach of AIs. For this to be an useful argument against claims of AI doom, it needs to cash out as the human mind doing some sort of work that shows up as output in the world, like a symphony or a beautiful masterpiece on a canvas. The theory of computation is very different from actual computer engineering, and the Aeon magazine writer seems to not understand this. It doesn't say anything about bytes, files, subroutines, operating systems, databases, images or buffers, just that there is some finite-length (but probably very long) lawful process that generates the speech or movement that shows that the thinking happened, and that the process could be translated to be run by a Turing machine.
I'm not a theoretical physicist but I'm pretty willing to bet that a physics paper that appeals to Gödel's incompleteness theorem for wide-ranging claims about the ultimate nature of reality will not end up receiving wide scientific agreement. The Gödel argument is basically the same thing Roger Penrose goes on about, and it goes back to John Lucas in 1959. It's had plenty of time to convince people and as far as I understand it by and large hasn't done that.
Apparently a previous reply was eaten, my sincere apologies if this ends up a double-post.
The fact that "people" latch on to an easy metaphor does not necessarily indicate that the metaphor is good. The fact that the people most familiar with computers latch on to this metaphor also does not necessarily indicate that the metaphor is good.
This wasn't my claim, though.
The Aeon author did tackle the idea that the mind is an algorithm, which is, as I understand it, part of the theory of computation. We have good reasons to think the brain does not run on an algorithm; as the author of the piece I linked to points out, memory is extremely inexact, which is the opposite of what we would expect if the brain operated in an algorithmic manner.
But to take a step back, even if we wish to draw a distinction between "computer as hardware" and "computer as information processing device" the linguistic overlap invites us to confuse the two. And I don't think this is good; the analogy breaks down quickly in practice and invites us to forget the massive differences between the brain and electronic computers; it's true the brain uses electrical impulses but it also uses chemicals and is much slower than a computer. This metaphor, turned loose into the wild, has led to the popularization of what should be obviously implausible ideas, such as "mind uploading" or even that a computer could have emotions that we know in humans are substantially influenced by hormones.
In short, the idea that the mind is a computer is a sloppy one even if the motte is more defensible than the bailey by far precisely because the word "computer" makes it inherently a metaphor that yields a motte-and-bailey, even subconsciously.
I am not a theoretical physicist, or a mathematician, or a neurologist, but I am pretty sure you are wrong.
As I understand it, it works something like this. Gödel's incompleteness theorem says you can't algorithmically "solve" math (in the sense that there's not a super-algorithm that can do all mathematics). Penrose said "aha but humans can so we're BETTER THAN TURING MACHINES." The skepticism of Penrose isn't that Gödel is wrong, it's about whether or not humans can do that. If Gödel's incompleteness theorems suggest that our universe isn't a simulation, that's a different line of argument.
Yep, this is a much less prone to confusion way of saying it than "the mind is a computer".
And this is utterly confused. Douglas Hofstadter's cartoon illustrated the error pithily way back in Gödel, Escher, Bach. The algorithm is exact (the small, correct sums in the Hofstadter cartoon), but it's also too precise and constrained to do mind-like stuff directly in the small. Instead, the mind runs on a sort of virtual machine (big numbers built from the small sums in the cartoon) built up by the algorithm that can do complex pattern recognition and creative solutions, but is also constantly getting things wrong. As we see from AIs, virtual machines like this can be implemented on silicon just fine and they exhibit the same behavior of being able to do difficult useful stuff but also constantly getting details wrong on their own.
I sorta agree here. It's basically an accident of history that "computers", things with hard drives, keyboards, operating systems, files, RAM and CPUs, and "computation", the evaluation of primitive recursive mathematical functions which matches what a Turing machine (which, again, isn't a "machine" that you build from wires and bolts, but a mathematical construct), ended up using the same terminology up to "computer" being right there in the name "computer science". This is why the cognitive science school is called "computationalism" instead of "computerism" and the practitioners optimistically thought that given a name like that, obviously people would think Turing machines, not quad core Mac Pros.
The problem with Penrose's argument is that humans are doing math pretty much as you'd expect if constrained by Gödel. By stumbling into theorems, working hard trying to prove them, and sometimes finding themselves stuck and unable to show something as either true or untrue. The crackpot smell with the physics paper is that Gödel's theorem is ultimately pretty limited. It says that any formal system powerful enough to do any sort of interesting math in allows stating the equivalent of the liar's paradox, which cannot logically resolve to be either true or false, therefore you can't have a mechanism for determining the truth of any proposition because you have liar's paradox propositions floating around. The equivalent impossibility theorem for computer science is the halting problem, you can't write a program that looks at the source code of any program and tells whether the program will terminate. For simulations, this would be saying something like that you need to actually run the simulation to see what kind of state it ultimately ends up in (and whether it stops at a steady state or goes on forever), and can't just look at the simulation's source code and figure it out. But it doesn't prohibit running the simulation and looking at what happens in it while it's running.
Even assuming the article is correct, I'm not sure it'll tell us anything useful about human capabilities versus silicon. Halting problem style arguments do claim that we can't build a literal machine-god that can figure out the exact trajectory of our universe ahead of time just by thinking hard. But that's not necessary to have machines that are better at doing everything humans value doing.
This is a possible explanation, but as far as I can tell, not a necessary one, except inasmuch as one could stretch the word algorithm - which carries a connotation (or perhaps definition, if you cherry-pick one) of precision and repeatability - to encompass any process - although perhaps we are talking past each other here. Certainly the brain has deterministic aspects. But because it's a physical organ, it doesn't seem to behave algorithmically. Even if there is an underlying algorithm (and certainly I imagine there's an underlying process or, more properly, series of processes) it's so confounded by biological processes that I still have qualms about the word choice.
Yes, I think that's right. I brought it up because the universe is a physical system that can do things an algorithm can't.
Yes, and I am much more irritated by the former sorts of arguments than the latter sorts of arguments.
My personal take is that AIs are likely to continue to be "spiky" in their intelligence for the near future but that's not because of abstract beliefs so much as it is just observing their overall trajectory and what I know about how they work. There will probably always be things that humans are better at doing, but I think that is a claim I can make with some confidence because humans like doing things like procreating, not because of Gödel's incompleteness theorem. Even if Penrose is right, it doesn't seem to me like it tells us much about the capabilities of silicon in most practical matters.
No, I think this makes sense. But it's not the crux of the argument. The virtual machine approach comes down to something like chaos theory. You can have a fully deterministic system, like a mathematical model of the weather, and still can't predict where it ends up without running a simulation. The inputs and inner workings of a mind-like system are so high-volume that you don't get clean and complete predictability at the high level where you deal with things like everyday concepts and decisions. There's noise. Humans err and LLMs hallucinate. But the system substrate is still fully deterministic for LLMs, and neurons firing for humans might doesn't need any inherent fudge factor for humans either for the firings to add up to doing stupid or confused stuff for the human.
The interesting arguments for human specialness usually postulate some sort of qualitative difference for humans and the people who posit them tend to not sound like they'd agree that you could run something that behaves more or less exactly like a live human in a large enough computer simulation. Nobody is arguing that humans are algorithm-like in that they don't make sloppy errors when responding to high-level stimuli. People are arguing that humans are simulatable given a big enough simulation, which means that replacing humans doesn't require God's divine power to imbue an immortal soul in crude matter that allows it to sense and reason, but might instead be done just by coming up with a clever way to compute a very large number.
Humans are going to keep being better at conceiving babies and nursing them, sure. But will they stay better at something that keeps them from being killed by Skynet or being reduced to animals who mostly just eat, shit and fuck because all productive work and scientific thinking is being done much better by machines?
I don't think you need a very big simulation to simulate people, even, at a high level or a very granular level I'm sure it's doable with a fairly simple algorithm. Obviously the more detailed you get the more detailed and perhaps more stochastic your algorithm needs to be. I don't see any reason in principle that a sufficiently detailed dataset couldn't model human behavior at a very high level of fidelity.
What I am more skeptical of is that any such simulation works in the same way that a person does. (I am also skeptical that you'd be able to create a perfect simulation in the sense that it would be able to predict a human with unerring accuracy, but if your algorithm was stochastic then of course you wouldn't be able to predict a simulated human with unerring accuracy, either). This matters because people have a tendency to think that something capable of (or surpassing) human behavior must have other human characteristics. But I don't think this follows. We don't fool ourselves into thinking that AlphaGo or Claude need to eat, and we should be wary of ideas that cross in the other direction, too, even if our simulations are really good!
So no, I don't think you'll be able to "replace humans" any more than Excel replaced accountants or cars replaced horses (pick which analogy you prefer based on your projections of where AI ends up). A fully functional replacement human would be a human.
I suppose we will see. I think LLMs are very impressive and I am rather skeptical of the idea that they will take all the jobs. But that's because I have some familiarity with them, not because "humans aren't computers" - there are lots of machines, including plenty of non-computers, that are better at doing given tasks than humans.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link