Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?
This is your opportunity to ask questions. No question too simple or too silly.
Culture war topics are accepted, and proposals for a better intro post are appreciated.
Jump in the discussion.
No email address required.
Notes -
I'd be curious about your review on Blindsight. I'm rather fond of the book in question, so I'd be interested in reading your take on it.
I'd have to write the review from scratch, but if you want a TLDR:
https://www.anthropic.com/research/introspection
I still wouldn't go as far as to claim that LLMs are conscious, since we're awful at conclusively identifying consciousness in humans, let alone animals or AI, but they seem to possess at least some of the necessary elements.
I fucking hate the Chinese Room, it's an impoverished excuse for a thought experiment with an obvious answer: the room+human system speaks Chinese, even if no individual component does. You speak English, even if no single neuron in your brain does. I find it ridiculous that it's brought up today as if it means anything. The aliens in the story are specifically described as Chinese Rooms, and you can guess what I think of that. If I was writing a full essay, I'd add more about the sheer metaphysical implausibility of p-zombies in general, but those aren't original observations.
If I'm nitpicking (some very annoying nits), the baseline humans and their pet AGIs show suicidal incompetence in universe. You've got hyperintelligent autistic superpredators on the loose? And you let them walk around? Break their spines and put them in a wheelchair while on enough enough oestrogen to give them brittle bones/spontaneously manifest programming socks. The only reason that the primary safeguard was an aversion to straight lines intersecting at right angles is Watts trying to launder in the classical trope of vampires being averse to crucifixes. It's deeply dumb as an actual solution. Also, why didn't the supersmart AI actually do something about the vampire takeover? Are they stoopid?
Summing up: the case for the theories in Blindsight is weaker than at time of publication, even if no one can outright falsify them.
Edit: It's worth noting that I still love the books, it's in my top 10, maybe top 3. I even separate art from the author, I'm not sure if Watts is terminally depressed or terminally misanthropic, but I suspect that the combination is the only thing preventing him from becoming a low-grade ecoterrorist (this is mostly a joke). I still highly recommend it to new readers, as long as they don't overindex from the existential crises.
I think so. I had the misfortune of reading the Rifters trilogy. Book one is great. Can't recommend it enough. Book two is a huge drop in quality, but not horrible. Book three has multiple lengthy portions focused on the sexual torture and mutilation of the only moral character in the series. Raped with jagged broken pieces of furniture; breasts cut off and seared with hot irons; hips dislocated for easier access. It was shocking reading the descent of this author from good writer to the lowest. I'm entirely soured on him at this point. He's capable of good writing. He chose this instead. The weirdest part is the rest of his writing doesn't have this. Just one horrible unforgivable book of sexual torture fetish. I tried googling what the hell happened and found his personal website in which he gets super salty with fans. He's a mean misanthrope who also can write well.
At the end of Blindsight it explains that the AI were in charge the whole time. The AI speaks through the vampire right at the end to explain this by burrowing electronics into his brain and using him as a puppet. Maybe mankind was doomed merely by making the AI. Adding in the vampires certainly didn't help.
More options
Context Copy link
Hah. It's funny; while I do indeed like Blindsight(I still find it a fun read), I've never actually agreed with the take as presented in the book. It's always good to remember that the PhD he has in Marine Biology, and while that gives him some chop, he's nowhere near an actual neurologist, and even when I first read Blindsight, I could poke holes in a few things just on pondering it for a while.
(He says, arrogantly, having neither a PhD or degree focused in matters of the brain.)
In alot of ways, Watts reminds me of Michael Crichton, who would basically stumble across a bunch of neat ideas among various scientific disciplines, autistically obsess over them for a bit, and then stitch them all together into a cohesive whole.
(And I know I just likely threw at least one Peter Watts fan into a vitriolic rage at the comparison.)
And as fun as I find the idea of actual, science-based vampires as the way Watts presents them... he goes far, far too much into passing off thier neurological capabilities as 'fucking magic'. It gets even worse in the sequel, imho.
And if you've ever read any of his other novels, you wouldn't have to guess, you'd know - he's basically a full on, 10/10 raging misanthrop, with a weird obsession about praising China.
Mind, it's been a while since I delved deep into his stuff. Don't know if his attitude has changed any over time. I doubt it.
More options
Context Copy link
I don't know if we read a different edition or something, but I read Blindsight recently and IIRC it mentioned the sleepwalking thing a lot.
I'm probably misremembering. I think I've read the book at least 5 times, but the probably over a year ago.
The point still stands: we have limited insight into the actual degree of consciousness in a sleepwalking state. It's clearly abnormal, but our understanding of neuroscience can't confidently say that since the ability to form longterm memories is largely disabled, that means that consciousness, if present, can't be reported by the sleepwalker later (the same reason you start forgetting a dream as soon as you wake up).
If you've ever lucid dreamed (I haven't, sadly) then that demonstrates the ability to be aware and at least partially conscious during REM sleep. Sleepwalking is NREM behavior, sure, but it's not possible to say that the sleepwalker is entirely unconscious, we just don't know.
Even if they're performing complex motor behaviors, I strongly suspect that overall performance is hampered. They might (in rare cases), drive a car, but I doubt they drive as well as they would fully awake. I could be wrong, but without the ability to subject an active sleepwalker to a battery of cognitive tests, I'll stay here. It's a very tricky subject to study.
I've never lucid dreamed, but I do experience vivid dreams in a REM state and would describe it as a variation of consciousness. I usually forget my dreams, but immediately after waking I can typically remember the whole narrative experience of a dream, which felt real while I was in it. I tend to think of REM as a state of consciousness where sensory input is turned off, things that would set off "this isn't normal" alarms are somehow disabled, and conscious awareness is redirected to... randomness? emotionally unprocessed experiences? fears? Something like that.
With how vivid my dreams are I do experience them as places where I am making decisions, they're just decisions that are enthralled to the content of the dream. Lucid dreaming is the ability to know you're dreaming, and thus control the content of the dream to an extent.
I don't know how that lines up with current neuroscience, but that's my impression of how my own dreams work.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Re: Watts' vampires, they were boring. I did like the nod to "and this is why vampires are repelled by crucifixes" but the rest of it? Oooh they're scary dangerous predators that would murderise us all if they could. Yeah, and so could great white sharks, with their dead shoe-button eyes. We're not going to be murdered by sharks any time soon, and the sentimentality around the way some people treat them accords perfectly well with the stupidity of, as you point out, letting the vampires walk around unfettered. I can easily believe some people would be greedy and stupid enough to think they could make pets out of vampires and use them for PROFIT. But the vampires themselves? There's nothing there, they're just automata. Or sharks, perfect killing machines but no higher goal than that.
Eh, I have mixed feelings on the topic. Watts did his best to rationalize the concept with evobio, but that only gets you so far with vampires. It's kinda cool, but they're far from plausible organisms.
Unlike sharks, vampires are depicted as both amoral/murderous, and more intelligent than us silly humans.
The thing is, they don't roam around entirely unfettered! In-universe, they're recognized as highly dangerous, and mitigation measures are put in place:
The original vampires were highly territorial hypercarnioveres who couldn't stand competition. The resurrected ones had those tendencies ramped up, they were described as murdering each other if allowed to enter close proximity. Think shoving two male tigers into the same enclosure.
Their handlers thought that this instinctual intolerance of their own kind would prevent scheming and conniving. They were very, very wrong. The exact mechanisms by which the vampires coordinated their rebellion are excellent, probably one of the best depictions of the power of decision theories for modeling and coordination. They just imagined what they'd do if in the place of another vampire, and vice-versa, solved for the equilibrium, and acted, independently and simultaneously, without ever having to actively exchange information with their kin. Hats off.
The crucifix glitch was weaponized against them, the belief was that if they went off the reservation, they'd die painfully as soon as the drugs that stopped them from having painful and lethal seizures wore off.
The humans weren't entirely complacent, but they were still unforgivably insufficiently paranoid about creatures smarter than them, which they knew to be hostile by default. The Vampires consistently use their superior physical prowess to murder normal humans, not just their brains.
So why even let them have that physical prowess? It doesn't take a genius to say that "hey, maybe we should give them the grip strength of an obese 4channer". The Vamps were kept around for their brains, not their brawn. It added nothing while making them a greater threat. This is, as far as I'm concerned, giving the humans an idiot ball. The ways the vampires circumvented their other shackles is understandably hard to predict without the benefit of hindsight. Tearing people apart with their bare hands isn't.
The hyper-intelligence didn't really convince me, not as Watts tried to sell it. Sharks don't need to be hyper-intelligent, and neither do the vampires. They're apex predators. They were able to kill their way to dominance over the humans until humans figured out survival strategies. But Watts is the author, and what happens in a book is what the author wants to happen, and since Watts is all in on "humans are dumb and smelly and should all die, horribly if possible, and my wonderful no-consciousness hyper-intelligent murderbots will do that and take over from them, stupid dumb humans ha ha", then that is what we get.
"Imagining what another vampire would do" requires some degree of consciousness, I would argue. But Watts wants to eat his cake and have it, both. If I am imagining what another vampire would do, I first have to imagine that the other vampire also wants to rebel and escape. Maybe not all vampires want that? Okay, they hate humans and are murderbots, let's grant they all want to escape. After that, though: I instinctively know that my vampire-others will all be in places where they can do things in unison so the plan will come off with perfect timing. Well, that's convenient: no single one of the vampires is going to be held up in traffic, as it were? Nice things happen when the universe decides to drop the plot keys into your lap!
EDIT: I forget, how do vampires reproduce, if they can't be in proximity without ripping each other to shreds? Watts must have found some way around it, but I can't remember and can't be bothered to look it up. Hyper-intelligent murder machines that die out in one generation because they can't have offspring because they murder one another if close enough to mate aren't going far.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
The characters do seem to think this but it's not clear what in the story actually supports the suboptimality claim. Sure Rorschach is more advanced than humanity, but that obviously doesn't prove that consciousness is a drag any more than someone taller and balder than you indicates that hair is keeping you short.
The funny thing is, that's not necessarily true. They make a point in the book that the tele-matter drive that allows Theseus and the explorer probes to pull off it's bullshit was enough of a surprise that it caught Rorschach with it's pants down.
So it's technically not a complete one-sided stomp. Score one for consciousness, atleast.
More options
Context Copy link
Rorschach is explicitly described as a p-zombie/Chinese Room, and is used as an existence proof for superintelligence without qualia or consciousness. I struggle to separate in-universe speculation from author fiat, I doubt that Watts is the kind to devote that much screentime to an idea without partially endorsing it.
It's the most technologically advanced entity in Sol, it's doing very well for itself, and all without being conscious. I think that constitutes a claim that consciousness isn't particularly important.
Anyway, after writing this, I had GPT 5.2 Thinking check the version hosted on Archive for direct quotes:
From Siri’s internal monologue near the end (the book’s most on-the-nose anti-sentience passage):
“It begins to model the very process of modeling. It consumes ever-more computational resources, bogs itself down with endless recursion…” � Internet Archive
“Metaprocesses bloom like cancer, and awaken, and call themselves I.” � Internet Archive
“The system weakens, slows… advanced self-awareness is an unaffordable indulgence.” � Internet Archive
“This is what intelligence can do, unhampered by self-awareness.” � Internet Archive
That last line is basically your exact request in one sentence.
In the Notes and References: consciousness as interference, nonconscious competence In the back-matter discussion of consciousness (Watts stepping partly out of “story voice”):
“Consciousness does little beyond taking memos… rubber-stamping them, and taking the credit for itself.” � Internet Archive
“The nonconscious mind… employs a gatekeeper… to do nothing but prevent the conscious self from interfering…” � Internet Archive
“It feels good… makes life worth living. But it also turns us inward and distracts us.” � Internet Archive
“While… people have pointed out the various costs and drawbacks of sentience, few… wonder… if… it isn’t more trouble than it’s worth.” �
It also found a full interview where Watts, out of universe says:
https://milk-magazine.co.uk/interview-peter-watts-sci-fi-novel-blindsight/
The irony being, our attempts at AI are now all circling around "are they really conscious or beginning to be?" because in order to be agentic, they have to want things, and in order to want things, there has to be something there to do the wanting.
Watt's Very Smart Machine is perfectly fine, I don't think machines are or will be conscious. So load it up with programming about being Very Smart and winning at trading on the stock market or whatever, sure. But that's not at all the same thing as "now invent an entire system from scratch to be part of"; the AIs are operating in the system we humans have created. Ex nihilo, why would they care about scientific discoveries or whatever, before we asked them to think about these things?
To get back to the analogy with sharks, I don't think sharks are conscious, or very much if at all. They don't have to be, they're optimised for what they do and they do it so well they really haven't had to evolve with the times. So murder machine vampires? Yeah, sure, why not. Hyper-intelligent, though? Not needed if not conscious and if optimised to be murder machines. Very finely tuned to be the optimum at being murder machines in their environment might look like hyper-intelligence but I think instinct would be doing most of the heavy lifting there.
But Watts wants these creatures in order to show that humans are a mistake, so let him have his toys.
More options
Context Copy link
Rorschach proves that you can be very advanced without consciousness. Does it imply that consciousness carries no benefit, or even carries a harm? Plainly, no.
Sure, if we're being strict about things. But then there's everything else Watt says, which makes me feel justified in saying that was his subtext/implication. He comes out and says so!
While we're on the topic of subtext, the subtext of my comments is that even in this alternative world Watts created in which humanity is powermogged by Rorschach, Watts fails to demonstrate a compelling reason that consciousness would be maladaptive.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I've read a compilation of his short stories, aptly titled An Antidote to Optimism in Polish. I don't think your "mostly a joke" is actually a joke, at least for me.
Huh, I haven't heard of that one before, and up till this point, I thought I'd read pretty much everything he's ever written. Maybe it's even more misanthropic when translated to Polish? You guys aren't known for your sunny vibes and general optimism.
In general, I agree that Watts is deeply, borderline-fanatical levels of misanthropic. I regularly check in on his blog, and a running theme is his sentiment that humans have Wrecked The Planet (ecological collapse, global warming), and we're going to pay for our sins/hubris by quite possibly going extinct. There is such a thing as overstating the seriousness of what is otherwise a real problem. Global Warming is an eminently solvable problem, for very little money should we get over our civilizational allergy to geoengineering. Of course, the idea of using technology to solve things instead of degrowth and industrial regression is deeply antithetical to his worldview. Recently, he's slowly migrating to AI-bashing, which is a very modest directional improvement.
For now, he's busy writing polemics and giving talks at moderately populated scifi seminars. A retired academic in Canada has largely aged out of active terrorism, that's a young man's game.
Oh? Why? Shouldn't he love AI, here is proof you don't need to be conscious to be smart?
More options
Context Copy link
The moment in Blindsight where Sarasti the Superpredator (watch out, he looks at screaming faces to visualize data!) berates our clueless hero for not caring about climate change was absolutely kino.
Did you know that visualizing data in the form of faces is an actual technique?
https://en.wikipedia.org/wiki/Chernoff_face
Making them screaming faces? Subtlety is a lost art.
I realize it's an "actual" technique in that a guy published a paper about it and it's been included in a couple of scifi books. It isn't an "actual" technique in that I've never seen it used anywhere and it seems unlikely that in the future we'll realize we've been sleeping on this.
https://x.com/lauriewired/status/2020006982598685009?s=20
This is the closest I've ever come to seeing usage in the wild, and Laurie claims it's applied by some flavor of analyst. I suppose it's neat?
And even this isn't a face. I guess reality just keeps disappointing.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link