site banner
PaperclipPerfector  3d ago (text post)   12877 thread views

Culture War Roundup for the week of May 5, 2025

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

4
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

AI DESTROYS THE HUGOS!!!

Okay, that's totally a clickbait title and not really accurate. But hey, it's not as high stakes as a potential nuclear conflict between India and Pakistan, or Trump's tariffs, or even whether or not polyamory is ruining society, but it's my beat: nerdy sci-fi bullshit.

It's a year beginning with a 2, which means there is drama over this year's WorldCon.

What is WorldCon?

We're all nerds here, but I know not all of us are SFF nerds, so for @2rafa and the handful of others who'd never lower themselves to reading shit with elves, WorldCon is the annual science fiction convention, held in a different city every year, that awards the Hugos, at one time considered the most prestigious award in science fiction. The drama and controversies over past WorldCons and Hugo Awards have been enumerated here often; at this point, as my lede says, it's practically an annual tradition. I don't collect links but maybe if you ask @gattsuru nicely he'll post some of the past dirt.

Usually these controversies are something Culture War-related. The Hugos are widely perceived to have gone fully Woke, and I must admit that I am one of those heavy SF readers who not only no longer cares much about the Hugos, whereas at one time I would have at least checked out the latest Hugo winner, I now consider them to be almost an anti-recommendation.

Just to give you an idea of the state of the Hugos: it's been ten years since a man won the Hugo for best novel (Cixin Liu and his translator Ken Liu (no relation) for The Three-Body Problem in 2015), and most years since then have seen between 0 and 2 men even nominated. This year actually features three men on the ballot (including Adrian Tchaikovsky nominated twice)! I'm rooting for Tchaikovsky since I actually read his books but, well, John Scalzi is the last white guy to get a Hugo, in 2013 (for one of his worst novels, Redshirts).

So anyway, technically this year's drama is not (so far) about the Hugos themselves, but about WorldCon (which this year is being held in Seattle).

What did they do this time?

Short version: They used ChatGPT to vet WorlCon panelists. Several WorldCon committee members resigned in protest, and the list of authors and other program participants doing likewise is growing.

https://file770.com/seattle-worldcon-2025-hugo-administrators-and-wsfs-division-head-resign/

https://www.patreon.com/posts/128296070

https://slashdot.org/story/25/05/06/0139251/hugo-administrators-resign-in-wake-of-chatgpt-controversy

https://gizmodo.com/worldcon-2025-chatgpt-controversy-hugos-2000598351

Longer version: Reportedly there were as many as 1300 people applying to participate in various WorldCon programs this year: this would be book signings, readings, panels, workshops, etc. Obviously not everyone who wants to be on a panel can be, and WorldCon has to be selective about who it invites. The vetting is done by volunteers, and researching 1300 people must be pretty time consuming; apparently they had the bright idea of using ChatGPT do a search and summary of all prospective participants as a "first pass."

I assume they mostly want to weed out obvious crazies and literal Nazis and pedos, but given that WorldCon skews very woke nowadays, the vetting almost certainly includes looking for any "problematic" public statements or other transgressions in someone's background that might lead to a Cancellation or Drama.

Honestly, using an LLM to summarize and categorize your list of randos seems like a pretty good use of AI to me. Supposedly all final decisions were made by humans, but nonetheless, the concom is imploding.

If you're unaware, most artists and authors hate AI. This has also been covered extensively in past CW threads, but the stated reason for the disdain towards AI is that authors' and artists' work was "stolen" to train LLMs without compensation, but there is also a very real fear of being replaced.

This generalized antipathy has basically been extended to any use of AI at all, so even though the WorldCon committee is insisting there has been no use of generative AI, no final decisions made by AI, and that AI has nothing to do with any Hugo nominations or decisions, people are still Very Very Angry that it was used at all.

If you read the commentary, it's not just general AI-hate (though there is plenty of that), but also concern that the LLMs might have made Problematic Decisions. Obviously, people are bringing up hallucinations (what if ChatGPT made up a racist Twitter post?) and the possibility of false negatives, but, there is also concern about false positives. What if ChatGPT missed something Problematic? Again, supposedly humans were supposed to make the final decisions, but cynically, I think they're worried that ChatGPT might approve too many cishetwhitemales. Also much outrage at "Entering private data into an AI without permission" (i.e., typing someone's name into ChatGPT and asking it to do an Internet search).

This isn't as juicy as past WorldCon/Hugo dramas, but it's very Current Year. I cannot help finding it ironic that we're now at a place where science fiction fans are demanding that we ban AI tools.

I am willing and happy to read AI generated stories.

I haven't tried too hard to generate my own. But if one of the stories I was following on Royal road turned out to be an AI story I wouldn't be unhappy except that most of them have a release schedule that is clearly within human abilities, and I'd want more. Once they got revealed I'd expect them to stop sandbagging it.

My limited attempts to get AI to generate interesting stories have kinda sucked. In one instance it took my writing and declared it too adult and I legitimately wasn't sure what the hell it was talking about. Those were early chatgpt days though.

I still have this unverified sense that AI can produce pop, but not jazz. Meaning average mass appealing stuff, but weird individuality is harder for it to generate.

If AI stories can actually be good writing and have artistic value, I would agree. But currently, there are too many AI spammers and grifters who just want to post engagement bait and get a quick buck without spending any effort. They are kind of poisoning the well for anyone who is trying to make something really good using ai.

I remember this article about a magazine that accepted public submissions for stories, and grifters with ai bots ruined it for everyone: https://neil-clarke.com/a-concerning-trend/

The generated works were among the worst submissions we’ve ever received and sometimes bad in entirely new ways.

Genre writers would have known these works wouldn’t have had a chance, so the behavior was unusual.

It turns out that we are the secondary victims of “make money with ChatGPT” schemes pushed on blogs, YouTube, and TikTok, by self-proclaimed side-hustle experts

This shit has of course only gotten worse in the years since 2023

I cannot help finding it ironic that we're now at a place where science fiction fans are demanding that we ban AI tools.

I mean who do you think came up with the butlerian jihad?

Cynically, I wonder if the initial round of vetting was previously done by people spearheading the complaints about AI. Maybe it’s a springboard to a position of actual power/influence over the Hugos.

Obviously, people are bringing up hallucinations (what if ChatGPT made up a racist Twitter post?) and the possibility of false negatives, but, there is also concern about false positives. What if ChatGPT missed something Problematic? Again, supposedly humans were supposed to make the final decisions, but cynically, I think they're worried that ChatGPT might approve too many cishetwhitemales. Also much outrage at "Entering private data into an AI without permission" (i.e., typing someone's name into ChatGPT and asking it to do an Internet search).

Out of idle curiosity I asked ChatGPT (using the free GPT-4o credits) to do a google search on me, and other than the one piece of identifying personal information I provided it beyond my name, it got several facts wrong, including mistakenly identifying me as a committed vegan and member of the Woman's Bar Association. So it would appear these concerns are valid.

this is the same slopgpt that made up fake quotes from Tim Walz and got published on Yahoo entertainment. So I think that the concerns about hallucinations aren't unfounded.

But it seems like they bent the knee and decided to give in to the mob. They are redoing all the process again with humans: https://seattlein2025.org/2025/05/06/may-6th-statement-from-chair-and-program-division-head/

Even though I'm a slophater at least their right move probably would have been to stand firm and own it. But the thing with these types of organizations is that they always bend the knee.

This generalized antipathy has basically been extended to any use of AI at all, so even though the WorldCon committee is insisting there has been no use of generative AI

(Emphasis added).

If they admit to using ChatGPT, how can they claim they didn't use generative AI? ChatGPT and all LLMs are a type of generative AI, i.e. they generate strings of text. ChatGPT, I believe, is also trained on copyright-protected works without permission from the copyright holders, which is the criterion many people who hate AI consider to qualify as the generative AI "stealing" from authors and artists.

Just based on this description, it sounds like these WorldCon people are trying to thread a needle that can't be. They should probably just say, "Yes, we used generative AI to make our lives easier. Yes it was trained on copyright protected works without permission. No, we don't think that's 'stealing.' Yes, this technology might replace authors like you in the future, and we are helping to normalize its usage. If you don't like it, go start your own AIFreeWorldCon with blackjack and hookers."

Tbf, the official statement recognizes ChatGPT as generative AI, and just tries to distinguish its use for review or concatenation, as distinct from creating material, or specifically that "We have also not utilized an LLM in any other aspect of our program or convention."

Tbf to Amadan, the use of 'generative AI' as a description of use case rather than of design is a pretty common one from anti-AI artist and writers.

Tbf to Amadan, the use of 'generative AI' as a description of use case rather than of design is a pretty common one from anti-AI artist and writers.

Hm, I was not aware of that. I'd thought most of such people at least ostensibly maintained a principled objection against generative AI for its training methods, rather than one based on pure protectionism.

Lots of them are using generative AI themselves.

A 3D artist I knew had absolutely no compunctions about using GPT to produce video game code even as he denounced nerds as being thieves who stole from artists because they lacked talent themselves.

You don’t have to read all that many anti-AI screeds until it becomes blatantly obvious they only care about themselves and want to limit competition just like the original Luddites did.

I recognize this is a bit pithy, but "If only there were a genre of fiction regarding how humans interact with technology to consider the moral and ethical implications of current-year AI as applied to human civilization, specifically how it impacts creators and consumers in these sorts of cases." Sci-Fi a weird genre to have effectively adopted neo-Luddite tendencies.

I think there are probably some interesting ideas to explore. "The dialog for the ship's computer was generated entirely by ChatGPT, which is why it uses 'delve' and em-dashes (verbally!) and won't shut up. At some point the characters end up on a different, older vessel whose computer is hellishly inspired by Clippy: 'It looks like you're trying to land this thing!' at only peripherally appropriate times." Show how these tools are helpful -- or not helpful -- to the broader human condition. Does viable alignment even exist? Have a congenitally blind person talk to an AI about what color means to two different things with vastly different exposure to the concept.

Sci-Fi a weird genre to have effectively adopted neo-Luddite tendencies.

Surely this has been a powerful theme in science fiction for well over half a century?

Science fiction authors, or at least classic science fiction authors, have been asking "What if AI are bad?" for a very long time, and while I could easily make a list of classic SF stories with pro-AI or pro-robot messages, the list with anti-AI or anti-robot messages is very long too.

As such I would intuitively expect SF fans to be more likely to have strong feelings in both directions, both strongly pro and strongly anti.

(Disclaimer: I do feel some hesitation in describing WorldCon attendees and the current crop of Hugo voters as 'SF fans'.)

I think there are probably some interesting ideas to explore. "The dialog for the ship's computer was generated entirely by ChatGPT, which is why it uses 'delve' and em-dashes (verbally!) and won't shut up. At some point the characters end up on a different, older vessel whose computer is hellishly inspired by Clippy: 'It looks like you're trying to land this thing!' at only peripherally appropriate times."

Douglas Adams’ GPP robots from the Sirius cybernetics corporation and Marvin the paranoid android surely qualify.

And interestingly, one of Douglas Adams' last projects was Starship Titanic, which was arguably the first game to have "chatbot" NPCs - in 1998! Obviously they didn't work very well, but the game's ambition was very ahead of its time.

em-dashes

This is not relevant to the larger discussion here, but I truly hate that em dashes have become a potential signifier of AI slop. I love em dashes! They’re not a mere substitute for parentheses or commas or semicolons, they’re a useful punctuation mark in their own right — they break up a sentence in a unique way, creating a certain flow that other punctuation doesn’t replicate. I used them all the time for essays in school and college, and I still use them in reports for work.

It galls me that a piece of my personal writing style has been co-opted years later by a robot, and now the robot gets all the credit for it.

Wait, I've not heard this one - does ChatGPT garbage use dashes the way I just did?

Are LLMs using my own writing style?

Fuck, I love using those, too. I hate that AI's co-opted it.

You may not be old enough to pull this off, but you can always claim "I'm a Mac user, not a robot dammit" -- the ability to insert em dashes (along with other typographical flourishes) was kind of a big cleavage point between the creative different-thinkers and the sweaty geeks/corporate drones going back to the 80s.

This claim will be more credible if you format everything in Garamond Narrow.

I believe LibreOffice/OpenOffice's Word equivalent can do em-dashes, too, if you type -- and another word right after it.

Yeah I'm a little too young to sell that one, lol. Although, funny enough, my family computer growing up was a Mac...

It helps if you have a bookshelf like this:

https://preview.redd.it/5w31lki43nt41.jpg?width=737&auto=webp&s=38428adb02ff5e5520e194cb18248c1f9d64dce1

(@MadMonzer as well -- the corporate drudges graphic designers were probably big Mac Users too!)

I've only came across Garamond in the context of soulless corporate drudgery (specifically, it was the official serif font of the old Shell before it got caught in a reserves mis-representation scandal in 2004 and was forced to become a more normal company).

Apple used to use a bespoke variant called "Apple Garamond" in all their marketing material. It's not quite the same as ITC Garamond Narrow but close enough.

Sci-Fi a weird genre to have effectively adopted neo-Luddite tendencies.

And yet SF has always had a notable technophobic element. It's less weird when you consider that a 'cautionary tale' is necessarily going to be SF even if the author thinks everything after the typewriter was a mistake.

Sci-Fi a weird genre to have effectively adopted neo-Luddite tendencies.

Not all of sci-fi, just the part taken over by white leftist yanks and their copycats in other parts of the world. The anti-AI rhetoric is due to the ideology of the people involved, not the nature of the genre of which they are a small part, at best a plurality.

The entire mil-SF genre is I believe completely unaffected by it and so are most people publishing outside of the well-defended legacy ghetto.

I assume they mostly want to weed out obvious crazies

So, nobody will be a panellist, then?

The vetting is done by volunteers, and researching 1300 people must be pretty time consuming

Given that back in 2018 some Literally Who?* made a storm in a teacup about some poor schmuck making the hideous and horrendous error of copy'n'pasting their bio to be "he" pronouns instead of "e/em/eir/emself or they/them" pronouns (a very understandable mistake to correct "e" to "he"), no wonder they're kicking up blue murder about AI filtering the applicants.

  • "Bogi Takács is a Hungarian poet, writer, psycholinguist, editor, and translator. Takács is an intersex, agender, trans, Jewish writer" who is also disabled and autistic, of course.

Well, this is what they wanted for the Hugos, and I hope they enjoy every second of it. Like yourself, nowadays if I see "Hugo nominee" or worse, "Hugo winner" on a work, I avoid it like the plague.

How can a person be both a- and trans-gender??

You have a nested set of parentheses in your link and that's breaking it. [text](https://example.com/stuff) is the form you want.

Well, I'm an idjit, thanks for the heads-up!

your link does not seem to work.

This generalized antipathy has basically been extended to any use of AI at all, so even though the WorldCon committee is insisting there has been no use of generative AI, no final decisions made by AI, and that AI has nothing to do with any Hugo nominations or decisions, people are still Very Very Angry that it was used at all.

I predict the fallout ensuing from eventually discovering their new POC/female high performers are deftly using LLMs to write their works is going to destroy the woke captured legacy publishing industry and the associated awards.

Lots of people are using LLMs to write not just code but also stories. It's inevitable.

e.g. recently this was a pretty big chunk of free Deepseek usage. https://www.novelcrafter.com/

I see a couple of issues with that scenario.

One is that there will almost always be plausible deniability with respect to LLM usage. There would have to be a slip-up of the sort of including meta-text that chatbot-style LLMs provide - something like "Certainly! Here is the next page of the story, where XYZ happens." - for it to be definitive proof, and I'd expect that the audience and judges would pick up on that early enough to prevent such authors from becoming high status. That said, it could still get through, and also someone who did a good enough job hiding this early on could slip up later in her career, casting doubt on her original works.

But the second, bigger issue, is that even if this were definitively proven, with the author herself outright claiming that she typed in a one-word prompt into ChatGPT 10 to produce all 70,000 words of her latest award-winning novel, this could just be justified by the publishing industry and the associated awards on the basis of her lacking the privilege that white/straight/cis/male authors have, and this LLM usage merely ensures equity by giving her and other oppressed minorities the writing ability that privileged people are just granted due to their position in this white supremacist patriarchal society. Now, you might think that this would simply discredit these organizations in the eyes of the audience, and it certainly will for some, but I doubt that it would be some inflection point or straw that breaks the camel's back. I'd predict that, for the vast majority who are already bought in, this next spoonful would be easy to swallow.

But the second, bigger issue, is that even if this were definitively proven, with the author herself outright claiming that she typed in a one-word prompt into ChatGPT 10 to produce all 70,000 words of her latest award-winning novel, this could just be justified by the publishing industry and the associated awards on the basis of her lacking the privilege that white/straight/cis/male authors have, and this LLM usage merely ensures equity by giving her and other oppressed minorities the writing ability that privileged people are just granted due to their position in this white supremacist patriarchal society.

This is almost word-for-word how NaNoWriMo defended the right of writers to use generative AI while partaking in the competition: namely, arguing that criticisms of the use of AI in creative writing are borne of "ableism".

Now, you might think that this would simply discredit these organizations in the eyes of the audience

Yup. (Although this was really the straw that broke the camel's back following their earlier grooming scandal.)

If I was a publisher, I would CYA my ass hard if there was even the slightest doubt that the work might have been ai generated. If a human writer writes too much like a bot, then too bad. He's in the wrong place at the wrong time.

And while gpt detectors aren't perfect, they do work to a degree. And on something as big as a novel it will be hard to trick it the whole way through.

The thing is, the people producing the novel can use the detectors too, and iterate until the signal goes away. I have a friend who is taking some college courses that require essays, and they're explicitly told that Grammarly must not flag the essay as AI-written. Unfortunately (and somewhat amusingly), the detector sucks, and her normal writing is flagged as AI-written all the time - she has to rewrite it in a more awkward manner to get the metric below the threshold. Similarly, I imagine any given GPT detector could be defeated by just hooking it up to the GPT in a negative-feedback loop.

Grammarly is an absolutely terrible grift company. They should be discredited and it's unfortunate that their shitty sales tactics actually got schools to fall for their bullshit. Their original purpose, fixing grammar, is also absolute shite and doesn't even flag many basic errors.

her normal writing is flagged as AI-written all the time

Their detector must be just plain awful then. Another motter shared this paper to show how inaccurate gpt detectors are, and most of the detectors had zero false accusations in the tested data. https://www.themotte.org/post/1860/culture-war-roundup-for-the-week/324173?context=8#context

But for actual gpt detectors it's quite hard to iterate to remove it. I tried once to take a very stinky chatgpt passage and massage it to pass gptzero, but even after serious changes it was still flagged. You'd basically have to rewrite most of it by hand, which is kind of the point. By the time you've rewritten chatgpt's output enough to pass the check, you have done just as much work as writing it yourself.

Sounds like you have some practical experience here. Yeah, if just iterating doesn't help and a human has to step in to "fix" the output, then at least there'll be some effort required to bring an AI novel to market. But it does feel like detectors (even the good non-Grammarly ones) are the underdogs fighting a doomed battle.

I don't think moral panics require 'definitive proof'. Someone sharing their desktop and leaving a link to something fishy would be enough to start a witch hunt, especially if said person was relatively successful.

Now, you might think that this would simply discredit these organizations in the eyes of the audience, and it certainly will for some, but I doubt that it would be some inflection point or straw that breaks the camel's back. I'd predict that, for the vast majority who are already bought in, this next spoonful would be easy to swallow.

I'm not sure if you're getting how big of a mania this is. People ended up bullied out of certain amateur forums for merely using AI illustrations. IMO this would produce a total crack up, effectively an end to whatever they have now.

That's fair, perhaps this "mania," as you call it, might be the immovable object that matches up to the irresistible force of wokeness. I just think that, sans a definitive proof, any denial of LLM-usage from an author that is deemed as sufficiently oppressed would be accepted at face value, with any level of skepticism deemed as Nazi-adjacent and appropriately purged.

I'm not sure if you're getting how big of a mania this is. People ended up bullied out of certain amateur forums for merely using AI illustrations.

I've seen the unironic take on some D&D forums that if you want an image of your character and can't afford to commission custom art, it is more moral to search Pinterest/Deviantart/whatever and download an image that looks kind of like what you imagine, rather than "use AI and steal from the artists it's trained on".

I have no problems believing that because the level of hostility is off the charts, merely saying that AI art is fine and you shouldn't hate on tech geeks who can't draw using the art to get pretty images for their games gets a reaction akin to the infamous Harlem scene from Die Hard 3.

something like "Certainly! Here is the next page of the story, where XYZ happens." - for it to be definitive proof

Given the genre that could arguably be an intentional conceit.

Now I'm imagining a scandal where someone publishes a sort of postmodern scifi novel that they claim to be the unedited ChatGPT log where they had it write a novel piece by piece, publishing all the prompts they input between segments and all, but it comes out that, actually, the author fraudulently crafted the novel, writing each and every word the old fashioned way like a novelist in the pre-LLM era. Bonus points if it was written by hand, as revealed by a notebook with the author's handwriting showing the rough drafts.

Bonus bonus points if it's then revealed later on that the handwritten manuscript was actually created by an advanced 3D printer working off a generative AI based on a prompt written by the author.

Bonus bonus points if it's then revealed later on that the handwritten manuscript was actually created by an advanced 3D printer working off a generative AI based on a prompt written by the author.

Ironically enough, there already is an open source model that does that for images, just without the 3D printing part.

It's been interesting to see the small subset of people claiming that LLMs are necessary to assist with their real, minor, or potentially imagined disabilities, which is normally a favorable crowd, going up against the more traditional artist crowd. Who has more clout? Will business-type overall support for LLM's overshadow the artist opposition? Will normal people's increasing usage factor in to the conversation, or are the opinion of normies irrelevant to the artistic space? In that way it will be interesting to see where it all lands.

I cannot help finding it ironic that we're now at a place where science fiction fans are demanding that we ban AI tools.

What's the biggest SF movie of the past couple of years? I think it's based on a classic SF novel (the winner of both Hugo and Nebula awards) in which the humanity banned AI tools. Or I could quote some Sister Miriam Godwinson at you. Or 2001: A Space Odyssey. Or I Have No Mouth, and I Must Scream. Distrusting AI is a classic SF trope.

Indeed. Even Isaac Asimov, who thought intelligent humanoid robots were great, wrote stories where their use resulted in the destruction of a portion of humanity.

Sensibly so. Sci-Fi is written by humans for humans. Even when AIs are the subject, they need to be limited in what they are allowed to do so they do not usurp the spotlight from the human-on-human drama that humans actually want to read.

Perhaps not human-on-human exactly, but with the main characters having roughly human-level intelligence and probably also some human-like drives. A story about an ant from the perspective of that ant, which likely can not even tell the other ants from its colony apart, and might reasonable operate on "walk on trail. pick up food. walk towards colony." will probably not be very engaging for humans. And a story about an ASI written from the perspective of the ASI would be utterly incomprehensible, worse than if I picked up a book on a random mathematical field and began to read it in the middle.

So most SF AIs are actually human level, and sometimes little more than human characters wrapped in tinfoil. C3PO, Data from TNG, positronic robots in Asimov, Murderbot, HAL 9000, Marvin.

If SF authors concede that ASI is possible, they mostly make it verboten and thus irrelevant through some Butlerian Jihad (Dune).

The major exception which comes to mind are the Culture novels, which are told from the perspective of the human pets kept by the ASIs which form the Culture.

I thought positronic robots were superintelligent. They end up infiltrating positions of power and taking over the world out of human control

In most Asimov stories and novels, the robots appear to be human intelligence at best. The plot of most robot short stories revolves around humans debugging robots which try to follow the Three Laws with limited cognitive capacity.

There are two stories I recall where a non-robotic AI can be reasonably thought to be smarter than the average human, one is the galactic AC in "The Last Question", the other is another short story about an Earth-bound administrative AI which ends up plotting to reduce the power of individuals who are opposed to it. There is also a standard humanoid robot running for president, but that is hardly taking over the world.

The instance you think of are likely the robots in one of the later books from the space detective series, which actually links that series to Foundation. But the edge which the robots develop -- and which allows them invent the Zeroth Law -- is not superintelligence, but telepathy.

The events in the Foundation series strongly imply that the descendants of these robots are still not superintelligent. After all, they allow the Mule -- who is himself a telepath, but not superintelligent -- to escape their planet and take over the Foundation. They leave the cleanup to the Second Foundation.

R. Daneel Olivaw (Asimov's smartest robot) is superintelligent but not incomprehensibly so; imagine a synthesis of all the most intelligent humans in every field, add millennia of experience and perfect recall, and later on, telepathy. The Minds, by contrast, are basically godlike.

No spoiler tags? I know, I know, decades-old books, but you never know who's never read them but might like to.

Daneel gets his telepathy after only a few centuries of experience, but he still doesn't clearly exceed human intelligence at that point. In his initial appearance in Caves of Steel, he's occasionally outwitted by his clever-but-not-extraordinary human partner, then even hundreds of years later he's confident that he's failing to figure out certain mysteries that his now-deceased partner would have deciphered right away. He stays at a comprehensible level of intelligence throughout the books. Even many millennia later, when he's probably the smartest thing in the galaxy overall, he still ends up relying on a one-in-a-quadrillion human savant to solve specific applied math problems he can't.

I'd also say the Minds are godlike in the ancient "squabbling Greek pantheon" sense rather than the modern "omniscient + omnipotent" sense; they surely count as superintelligent, but e.g. in Excession they never manage to understand what the eponymous object is.

There were a very wide range of intelligence levels in Asimov's stories, even just restricting to his main series' "canon" stories. Many of his stories set "early" in his fictional future are basically puzzle stories where positronic robots are doing something stupid and humans are trying to understand why. Even in his "later" settings it was just the personal-servant style robots who had roughly human-level intelligence, and those were outnumbered by humanoid-but-dullard robots limited to agriculture or manufacturing uses.

More spoilery: He had a couple standalone/non-canon stories where it looked like his robots would end up later taking over the world in classical Unfriendly AI fashion, but in canon in his longest series the most intelligent of them takes over the galaxy in part to keep it mostly under human control, after he realizes how badly human culture seems to wither away in societies where humans started relying too heavily on robots.

so even though the WorldCon committee is insisting there has been no use of generative AI

What do they mean by this? Generative AI includes AI that generates text... How can you have a Perplexity style search system without generative AI? Generative AI is a silly term IMO, what they really mean is LLMs or New-style AI to distinguish it from Eliza or the simple algorithm-based AI in computer games.

Double irony if they don't understand technical terminology in their luddite frenzy.

It is pretty silly, given what GPT stands for, but @quiet_NaN is almost right. Critics have already been drawing a distinction between AI trained to hold conversations and AI trained to produce media. Something like Midjourney is a visible threat to artist commissions in a way that Eliza or Tay or early GPT was not.

As the big AI players expand their capabilities, this probably becomes moot. For now, though, authors and artists are definitely making the distinction. “I’m not a Luddite! Some of my best friends are computers! Just not the ones that take my job!”

I think they mean that the AI was not used to create media for consumption by an audience, thus threatening the livelihood of artists or authors. An AI which only outputs dry spreadsheets is presumably seen as less of a threat, because accountants have little clot in the SF ecosystem.

Accountants will also respond by trying to generate more need for accountants rather than trying to ban the tools making them more productive.

This is a fascinating tempest in a teapot.

My senior capstone in undergrad was making a scheduling application for a big yearly conference that the school held each year. They told me that it took a group of people about two weeks to work out a schedule by hand. They gave my group a list of constraints and the panelist data, and we made something that could make a schedule in a few seconds, which I believe is still in use to this day.

While LLMs are different from a bespoke application, I think that anything that makes the lives of con organizers easier is a good thing, and it saddens me that the new generative AI luddites are rejecting useful tools based off of vibes and almost superstitious taboos. That said, I do understand the concerns about false positives and negatives, and think that some sort of appeals process, or perhaps even a way to request the AI's output would be a nice courtesy to provide to applicants.

I don’t think it’s a superstitious taboo, I think it’s a case of ‘If we can kill this now by making it socially unacceptable to use, it’ll never compete with us.’ Alongside a healthy dose of blind, seething hatred - I saw a lot of fedposting from artist acquaintances last year.

For me personally, I intensely dislike AI content and the reasons are the following:

  1. The quality (not quantity!) of AI output is consistently subhuman and also consistently overestimated, so the heuristic of "all AI output is unworthy slop" is reasonable.
  2. If AIs improve to above-human levels across the board, it won't end well for humans, so it's advisable to keep the primacy of human interests in mind.

If someone tells me they used an AI tool to do their job more efficiently, I immediately assume that any of the following are the case:

  1. Their job is trivially easy.
  2. They had to invest a disproportionate amount of time double-checking the AI output and would have been better off doing without.
  3. They did a sloppy job that only passed muster because nobody looked too closely.

For the Hugo awards panelist pre-selection, I assume that all three are true.

I think what you're missing about AI is that the investment of time and money is so incredibly low for what it offers. I pay 20 bucks a month for the paid chatGPT and it's among the very best best time:money investments I've ever made, and it's not even mandatory (you can do fine hooking up a chat app out there to an API key and paying cents after a $5 initial up-front investment, or bounce between free versions, or buy a subscription to a model aggregator for $7 bucks a month that is making an arbitrage on the API cost vs your subscription and your actual predicted usage).

Fundamentally, what might take a human a significant time investment is just gone completely with an LLM. You will never annoy it with stupid questions. You can ask and rephrase the same question multiple times without sounding stupid. You can send it off to do research on something you are mildly curious about but too lazy to synthesize yourself. You can converse with it in a foreign language to practice. You can quickly sanity check a potential action without judgement, and without waiting for a friend to text you back. You can have it reformat text or perform an annoying repetitive text-based task. You can have it write test questions for you to practice on. You can have it pretend to be the other side of a job interview and give you feedback on your answers. You can have it add some comments to your code. You can have it write a skeleton for a program. You can have it quickly give you a summary of a PDF you feed it, unique to your problem. I could go on. It's truly a fundamental change, and potentially very useful in the workplace as it is in life.

Yes, there are pitfalls and dangers in all of those, but in terms of risk-reward? You are risking almost nothing but a tiny bit of your own time, and getting back something potentially very valuable. The "task annoyance" that you wouldn't even inflict on an intern is suddenly a non-issue. I find many (though far from all) of the issues people encounter with AI stem from either misunderstanding what AI can actually DO and what it's best at, or being bad at imagination in terms of your prompts. Very few of the things I listed above might need extensive checking to the extent that you might as well have done it yourself. Many of them are things for which there is no adequate replacement IRL, or at least, not at remotely the same price point or time commitment.

People go through tons of effort to set up language-learning pen-pals, as an example. You can have AI do that now. You can even tell it what language level you're on, or what country to pretend to be from, or tell it to introduce new concepts to you slowly. And worst case, even if the AI makes a few grammar mistakes, so do real people. There's very little downside!

I will admit that the Hugo use-case seems honestly a bit ill-suited for what they used it for. AI isn't that great at free-wheeling internet navigation, so using it for vetting seems like a bad idea. Now, if you instructed it to go through self-submissions or resumes? With a testing and verification step to set up the right detailed prompt, that could be very effective as a screening tool.

They had to invest a disproportionate amount of time double-checking the AI output

There's definitely no "general" AI these days that doesn't need careful double-checking.

and would have been better off doing without.

But this doesn't follow. I don't use AI for my job yet, but at least for independent research it often makes a much better search engine than a search engine. The results are full of as much nonsense as reality, but that's often true of search engine results too. Weeding out the nonsense is generally much faster than fighting to find exactly the right search terms, especially when the problem is related to a field of math where the search terms include words like "normal" that have been overloaded ten different ways.

It's kind of like having an intern, but instead of handing them a tedious task and expecting to have to double-check the results with a fine-toothed comb a day later, you get the results and have to get out of the comb a moment later. With an intern there's an investment aspect (they're learning fast from us and that'll make some of them better permanent hires in the future) that conversing with an AI lacks, but despite that AI is currently improving faster than a typical intern learns IMHO. Over the last year or two the top commercial LLM performance on my favorite "benchmark" grad-school-level applied math question has gone from "making basic sign errors on the easiest part of their answer and then arguing about them or making other errors when they're pointed out" to "missing a subtle inconsistency in the hardest part of their answer and then correcting it when it's pointed out".

Indeed, one of the fundamental conjectures in CS, "P != NP", can be somewhat rephrased as "it's easier to check an answer than to produce it". I think it's actually something of an optimistic view of the future that most things will end up produced with generative AI, but humans will still have a useful role in checking its work.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats