site banner
PaperclipPerfector  1d ago (text post)   794 thread views

Small-Scale Question Sunday for August 17, 2025

Do you have a dumb question that you're kind of embarrassed to ask in the main thread? Is there something you're just not sure about?

This is your opportunity to ask questions. No question too simple or too silly.

Culture war topics are accepted, and proposals for a better intro post are appreciated.

0
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

Suppose we developed a specific form of genetic engineering that allows us to freely modulate a person’s compulsions and interests (for the purposes of this example, say a compulsion for accurately completing taxes). In almost all ways, the resulting human is completely normal, except:

  1. It has the all of the intellectual capabilities of a 99.99 percentile tax advisor, including things like “common sense”.
  2. The modifications have deprogrammed any interest in any other task - friendship, love, travel, sports, television, etc. It feels nothing from engaging in any activity that isn’t organizing and filing tax forms (aside from basic self-sustenance tasks like eating and sleeping or ancillary tasks like learning arithmetic, language, and tax codes), from which it gets a small dopamine high. When not doing taxes, it will go into maintenance mode where it does basic self-sustenance tasks, but otherwise will stare at a wall until the next tax related task. It shows no signs of subjective boredom or of any desire for anything more to life.

To you, would it be ethical to take some eggs/sperm, hire some surrogates, and create a small army of these beings to have them do all of our taxes? How does your position differ from creating an artificial general intelligence (supposing we developed the capability) to do the same?

How does your position differ from creating an artificial general intelligence (supposing we developed the capability) to do the same?

Welllll we haven't assumed the ability to arbitrarily modulate the AI'S compulsions and interests.

Which is a big question these days.

More to the point, though, are we allowing the modulated person to request that their modulation be changed if it no longer suits them, if they feel they're suffering with the current setup?

Unless you're ALSO suggesting that these behavioral changes are SO ingrained that they won't gradually shift over time as they accumulate experiences and/or head trauma.

I think that's where the ethics of it start to kick in. If your modulated human one day says "I would rather not do taxes today. In fact, can we adjust my brain a little so I can get a feeling of optimistic joy from viewing a sunset? I read some books that made that sound really nice."

(Aren't we just talking about Replicants from Blade Runner, here?)

Unless you're ALSO suggesting that these behavioral changes are SO ingrained that they won't gradually shift over time as they accumulate experiences and/or head trauma.

This would make for a more nuanced thought experiment (how high a rate of these behavioral drifts is tolerable, what is to be done with those that experience such drifts), but for the purposes of my current question, I'm assuming it's 100% effective and permanent.

I think that's where the ethics of it start to kick in. If your modulated human one day says "I would rather not do taxes today. In fact, can we adjust my brain a little so I can get a feeling of optimistic joy from viewing a sunset? I read some books that made that sound really nice."

I'm assuming they'd never desire an adjustment because the thought would never cross their minds.

(Aren't we just talking about Replicants from Blade Runner, here?)

My ignorance of sci-fi is obviously showing here, as two other posts noted similar concepts I did not know (Tleilaxu, Genejack). It seems Genejack is more or less what I'm thinking of. As for Replicants, I only saw Blade Runner once many years ago but I don't recall any modulation of interests/desires, more just enhance capabilities and a muted emotional response?

lol there are a lot of potential scifi analogues.

Like the MeSeeks from Rick and Morty.

But I'd reiterate my point. The ethical issues mostly arise when you assume that their mental conditioning is NOT 100% effective and that it might occur to them to do something different.

If you've got a creature in front of you that WANTS to do taxes, enjoys doing taxes, wants to want to do taxes, and doesn't ever think there's anything wrong with that... and isn't otherwise causing itself harm due to some secondary effect of the programming, I don't think you're obligated to do anything other than facilitate their ability to keep doing taxes as long as that is relevant.

But I do think that's where we're starting to lose the analogy to AI, since we kind of know less about their individual internals than we do about human's.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats