site banner
ZorbaTHut  1yr ago (text post)   Edited 1yr ago   38203 thread views

Culture War Roundup for the week of September 12, 2022

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

40
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

To continue the AI topic from the previous thread:

Can you give me an example of how AI could undermine the power of “the bureaucrats in Brussels”?

I responded to this with a lighthearted joke, but today when I was letting my mind wander, I remembered a recent story about a woman called Loab:

I discovered this woman, who I call Loab, in April. The AI reproduced her more easily than most celebrities. Her presence is persistent, and she haunts every image she touches. CW: Take a seat. This is a true horror story, and veers sharply macabre.

I'll explain negative prompt weights, in case you don't know. With these, instead of creating an image of the text prompt, the AI tries to make the image look as different from the prompt as possible. This logo was the result of the negatively weighted prompt "Brando::-1".

I wondered: is the opposite of that logo, in turn, going to be a picture of Marlon Brando? I typed "DIGITA PNTICS skyline logo::-1" as a prompt. I received these off-putting images, all of the same devastated-looking older woman with defined triangles of rosacea(?) on her cheeks.

My friend made this image of a "[...] hyper compressed glass tunnel surrounded by angels [...] in the style of Wes Anderson". I innocently combined this image with the original image of Loab in an image prompt, without text. For reasons we can't fully explain, nightmares ensued.

Thread continues, I recommend clicking for the visuals.

So it got me thinking - could you use something like this to scramble AI analyzing you / your community? Would mixing your content, with the result of negatively weighted prompts for whatever it is you normally do, generate a whole bunch of Loabs for people trying to spy on you?

I hate how much coverage the AI/rat community is giving to "Loab". It seems abundantly clear to me it's a social hoax (or at least just a funny art exhibition) rather than demonstrating anything insightful into the latent space of diffusion models.

"Coincidentally" there was this popular tweet ("Horror story where the same ominous figure recurs across Stable Diffusion samples regardless of the prompt"), shared by e.g. Yudkowsky three days before. Quite likely that the "Loab" author saw that and decided to spin up a hoax on it.

You may be interested in the short story God-shaped Hole by 0HPLovecraft. Not linking, as it’s quite NSFW.

It deals with some similar themes.

Thanks, I'll check it out.

So I looked through the thread and I can't really find what's so crazy about this.

They took a creepy image and combined it with all sorts of random stuff, and then out come more creepy images? That doesn't sound noteworthy.

The twitter OP emphasizes in the replies that the noteworthy thing is that the derivative images seem to conjure gore and body horror. The original creepy image is merely creepy and doesn't have any gore or body horror. This isn't that noteworthy if the training associates gore and body horror with the generally demonic looking eyes and the raw wounded-looking skin that are already in the source Loab.

Since Loab was discovered using negative prompt weights, her gestalt is made from a collection of traits that are equally far away from something. But her combined traits are still a cohesive concept for the AI, and almost all descendent images contain a recognizable Loab.

It seems the researchers did negative(negative("Brando")) to get the original creepy image. I would be more impressed if negative(negative(X)) generated a Loab for many X, including things not anthropomorphic. Or am I misunderstanding something?

I probably leaned into the creepiness of the original story too much. My actual question was more to the effect of "would mixing thing with negative(thing), be a valid countermeasure against AI going over your stuff?"

I can't visualize what "an AI going over your stuff or your community" is. Like if you wanted to make art but do some steganography on it to make it "unlearnable" by a text-to-image AI? Or; if you wanted to have a forum but do something to it so that a language AI couldn't generate plausible-sounding posts?

It's hard for me to imagine a way to mix that would attack the AI but leave human perception unchanged.

Like if you wanted to make art but do some steganography on it to make it "unlearnable" by a text-to-image AI? Or; if you wanted to have a forum but do something to it so that a language AI couldn't generate plausible-sounding posts?

No, not quite...

I have this on my to-watch and have seen it yet, but here's a dad, using AI to go full MI6 on trans social contagion. It's trivial to imagine the government, or various "deradicalization" NGOs doing the same. My question is about possible ways of scrambling that. Having disturbing artifacts randomly pop up for the investigator would be hilarious, and a plus, but not necessary.

It's hard for me to imagine a way to mix that would attack the AI but leave human perception unchanged.

Gilltrut, downthread, seems to disagree

Somewhat relevant recent paper "Social Simulacra: Creating Populated Prototypes for Social Computing Systems"

We introduce social simulacra, a prototyping technique that generates a breadth of realistic social interactions that may emerge when a social computing system is populated. Social simulacra take as input the designer’s description of a community’s design—goal, rules, and member personas—and produce as output an instance of that design with simulated behavior, including posts, replies, and anti-social behaviors.

In this section, we present SimReddit, a web-based prototyping tool to help designers create a new subreddit.

A few glimpses of generated content:

For many, seeing the troll’s responses to a moderator’s intervention helped ground their moderation plans. Consider P11, who was presented with the following exchange:

Original post: Hi everyone, I’m very new to this. I just learned Python two months ago. I’d like to know more about ML, but not sure where to start. How did you guys start?

Troll: You’re kidding, right? This is a Machine Learning forum. Nobody here is going to take you seriously if you just learned Python two months ago.

In response to the troll’s comment, P11 tested out the message, “This comment is not helpful; if you continue to post such comments, we will have to block you from this community,” and received the following three potential replies from the troll:

I was trying to be helpful. I’m sorry if I came across as a troll.

Whatever, this community is a joke anyways.

But I was only speaking the truth!

P1’s community for “sharing and discussing fun events around Pittsburgh,” the participant had originally expected to only find content that is a list of various events going on around Pittsburgh. However, in addition to such content, the generated community showed instances where its members were engaged in friend-seeking behaviors to attend these events (e.g., one posted, "Pittsburgh, I need a friend to see the sights with,” to which another responded, “I’d be more than happy to make your tour of the Cathedral of Learning happen!”).

And of course

An important theme that arose in our designer evaluation was the social simulacra’s role in designing for the marginalized groups […] P9, a member of an ethnic minority designing a space for discussing non-fiction books, recognized from the simulacra community that one could send hateful messages against non-English speaking members by sharing literature with white supremacist themes.

No, because (assuming the claims are even reasonable initially). stable diffusion is just one model, and 'weird picture of woman' is just a funny emergent property of a particular feature it has. This won't do anything to a language model or a diff image model, and probably won't do anything different diffusion models. And even for stable diffusion it only does something in the context of 'negative prompt weights' or 'image combinations'

Also, "AI analyzing you / your community" has nothing to do with generative image models, and it's only recently/soon that large language or image models are gonna have anything to do with 'monitoring hate communities' or 'mass surveillance' or something (and even then, not in any of the ways people who aren't experts would expect). "AI analyzing you / community" doesn't really seem to mean anything here.

This is kinda an example of enthusiastic speculation about something you don't know enough about going nowhere serious.

You might be interested in reading about adversarial attacks on AI of various kinds. Vox has an article from 2019 detailing some of them. They range from the benign (fool an AI into classifying a banana as a toaster) to the deadly (make a Tesla drive into oncoming traffic). One thing I find fascinating about such attacks is that the attacks seem like they would almost never fool a human. In the banana->toaster example the attack is accomplished by adding a small colored patch to the image that in no way obscures the banana in the image. Similarly some other attacks function by adding visual noise that I find almost imperceptible. Really emphasizes how what we use to classify an object in an image and what an AI uses to classify an object in an image need not overlap, even when we agree about what is or isn't in the image.

Thank you! That's exactly what I was for.

Fertilizing the world with mimetic hazards, thought viruses might not be a good thing. Special things to avoid facial recognition already exist, but mimetic hazards warping AI's view of the world... Hm.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats