site banner

Culture War Roundup for the week of October 10, 2022

This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.

Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.

We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:

  • Shaming.

  • Attempting to 'build consensus' or enforce ideological conformity.

  • Making sweeping generalizations to vilify a group you dislike.

  • Recruiting for a cause.

  • Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.

In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:

  • Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.

  • Be as precise and charitable as you can. Don't paraphrase unflatteringly.

  • Don't imply that someone said something they did not say, even if you think it follows from what they said.

  • Write like everyone is reading and you want them to be included in the discussion.

On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.

Jump in the discussion.

No email address required.

Seeing as no one else has discussed this, I'll try to give a brief overview of the Drama that has taken place in the Stable Diffusion community over the last few days.

On 7th Oct, most of the source code, private models, and proprietary technology of NovelAI is leaked on /sdg/.

NovelAI's anime art generation stack was (is) significantly better than the open source tech available, so tooling is quickly made available to work with the leaked models. More specifically, the developer for the most popular offline Stable Diffusion tool, AUTOMATIC1111, immediately implements features to work with NovelAI's "Hypernetworks".

Within a day, AUTOMATIC1111 is accused of code theft and banned from the Stable Diffusion community, a decision explicitly backed by Emad. This causes a lot of Drama to happen,

  • Because the stable-diffusion-webui project is extremely popular; no other open source tool comes close in functionality,

  • Because it's ambiguous whether any code was truly stolen or not,

  • Because NovelAI was discovered to have illegally copied open source code in their leaked repositories, an error that was admittedly quickly reverted by their devs,

  • Because of the optics of the situation -- Stability AI backing a closed-source company over a popular open source figure?

The drama expands further when links to stable-diffusion-webui are scrubbed from /r/stablediffusion, causing the former moderators of that subreddit to reveal that the moderation team of the subreddit experienced a quiet takeover by representatives of Stability AI. It is additionally claimed that a similar process occured for the SD Discord server.

And as an extra bonus, the coomers using SD have gone into high alert after Emad remarked in an interview that the release of SD V1.5 would be delayed to "handle edge cases" & (de)bias the model against "topless women".

Insofar as I have a take on all of this, it's going to be some blend of "holy shit Emad please stop doing bad PR" and the Seven Zillion Witches problem. I find it horrific that the mutually agreeable agenda of "let's keep AI open and available" is doing its best to self-destruct via the usual processes of internet catfights and the amplification of minor differences within what ought to be a united ingroup.

Streamable giving me a "couldn't find your video" error on that interview.

Okaay I have no idea what's going on with the comment box. The link I have in there right now when I click the edit button is:

but it's getting rendered as

You should reread your comment and laugh at it: The rendering bug you're pointing out is happening again, so your description isn't being posted accurately.

(You can see what people wrote by using the "View source" button. In this case, there should only be a single "/e/" in the URL, but the site is adding a second/third one.)


I was reading just the other day on /v/ about how a text based AI site was censoring/lobotomizing its video-game/anime characters. People were ERPing or otherwise messing with the anime girls but there were also some people who got pretty upset with the censorship, pic related.

There definitely seems to be a global trend towards censorship - AI dungeon had the same experience.


I followed Emad's twitter handle for a while, and... I kind of hate to say this, but doesn't he seem like an idiot? I don't know if he's trying to play-act at his impression of a visionary founder, but his twitter posts seem like they were written by a twelve year old, the vocalizations of his putative plans for Stability AI seem to pivot by the day, every post on his Reddit AMA is "yes" when people ask if he is working on a thousand different projects (many nonsensical), and his only achievement so far has been to pay a bunch of money to buy up an A100 cluster at AWS (I think?) and then offer it to an academic research team in Germany to actually build a model. That was a great move for the world, and would be a great move for an AI startup founder who knew how to channel that lightning in a bottle, but I haven't seen any indication that he has a second act.

I guess he was successful enough as an energy trader to afford the A100s in the first place, but there are lots of dumb traders who got lucky, and I'm not sure how transferrable the skills of a competent energy trader are to running an AI research organization.

I think Emad is substantially more intelligent than he lets on on twitter/discord, but also «visionary founder» is often to some extent an idiot; they both naturally exhibit those traits and learn this hot dog vendor chutzpah from the noxious superstimuli-ridden startup culture depicted in one famous sitcom*. (Does Musk's Twitter indicate intelligence?) It's good marketing at this stage of the project, if anything – followers love him.

In reality they are doing some clever things to work around their late comer disadvantage. For example, they're hiring all the memey Twitter ML stars who have been underappreciated by corporations; hardmaru is not Hinton or Le Cun, but he's good and also decades younger, so it's an interesting bet. I am also not sure if these nonsensical projects are not at least 50% real – everything he posts related to audiovisual generation is in principle doable and his hires ought to be chomping at the bit to do it, only hamstrung by compute.

He is also much colder and rational in business matters than the hyperactive tweeting indicates. This quid pro quo NovelAI-Automatic story is evidence enough; «community vibes» are very important but not as important as moneyed partnerships.

Ultimately, the idea seems to be «fuck it, we need compute, any ML talent we can get and visibility, the rest is easy». And even if the rest is still hard, I can see where he's coming from. It's jumping into the last door of the departing train. With what Stability has got, they are an entity at least theoretically capable of surviving the Purge by means of some of those galaxy brain business ideas. The second act, then, is triage.

April 24 / May 7, 1919

I’m rereading The Precipice [Goncharov’s Обрыв]. It is long but so intelligent and forceful. Still I have to force myself — given that I find these Mark Volokhovs so repulsive now. What a multitude of hooligans have descended from this Mark! “How can you creep into someone else’s garden and eat their apples?” “But what does this mean: ‘someone else’s garden or apples’? Why can’t I eat when I feel like it?” Mark is a brilliant creation, and here lies the remarkable business of artists: an individual captures, distills and reifies a type, one that has existed diffuse in the air, and does it so well that sometimes his presence and influence increase a hundredfold—in complete disregard to the purpose his portrait had. One wanted to poke fun at the remnants of knighthood, and has created a figure who never existed in life, but became the catalyst for hundreds of Don Quixotes to spring into existence. Another wanted to castigate all the things that were associated with Mark, but wound up giving birth to thousands of Marks whose origins were already not in life but in books.

Generally speaking, how does one distinguish between that which is real and that which books, the theater, and films give us? Very many living people who have taken part in my life have had, most likely, far less of an impact upon me than heroes of Shakespeare and Tolstoy. Others have allowed Sherlock Holmes to enter their lives, and some maid allows in some woman whom she saw in an automobile on the movie screen.

– Ivan Bunin, Cursed Days.

Incidentally, Thomas Gaiton Marullo's translation (at least my copy) is nonsensical, to the point he seems to confuse Goncharov in the author's speech with Cervantes and utterly mangles the passage about real people and characters. I tend to suspect that this is one reason for the weird reputation of Russian lit, which some Westerners adore and others take to be cryptic pretentious ravings of drunk madmen. Our prose is often philosophical, but it's more like philosophy of the naive Socratic and Western Moralist tradition and not any clever Postmodernist sort; a translator needs intelligence as well as common sense to work with it, not indecent infatuation with Ze Slavic Soul. One shouldn't let Muscovite culture pass through Brooklyn intellectuals on the way to the library.

Here's hoping, particularly if he continues to train and disruptively open-source interesting models. But... I dunno, I've heard a lot of successful founders speak publicly and privately and they usually don't sound like idiots to me.

Thank you for this write-up. Things seem to be happening very quickly with a lot of volatility, so it's hard to say where things will go, but reading this makes me a little more pessimistic about the future of AI art. I had hoped that Stable Diffusion would offer a sort of "AI of last resort" that users could rely on that was largely free from authoritarian control, but how AUTOMATIC1111 was treated, and the comment about "handling edge cases" are both very concerning. I'm reminded of how dreams of a democratic open web from the 90s and early 2000s have been dashed in the intervening decades through most communication being controlled by a handful of censorious companies, with the logistics of setting up alternatives being too high of a hurdle to overcome for actual open alternatives to exist in a meaningful manner. That said, I don't know enough about AI art software to tell if a similar thing is likely to happen here; at the least, the logistics of collecting and collating the data required to train such software seems like a major hurdle, even before getting into the legal issues.


And as an extra bonus, the coomers using SD have gone into high alert after Emad remarked in an interview that the release of SD V1.5 would be delayed to "handle edge cases" & (de)bias the model against "topless women".

That link goes to https://todo.later/. Were you intending to replace that with a different URL?

Yes indeed, that todo link should've been replaced with a link to a transcript of Emad's recent Interview.

I failed to find the transcript in my browser history, so I've relinked the video in its place.

Because NovelAI was discovered to have illegally copied open source code in their leaked repositories, an error that was admittedly quickly reverted by their devs

So when they copy open source code without having the proper rights to do so, it's a problem, but when they copy art on the other hand...

They aren't copying art though...?

I can’t think of how training could occur if you didn’t make a copy in RAM of the images that were being trained on.

Honestly though, that’s a technicality that doesn’t actually matter. The point is that if software developers can reasonably request that their publicly-available code not be used in a proprietary SaaS, then artists should be able to request that their publicly-available images not be used to train AI models.

make a copy in RAM of the images that were being trained on

Or on disk! But this is just the same thing your web browser does (perhaps literally - the training images are distributed as URLs). Various governments took an astonishingly long time to clarify that browsers displaying and even caching URL contents was legal, but I don't think many people were ever really expecting it to not be.

This is the same way that an art student "copies" the art of other artists by the light reflecting off the picture s activating their rods and cones which get converted to signals in the neurons in their brains which produce the perception of the image in their consciousness. By your definition of copying, all perception of media is copying; e.g. every time I listen to Bohemian Rhapsody, I'm copying the work of Queen. Which is a fine definition, but is also not exactly a common one and one that loses a lot of the meaning that's intended when people use that word.

As I already stated, the technicalities of what constitutes "copying" or not are quite beside the point.

Let's put it this way. Look at the license agreement for Visual Studio Community 2022. This is a freely (as in beer) available piece of software; anyone with an internet connection can go to Microsoft's website and download it and start using it. But the license agreement places all sorts of arbitrary restrictions on what you can and can't do with it - for example, you can't use it to develop software if you're an "enterprise" (according to their own arbitrary definition of what an enterprise is) and your application does not fall under the heading of "device driver", "SQL server development", or one of their other arbitrarily-decreed exceptions.

The question naturally arises: what gives Microsoft the right to control what I do with their bits and bytes like this? Obviously the answer is "because of the license agreement, duh", but why should the license agreement be binding? Why should they be allowed to write a "license agreement" in the first place? They put this sequence of bits (the compiled binary) on their website, in plain view, where anyone can look at it and download it. Once I download it, they can't take it away, or really have any direct control over what I do with it (modulo the fact that the application might call home sometimes, but we can assume that it doesn't, and nothing about the argument will change). Why can't I just tell Microsoft to sod off at this point? The bits are mine, I'll do what I want with them, I'll use them for enterprise application development and you can't stop me?

I do think that Microsoft should have the right to enforce this sort of license agreement, and I think it stems from a simple and general principle: people who create and distribute sequences of bits should have wide-but-reasonable latitude to determine how those bits are used, and they should be able to seek redress when those agreements are violated. It's hard for me to see how the frameworks of copyright and software licensing can even exist at all unless one endorses that principle, or one that is roughly equivalent.

Similar to how Microsoft can say "here are some free bits, just don't use them for enterprise software development", an artist should be able to say "here are some free bits, just don't use them to train an AI model". The fact that the artist's bits represent an image rather than a compiled binary doesn't seem like a relevant difference.

Naturally, this is all very vague by legal standards and makes no reference to actual IP law, by design. I'm not a lawyer and I won't pretend to know anything about the law. I'm approaching this from an ethical/philosophical standpoint and arguing based on what I perceive to be fair and equitable. The technicalities of the law are of secondary importance to me; laws can change, after all.

There’s an old article I’ve thought of when faced with definitional absurdities in aspects of IP law versus seemingly plain common sense: What Colour are your bits?

Yes, the concept of intellectual property brings along with it a number of intractable philosophical puzzles. But, so does every other concept that one could name, so it's in good company in that regard.

As I already stated, the technicalities of what constitutes "copying" or not are quite beside the point.

I mean, the whole subthread started from your comment:

but when they copy art on the other hand...

Which I guess you're walking back from with your further clarification in this post, I suppose.

Similar to how Microsoft can say "here are some free bits, just don't use them for enterprise software development", an artist should be able to say "here are some free bits, just don't use them to train an AI model". The fact that the artist's bits represent an image rather than a compiled binary doesn't seem like a relevant difference.

This is a good point, and I agree that the artist should be able to say that. Any artist who publishes their art somewhere where viewers agree to license that demands as such before viewing that artwork - much like Microsoft has the end user "sign" a license agreement at some point before running their free software - is on solid ethical (and legal I think, though IANAL) grounds to complain if someone uses it for AI art model training. Plenty of artists already do a similar thing with paid art on places like Patreon where the end user has to agree not to share the art with anyone else.

If they publish it on a public forum, then that's a different matter. They don't get to publish something publicly for free public consumption and demand that others don't view their art and take inspiration from them; that'd be having their cake and eating it too.

I have to wonder how much of this is "your first mistake was posting anything publicly on the internet." Even back in the 2000's and 2010's, people tried to enforce what were weak ethical norms for art (no tracing, no reuploading without permission). Ultimately, though, there aren't actual legal mechanisms beyond DMCA takedowns.

We're back to MAI Systems Corp. v. Peak Computer, Inc. territory with this kind of logic.

MAI Systems (who argued with that sort of logic) won that one, and the subsequent statutory exceptions do not cover art.

The analogous case to art being used to train a model is code being used to train a model, and that's what GitHub Copilot and OpenAI Codex are. Most software engineers like the idea.

I'm going to need a survey for that, most of my peers rightfully agree that this is blatant infringement designed to launder FOSS code into proprietary software.

It’s entirely unsurprising if they “like the idea”, because CoPilot is little better than autocomplete. It’s no threat to their livelihoods.

Naturally, I think that people who publicly release source code should be able to opt out of AI training.

And if they're copying art then its surely similar to how human artists 'copy' other artists' works whilst developing their style and skills.

AI artbots have put known artist's written signatures onto art. All the comic book people who copied Rob Liefeld's style in the 90s did not write his name on their "art."

They're all self-serving, what surprise, what shock, no one could have predicted this tragic outcome.