This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
How do you feel about ai watermarks à la Claude’s announcement. To me it feels like yet another huge eu overreach, even if Anthropic is happily participating. I know OpenAI is as well. To me it feels demeaning because it implies that if I use ai, that the content produced using it does not belong to me. As smart as AI’s get, if I use them, the outputs and productivity gain that comes from it should not be attributed to Claude. It should be attributed to the person that retrieved that output from Claude or ChatGPT etc. I know that’s apparently the case with intellectual property as companies wouldn’t use ai if they were concerned that ai providers would literally have some ownership of their goods / services. Though ai ideas can’t be copyrighted right? At what level of abstraction are you able to claim ownership of the idea you made with an AI?
I’m just frustrated all around - if they justified it with national security concerns or something re distilling, I may be onboard. But I WANT to use ai for everything - and watermarking all output makes that harder. It really sounds like (1) there are ways around it (2) all major model providers are going to be / are doing this to some extent.
And guess what? This pushes people who can’t afford a watermark to using open source models that don’t participate in arcane EU regulations. You’re just making students, authors, and others against the watermark move to another model provider to do what they want.
Honestly I understand the need to curate real human text, especially places like this, but increasingly the ai-using person becomes like a vessel to the model, using it to find some knowledge and communicate it outward, sometimes verbatim. They might think less deeply and understand stuff a little less than people rawdogging vocalizing their thoughts (like taking an airplane with no devices), but it’s still valuable. It sucks that people will soon be able to definitively say you were using ai output, honestly. Again, I get the hostility towards it, but i feel like if models keep improving at the pace they are, their language will start to sound more like the user.
Legally, it isn’t: ai produced works have no copyright protection.
More options
Context Copy link
From knowing a bit about the topic (very peripherally related to some research interests), I doubt it'll do as much as they propose. Existing algorithms might well work, but can't allow for an open "is this watermarked?" oracle, or it becomes fairly straightforward to start stripping watermarks, or even deriving the ways to generate them.
And if every attempt to check the watermarks requires going through Anthropic, it doesn't seem to be that much better than "is this in our set of generated artifacts?".
And that's in addition to the inherent fragility of such watermarks against edits and the amount of background data needed to hide one (I don't have a quantity in mind).
Distortion free watermarking is pretty well understood in the academic literature. I doubt anthropic is doing anything special to make it more resilient, so if a defeat script works on the oss version, it can also be used on this.
More options
Context Copy link
More options
Context Copy link
High school students, local business owners, people filing regulatory documents etc can be dumb, but they’re less dumb than the regulators think.
Pretty soon every one of them will know the meta is to either use Chinese/otherwise open source LLMs on grey market sites that don’t do this, or pass answers from regular ones through them if that’s a viable way of evading a watermark. No local LLM or powerful hardware required, although pretty soon even local models capable of running on the latest iPhone will do most of what normal people use LLMs for well enough.
More options
Context Copy link
This is actually - for maybe the first time ever - a case of "the innocent have nothing to fear". If you're not passing off AI work as your own, and being honest about the role of AI in your creative process, then a watermark isn't a problem. There are three cases you'd want to avoid a watermark: a) you are ashamed of your use of AI and trying to lie to your reader (in which case, good for the watermark), b) you are talking to a rabidly anti-AI crowd who will freak out at you just for using AI (what are you doing with your life, those people are not worth your breath), c) you have a philosophical objection to watermarks in general (understandable, but you have to accept that being the kind of guy who says "GNU/Linux" is not going to get traction in the wider population)
What I wonder about this is the extent to which the watermark causes performance loss in the model. 0.1%? 1%? 10%?
People are trying to coordinate a shaming campaign and I do not like such things. I am not ashamed but would-be-shamers deserve to be frustrated and confused in any way feasible. I advocate stripping off the watermarks or adding them to human generated text to make clear understanding practically impossible.
More options
Context Copy link
It doesn't hurt model performance. It just messes with the randomness used to sample tokens, but keeps the distribution the same.
More options
Context Copy link
More options
Context Copy link
This is once again a rear guard attempt by bureaucrats to curtail the rise of the technology that makes their power structure obsolete.
Whether they'll win is anyone's guess. The EU may have totally lost the technical battle, but Anthropic really wants to be a regulated monopoly, so it'll shop around for it's overbearing government
Moreover, the inefficacy of these policies individually doesn't quite matter since policy is always a slippery slope. People who claim that such and such regulation is bad law because it can't be reliably enforced forget that's the point. The enforcement comes later as we magically realize that we were so unprepared to enforce what is now the unquestionable will of the people (however many tries it took to get it into law).
My concern is less the bureaucrats themselves and more what's being assembled here. AI is the perfect excuse to require nominal cryptographic signatures for every single interaction as it will start to permiate every layer of communication, and this naturally extends surveillance and control into every single area of life. And it's coming together, mandatory electronic bills, digital id for social media, CBDCs, there is a vision at work here.
But once we all live in giga-DDR, what then? Do the people making this system genuinely think it's going to solve the structural problems of the current one by magnifying its worst traits? Or is there really no plan and people are simply reaching for something, anything, that will allow them to keep looting the treasury for one more day?
More options
Context Copy link
Well, it doesn't, anymore than if you asked a work colleague to write that report for you. Unless it's the case of a speechwriter or a ghostwriter producing work for hire, then someone (or something) else producing the output is not you.
What does "everything" mean? "Hey Claude, should I wipe my arse after taking a shit"? If you're on that level of dependence, you are not creating anything, the machine has replaced your brain.
Do you understand what you just said there? The human is the vessel for the model, not the other way round?
If you are a business owner, you unquestionably acquire copyright/ownership of anything written by people in your employ; contractors too under most normal employment contracts.
More options
Context Copy link
More options
Context Copy link
Why do you feel this way?
The opposite of this seems obviously true to me. If you use an LLM, the content it produces is not yours. You cannot and should not take credit for anything that an LLM did. If you prompted Claude to do something, Claude did it, not you, and Claude deserves the credit.
(To the extent that machines can earn or accumulate moral credit, which they cannot. Claude deserves the credit in the same way that an earthmover deserves credit for moving a pile of rock.)
Why does it make it harder?
The only effect is that it means that other people will know that you used a bot. Why should that trouble you? You did use a bot. You don't seem to think there's anything wrong with using a bot, so why feel ashamed?
Do you understand that this sounds like, "It sucks that I can't deceive people by passing off a bot's work as my own"? Is it not obvious why people might not be sympathetic to that complaint?
I think this is a good way at looking at it, but people who use earthmovers to move piles of rock get credit for that work. They get paid for it, and they can say, honestly, "I went and moved those rocks." As long as everyone understands that they are not claiming to have done it by hand, it's a perfectly fine thing to say and take credit for and get paid to do. Of course, if someone does actually move all that earth by hand we hold them in a different level of esteem.
However if you said "I benched 250 today" and it was actually your tractor that did it, you'd be taking egregiously false credit. And I think that using AIs to do your writing is bad for the same reason that getting a tractor to do your exercising is bad. Sure, taking credit for something you didn't do is bad, but also you're not really improving by it.
If you are accomplishing a goal-oriented task, you want to use the best available tools for the job. There are some jobs for which LLMs are the best (or one of the best) tools to accomplish that task. I don't think that there is anything wrong with using LLMs and taking credit for the work in that situation, if there is an understanding that you're engaging in a field or a task where an LLM is an acceptable tool.
However there are other tasks where the task is the goal. Nobody is getting paid to post on the Motte, and the point of this forum is to discuss and sharpen ideas with other people - if anyone wants to talk to an AI, they can do it themselves on their own time, but perhaps more importantly, if the forum devolves into AIs shouting at each other, nobody's really improving by it.
Perhaps it was a bad example. Using or piloting an earthmover does require skill, and the job that you use an earthmover in is certainly one that deserves compensation.
I hope you understand the point I'm getting at, at least. The human is doing some work in that situation, but that work is not physically digging out and lifting tons of earth.
More options
Context Copy link
More options
Context Copy link
...do you generally believe that an earthmover, and not its human operator, deserves credit for moving a rock?
No. It may have been a bad analogy. What I mean is that:
A human should not take credit for work a machine did.
Machines do not deserve credit for anything.
So, for instance, if you got a machine to move a large boulder, you should not expect to be praised for your strength. All the strength belonged to the machine. But nobody would congratulate the machine for its strength.
It is meant to be analogy to illustrate 1) you do not deserve credit for prompting Claude to write something, 2) Claude does not deserve credit for anything.
More options
Context Copy link
Usually when one wants a rock moved one hires a local firm. This local firm owns or rents an earthmover. At no point do you have to worry about whether Caterpillar Inc. will charge a licensing fee or have an opinion on whether or not the rock ought to be moved.
I believe you intended to respond to chooky a comment down?
More options
Context Copy link
More options
Context Copy link
Isn't that indeed how it works? If you need a rock moved, you hire a rock-moving business and pay them, I don't know, $500. The worker probably gets around $100 in wages. The other $400 goes to the business whose main function is to invest in earth-moving equipment.
More options
Context Copy link
More options
Context Copy link
Something might not be shameful, yet lead to people treating it as shameful anyway. Sometimes we forget in this world where claiming to be persecuted means high status, that actually being persecuted is no joke.
Isn't 'being treated as shameful' the same thing as 'shameful'? Shame is a social judgement.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
It's silly legislation, because in practice it will only affect the absolute laziest of AI users in cases where people want their AI usage to not be detected. Run the text through an old fashioned machine translator, and back to English and get rid of most of the signal. Or have an open source model running locally on your machine paraphrase the writing paragraph by paragraph. (You probably don't even need a particularly smart model for this. 7b parameter models fed two or three sentences at a time to rephrase on a loop could probably do this on even potato hardware, since you're just laundering already written text.)
Plus, even if Anthropic will have the tools to tell if the watermark is present, they'll never be able to make those widely available for free to the public, since an adversary could just keep testing against their watermark checker until the text no longer looks like it has the watermark. That means that the presence of the watermark makes no difference in the vast majority of cases, even if you don't get rid of it, since very few people will actually have the means of checking for the watermark.
So roughly 90% of them?
More options
Context Copy link
I think this describes quite a lot of people. Crucially, it will help a lot in education by making it so students can't just literally paste an AI response and receive full marks.
More options
Context Copy link
More options
Context Copy link
Generally? Yes. If they try to ban DeepSeek or regulate open models (including private hosting), that would be very bad (because I can't imagine how they'd non-invasively).
But for ChatGPT and Claude it's almost nonexistent, because when you use them
You already give up privacy.
The output is often obviously LLM, even if you prompt them to avoid the tropes. Survivorship bias I know, but if you can remove the LLM vibe, I'm sure you can remove the watermark.
More importantly, why does it matter to know something is AI generated? If that's the only information, it doesn't affect privacy. If the output is good, I promise most people won't care. And the holdouts, why do you care about their attention? like a black person trying to move into a sunset town, instead, let it be their loss.
I was in the national Korean War Memorial earlier today and noticed half of the exhibits had clear AI phrasing. Which is kinda nuts when it's only like 1000 words of English translated factual content required for the whole space and ostensibly it's the job of a lot of well-paid public servants to curate the exhibition. Who couldn't be fucked getting a manual translator/rephrasing it.
Do the common LLMs even produce the most obvious tells if you tell them to just translate a piece of text from language A to language B?
Laughing my ass off at the idea of ChatGPT translating some Korean grunt's journal entry as, "you're right, today's victory not only ensures the survival of worker's communism, but ten thousand years of Kim Il Sung."
More options
Context Copy link
More options
Context Copy link
I'm honestly more surprised that it's actually been updated in the past 5 years than I am that someone doing so would use AI.
Wild speculation but I think it's in the middle of a phased refresh since the section I noticed this in had 3 subsections, one of which was closed for renovations. Maybe I was wrong but I assume anytime I see 3 em-dashes in the same anything I'm most likely looking at AI output.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link