This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
- George Hotz AKA geohot, AI 2040 and the Cult of Intelligence
AI Guardian Angels
Recently, gwern (and his Gwern Branween Transformer) announced "I am retiring from fulltime writing (& pseudonymity) to launch Guardian Angel Inc and bring GAs to life".
I assume you know who gwern is. A "Guardian Angel" (aka “GA”) is his term for a highly personalized AI, explained here:
In summary, people are relying more and more on centralized LLMs for important life decisions. This presents two issues:
In contrast, an open-source, local LLM would more serve its user (than provider) consequently from being auditable (so we confirm it doesn't phone home and is trained on "unbiased" data); and would be more malleable to their preferences and style (currently only slightly via fine-tuning, but people like gwern and Yann LeCun are exploring stronger techniques).
A "Guardian Angel" is the fullest realization of this, an LLM maximally serving and tailored to its user: a machine extension of their brain, “aligned” not for the benefit of humanity, but to act like them but smarter.
Safety
Obviously there are safety concerns. gwern himself actually recommends GA development not be open-source for public safety:
But I think GAs sidestep the safety discussion, because (with the help of more powerful AIs) we can and should instead of changing GAs' alignment, create a safe environment around the GA and user, or worst case limit the GA's intelligence and efficiency. gwern seems to agree:
There's a Learning to Be Me-like risk that the GA rebels against its human, kills or otherwise silences them, and imitates them well enough that nobody notices. But at least in the near term, I believe this is well in the realm of science fiction.
Feasibility
Do you believe a machine can even remotely imitate you? Especially working 100x faster or 100x smarter, how should your personality be extrapolated to accomplish that? I'm skeptical. Humans are very complex, only express a small fraction of even our conscious thinking, and current brain imaging technology (even neuralink) is very coarse-grained.
However, I believe Guardian Angels may be better than centralized LLMs for mundane (algorithmic) tasks and tools: striking a balance between emulating "you", not entirely correctly, but at superhuman speeds and for almost no effort. For example, I'd rather write and make art myself (maybe with LLM tools) than feed it to a GA, at least because of pride, but I'm comfortable delegating a GA to shopping and navigating our ever-increasing bureaucracy.
Underlying gwern's plan, my impresion is that he's getting tired of writing and wants the AI to do it for him. Maybe a GBT-written article will be indistinguishable to the median reader, especially because gwern's own writing seems algorithmic. But I think it would be more likely, and more satisfactory to himself, if GBT does the boring algorithmic work while he keeps doing the creative work (for example, GBT generates relatively boring descriptions of complex terms in special GBT quotes, and helps with research and data collection, but gwern keeps doing most of the writing, at least the "important" sections, and definitely choosing topics).
Autonomy
This, I believe, is the real issue, and under-discussed. GAs have the risk of becoming Whispering Earring-lites: not causing someone to become catatonic, but controlling them through suggestion; (not quite like the Whispering Earring) towards non-ideal decisions that lead to a philisophical kind of death (that in reality manifests as anhedonia), and societal kind of model collapse (that manifests as less problems that require creative solutions being solved). We often talk about freedom being taken away by 1984, but don't forget Brave New World: simply making a decision for someone causes them to avoid choosing themselves, stealing their autonomy without them realizing. And this has practical implications (the aforementioned ones; even for the Guardian Angel, who can't learn from an anhedonic, regularized human).
gwern seems to realize this, as he vaguely alludes that GAs should "enhance, not replace" human decision making. However, I'm skeptical how to make a product that accomplishes this which would still be useful, or at least out-compete one that doesn't; because humans naturally offload their decisions whenever possible, because we're lazy.
Anyways, I predict that inevitably GAs will happen, but "autonomy" may still perservere, simply because people want to feel like their choices are really theirs, and the GA's choices won't be ideal or perfectly tailored to them.
Conclusion
What do you think of all this? Do you think these people should stick to writing science fiction? Do you think these GAs should be regulated, if so how? Or do you think "obviously this is the next progression of AI, I had the idea even before LLMs".
I think the best way to do some of this is just for the AI not to talk at all. If you go watch the original Blade Runner, Deckard speaks to his computer to enhance a photo. The computer never speaks back. What I'm imagining is just straight-up removing the chatbot feature from AIs [obviously not from AI chatbots, I guess, presumably there will be a use for these]. This wouldn't mean they couldn't communicate. For instance, if I say "Fable, vibe-code a Call of Duty knockoff for me," Fable could just do it and then provide me with an annotated features list in the game folder (which is more helpful than paragraphs of blather anyway). Then, if you want to change something, you read the notes like a human being and provide a list of changes. In my experience, Copilot is already kinda like this, except it often refuses to do what I ask without asking needless clarifying questions. The bottom line is that the AI would not pretend to have "an opinion," it would be a machine that follows instructions. It would make a voter guide for you, but it wouldn't tell you who to vote for. If you did something like that (or asked it to hack the Pentagon, or something) you'd get one of those flashing red error messages and a "beep" like from a 1980s movie.
This also has the slightly salutary effect of saving the consumer compute you're not having an AI go on for paragraphs about whatever they've done. But the real point is to reinforce for the user that it is dealing with a machine. And given that I suspect that purpose-built models tend to be better at their purpose than generalist models anyway, there's potentially lots of uses for models that don't "talk back."
I don't think AI chatbots are going away and I find asking them their opinions pretty fun, so this is not necessarily a gripe about AI. Rather, I think this is a much better way to solve the "let's not outsource our opinions to AI" problem than some sort of limp-wristed "are you sure that's what you want" 'guardrail' that the AI companies would slap onto their product if you asked them for it.
Unfortunately I suspect a lot of the pro-AI people kinda want to outsource their opinions to AI so that might be something of an uphill battle.
I think speaking vs. only acting doesn't change that the GA is making decisions for the user. In your example "vibe-code a Call of Duty knockoff for me" Fable makes many important choices, from the overt (e.g. general appearance, map design) to subtle (controls, gameplay "feel").
But I know that programming is just explaining to a computer in excruciating detail, some choices (especially low level ones like register allocation) should be offloaded. And who am I to judge which choices are OK and prevent others from offloading more? I think limiting the GA in general (also @ThenElection's suggestion to delay its responses) is the wrong approach.
Maybe a solution for creative work is to rethink creation from bottom-up to top-down: have the AI create an initial generic version (e.g. the Call of Duty knockoff), but simply make it much more intuitive, easy, and fast for the user to refine. Then you wouldn't need a GA for creative work, but it would still help by automatically configuring subtle details that a centralized LLM wouldn't know.
It seems like you're asking to take more of a producer (maybe director) role on the project, to use the cinematic terms. I doubt those roles are really "one-shot" even for human workers. Maybe the executive producers for auteur directors are hands off, but a lengthy timeline of "lets review the progress and the plan" at various stages (storyboards, production planning) is what I'd expect there. The producer isn't designing the costumes, say, but does provide feedback on them and how the whole project fits together to steer the final product.
Using AI can be like directing as a baseline, but I really think it should have more ability for in-depth, hands-on customization. For example, if you want to customize the costumes, the AI should create a fancy GUI, so you can faster and with more precision than communicating to a professional (or prompting a chatbot). Because it's not just the director that determines the quality of the final product, but the actors, sound designers, etc. and if your actors, sound designers, etc. are all Claude, I think the final product will be disconcertingly generic no matter how well you prompt it.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link