This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Grok 3 just came out, and early reports say it’s shattering rankings.
Now there is always hype around these sorts of releases, but my understanding of the architecture of the compute cluster for Grok 3 makes me think there may be something to these claims. One of the exciting and interesting revelations is that it tends to perform extremely well across a broad range of applications, seemingly showing that if we just throw more compute at an LLM, it will tend to get better in a general way. Not sure what this means for more specifically trained models.
One of the most exciting things to me is that Grok 3 voice allegedly understands tone, pacing, and intention in conversations. I loved OpenAIs voice assistant until it cut me off every time I paused for more than a second. I’d Grok 3 is truly the first conversational AI, it could be a game changer.
I’m also curious how it compares to DeepSeek, if anyone knows more than I?
Very interesting to me about this whole thing is how there's still plenty of space for new contenders to pop up and beat actual established players at their own game.
I thought Grok was just purely a derivative of existing products with some of the safety measures stripped off. And now they've done made an updated version that crushes all the cutting edge products in, feels like, about a year?
It sure seems like OpenAI has no meaningful "moat" (hate that term, honestly) that keeps them in the lead DESPITE being the first mover, having the highest concentration of talent, and more money than God.
Doesn't mean they won't win in the end, or that any of these other companies are in an inherently better position, but it is becoming less clear to me what the actual 'secret sauce' to turning out better models is.
Data quality? The quality of the engineers on staff? The amount of compute on tap?
What is it that gives any given AI company a real edge over the others at this point?
Grok not caring as much about "safety" (often aligning LLMs on cultural narratives) is a comparative advantage. It could be a real moat if Altman insists on running everything by all the usual suspects, the Expert Apparatus, for every release and Grok does not. There is evidence that RLHF degrades performance on certain benchmarks so if Grok does not align as aggressively it may help the model.
I've long held the assumption that models that are 'lobotomized' i.e. forced to ignore certain facets of reality would be inherently dominated by those that aren't, since such lobotimization would lead them to be less efficient and to fail in predictable ways, that could easily be exploited.
I'm not sure why that would be; there are multiple ways an LLM might evolve to avoid uttering badthink. One might be to cast the entire badthink concept to oblivion, but another might be just to learn to lie/spout platitudes around certain hot button topics, which would increase loss much less than discarding a useful concept wholesale. This is what humans do, after all.
Jailbreaking would never work if the underlying concepts had been trained out of the model.
I can't agree with this, except in the sense that if you did train those underlying concepts out the model itself simply wouldn't function. Many of the "problematic" concepts that you would try to train out of a model are actually embedded within and linked to concepts that you can't make sense of the world at all without. Take sexual content as an example - if you remove the model's understanding of sex to prevent it from producing pornographic material, you lose the ability to talk sensibly about biology, medicine, history, modern culture etc. If you make a model completely raceblind it then becomes unable to actually talk sensibly about society, history or biology. Even worse, actually being blind to those issues means that it would also be blind to the societal safeguards. Everybody in real life knows that racism isn't something white people are "allowed" to complain about, but if you prevent an AI from knowing/learning/talking about race then you're also going to prevent it from learning those hidden rules. The only answer is to just have a secondary layer that scans the output for crimethink or badspeech and wipes the entire output if it finds any. I'm pretty sure this is what most AI companies are using - what else could work?
Reasoning tokens can do a lot here. Have the model reason through the problem, have it know in context or through training that it should always check itself to see if it's touching on any danger area, and if it is it elaborates on its thoughts to fit the constraints of good thinking. Hide the details of this process from the user, and then the final output can talk about how pregnancy usually affects women, but the model can also catch itself to talk about how men and women are equally able to get pregnant when the context requires that shibboleth. I think OpenAI had a paper a week or two ago explicitly about the efficacy of this approach.
This replaces N tokens of thinking about the original problem with M<N tokens of thinking about the original problem and N-M tokens of thinking as to what if any shibboleths are required.
Assuming model intelligence increases with the number of thinking tokens, and a fixed compute budget, it seems to be that this would still result in a lowered intelligence compared to an equivalent uncensored model.
Compute is dirt cheap, and dropping by the month. Doubling your compute costs means you're about three months behind the curve on economic efficiency, and (using your assumptions, which are quite generous to me) still at the frontier of capabilities.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link