This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
How does Qwen 3.8 Flash perform? What kind of K/V cache are we talking about here, on consumer hardware?
I'm an AI fan but I have issues with even the biggest and strongest models for my usecases, which ironically enough is AI development (non-cheating AI in a strategy game, that is). They get there, we are making progress - but with no small amount of fumbling along the way. Testing various ideas and strategies takes time. Qwen 3.8 Flash is below Qwen 3.8 Max, which itself is below Kimi K3, right? And that's below Astra and Fable. And my usecase is nowhere near ASI development.
I agree that AI will improve in cost-efficiency but consumer hardware seems like a stretch.
Anything fast or nimble is going to roar and bellow from datacentre-grade compute if it can whisper on consumer hardware.
I can run Qwen-3.8-Flash (120B) at Q4, K/V cache, on an nVidia 3090 and a Core i5-14400 (albeit with a lot of now-expensive RAM), with simple llama.cpp run, around 5-2 t/s at 100k available 16-bit kv cache. Dropping the kv cache to 50k nearly doubles performance. Qwen-3.8 in general defaults to a heavy thinker, so that's worse than it sounds -- a moderately complex problem can burn 30k tokens -- but it's the sort of thing you can leave crunching on a problem for a while and be happy about the answer.
(Comparisons: Qwen-3.8-27B runs about ten times the speed, and Gemma4-26B runs basically faster than I can read it.)
For intelligence and capabilities, the comparison to frontier stuff is rough. Low-parameter models just don't have some information, and with either hallucinate or just nope out, no matter how well it had to be present in the training data. Indeed there's been some efforts to trim low-value knowledge from public models to optimize them for specific use cases, with weird results.
And home users have some rough spots. Both quantization and abliteration drive perplexity and errors, and the harnesses to find and debug them live aren't well-established in the open source (or free-as-in-beer) world. It's fascinating to read a logic trace that goes into surprising depth, but it doesn't do much if the program output doesn't work. The errors are small and embarrassingly simple for a programmer familiar with common JS errors, or for other models to catch, but non-programmers would likely struggle to explain what was even going wrong.
((Also note: the game's not good or fun, even when it does 'work'. That should be expected given the lack of specificity, lack of agent harness, or even a real iterative process, but it's also something no human would do this way even as the core idea it came up with is kinda clever.))
That said, intelligence can be surprising. If you want a model that can make connections between input tokens or parse through mounds of data, you can get away with stuff much smaller and more energy-efficient than you would expect. I would not, a year ago, have expected you could get spatial reasoning worth spit in a 27B model. A real big curveball isn't the most likely thing, and I wouldn't put a ton of money on specifically Jev doing anything ridiculous, but I wouldn't bet against someone coming out with a two-fold performance or intelligence improvement for inference in this model class before the end of the year, either.
More options
Context Copy link
More options
Context Copy link