site banner
ArjinFerman  Tinfoil Gigachad  1d ago (text post)   228 thread views

Tinker Tuesday for July 14th, 2026

This thread is for anyone working on personal projects to share their progress, and hold themselves somewhat accountable to a group of peers.

Post your project, your progress from last week, and what you hope to accomplish this week.

If you want to be pinged with a reminder asking about your project, let me know, and I'll harass you each week until you cancel the service.

1
Top Bottom Old Controversial Comments
Jump in the discussion.

No email address required.

Gemma4 is quantization sensitive enough that 31B-Q5 is probably the best bet, closely followed by Gemma4 26BA4B around Q4-Q6 where you want (much) higher speed and simpler questions. That said, Gemma4 is great for writing, mediocre for coding, and lackluster for complex spatial reference.

Qwen 3.x is really good for local coding, especially 3.6 35B-A3B and 3.6 27B. Depending on context length, you'll want either Q6 or Q5, make sure to get the MTP variant setup where available. If you just want to code, these two models honestly cover 99.9% of anything you'd be able to do locally with any practical single-GPU machine.

Nemotron 3 Nano Omni 30B-A3B (Q4-Q5) is a little weird and it's not as good for code generation as Qwen, but it does handle really big contexts for analysis better. It's a bit annoying to set up properly, though.

Llama.cpp can run GGUFs of models in mixed-inference mode, where some layers operate on the GPU and some run in system RAM. This can be slightly slower for MoE models (anything with AXB as a suffix), or much slower for dense models, but it lets you run stuff that'd otherwise be impossible. I've got GLM-4.5-Air running on a single nVidia 3090 and 128 GB system RAM at Q8... admittedly, at <5 tokens/sec. Still can be useful for things like draft review if you let it run overnight.

((If you're really desperate, you can even offload to NVME, but this is a very bad idea for drive wear reasons.))

The general rule-of-thumb is that even if you're willing to accept slow inference, quants under Q2 are usually useless and under Q3 are marginal at best, so this doesn't mean it's worth the bandwidth and drive space to go with something like GLM-5.2 at Q1 just because the parameter count is high. But there are some useful options, still. For your setup, some that might be worth evaluating:

  • GLM-4.5-Air (355B A12B). Was the big standard for writing for quite some time, and while it suffers from quantization, it's still pretty useful.
  • Solar-Open-102B A12B, probably Q4-Q5. Kinda general-purpose, nothing outstanding, but it's unusual for its size point and fits your specs pretty well.
  • Mistral 3.5 Medium 128B, probably Q3-Q4. More meant as a strong reasoning code model, unfortunately not a great one, but it's the best Euro model and for language support that sometimes helps.
  • MiniMax 2.7 (229B) at Q3 (or variants like IQ3_S). Not a huge fan of it, but has a following.
  • DeepSeek V4-Flash (284B A13B) at Q3 or Q2.
  • Qwen 3.6-27B at Q8 can be worth having around, despite the RAM offload. Most of the time the difference from Q5 isn't massive, but I have seen logic and parsing differences.
  • GPT-OSS (120B, effectively A5.1B). This one's weird because of the native quants, but some people really like it and some people find it very meh.

What is this place?

This website is a place for people who want to move past shady thinking and test their ideas in a court of people who don't all share the same biases. Our goal is to optimize for light, not heat; this is a group effort, and all commentators are asked to do their part.

The weekly Culture War threads host the most controversial topics and are the most visible aspect of The Motte. However, many other topics are appropriate here. We encourage people to post anything related to science, politics, or philosophy; if in doubt, post!

Check out The Vault for an archive of old quality posts. You are encouraged to crosspost these elsewhere.


Why are you called The Motte?

A motte is a stone keep on a raised earthwork common in early medieval fortifications. More pertinently, it's an element in a rhetorical move called a "Motte-and-Bailey", originally identified by philosopher Nicholas Shackel. It describes the tendency in discourse for people to move from a controversial but high value claim to a defensible but less exciting one upon any resistance to the former. He likens this to the medieval fortification, where a desirable land (the bailey) is abandoned when in danger for the more easily defended motte. In Shackel's words, "The Motte represents the defensible but undesired propositions to which one retreats when hard pressed."

On The Motte, always attempt to remain inside your defensible territory, even if you are not being pressed.


New post guidelines

If you're posting something that isn't related to the culture war, we encourage you to post a thread for it. A submission statement is highly appreciated, but isn't necessary for text posts or links to largely-text posts such as blogs or news articles; if we're unsure of the value of your post, we might remove it until you add a submission statement. A submission statement is required for non-text sources (videos, podcasts, images).

Culture war posts go in the culture war thread; all links must either include a submission statement or significant commentary. Bare links without those will be removed.

If in doubt, please post it!


Rules


Recommended Posts And Communities

Recommended Realtime Chats