This thread is for anyone working on personal projects to share their progress, and hold themselves somewhat accountable to a group of peers.
Post your project, your progress from last week, and what you hope to accomplish this week.
If you want to be pinged with a reminder asking about your project, let me know, and I'll harass you each week until you cancel the service.
Jump in the discussion.
No email address required.
Notes -
I'll caution that mixing different sizes of memory and generations of card means you're really going to want to stick with llama.cpp over vLLM, and be limited in hardware dtype support. For casual home use, that's fine, but there's a lot of talk about FP8, FP4, NVFP4, that just won't be worth it for you.
It's like you know me or something!
The underlying issue that I have is that my motherboard is not ideal for dual GPU setup either, offering only PCIE4x4 for a secondary GPU, so
whenif I find myself that deep in the rabbit hole, the deal that I am making with future me is that I'll evaluate my options at that point with an eye towards still trying to keep the price reasonable and thus getting a motherboard that'll give me 2x8 for PCIE and probably a second 3090, because if I start thinking too much about that now, I might be a little too tempted to roll the dice on one of the many superficially good-looking but probably fraudulent eBay classified ads that'll tie up a non-trivial amount of cash for a month or two while I work a CC dispute claim to its inevitable conclusion. It's a good story I'm telling myself, right?More options
Context Copy link
More options
Context Copy link