This thread is for anyone working on personal projects to share their progress, and hold themselves somewhat accountable to a group of peers.
Post your project, your progress from last week, and what you hope to accomplish this week.
If you want to be pinged with a reminder asking about your project, let me know, and I'll harass you each week until you cancel the service.
Jump in the discussion.
No email address required.
Notes -
Last week I whined that I really should have gone the Full Monty right away on my graphics card, mostly because I'm having so much fun dicking around with local LLMs but want more out of them. Today I stumbled upon the elegant solution of buying a used 3090 and a 1200 watt power supply (which I would have needed for the better card anyway) to get myself to a nice respectable 40 gigs of VRAM, which is oh-so-tantalizingly close to being able to run a full 70b model at the desired Q4_K_M. Odin dammit but do I ever wanna pull the trigger and spend another ~$1500 to net myself that headroom!
That is all. For now...
Semi-related graphics card question: My PC is 10+ years old right now and my mac can run some games but emulation and minimum graphics only get you so far. In the next 2-3 years will the current expansion of chip fabs lead to cheaper cards in the future? I'm hoping to build a cheap, medium-tier gaming PC for less than say $1500 (cheap peripherals). Thoughts?
With the caveat that my semi-informed opinion plus experience will, with $7, buy you a latte at Starbucks, I think that the new fabs will help, but prices will still be above average unless an AI crash tanks the chip market in that time frame. I don't think such a thing is out of the question, and I expect some sort of market correction is inevitable in the long run, but I now grok why AI demand for wafers in general and memory in particular is so insatiable, which in turn leads me to believe that looking for lower prices in the medium term will be a coin flip at best. I'm personally glad that I went the other direction and decided eff it, I'm getting in now on a good deal and I'll let the chips fall where they may, and my only ragret is that I didn't spend the extra ~$1600 when I had the chance and go whole hog. I expect that if you're willing to buy used, you should be able to find something more than suitable when you're ready to make the purchase, and even if you want new, the budget/mid-tier stuff is plenty capable of suiting your needs. The performance of the Steam deck is instructive there.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link