birb_cromble
No bio...
User ID: 3236
I've expressed here in the past that I'm not seeing the huge gains from LLM based coding that a lot of other parties are seeing. A new research paper suggests that maybe I'm not just prompting it wrong.
We present a systematic study evaluating how well agents handle structural constraints in multi-file backend generation. By fixing a unified API contract across 80 greenfield generation tasks and 20 feature-implementation tasks spanning eight web frameworks, we isolate the effect of structural complexity using a dual evaluation with end-to-end behavioral tests and static verifiers. Our findings reveal a phenomenon of constraint decay: as structural requirements accumulate, agent performance exhibits a substantial decline. Capable configurations lose 30 points on average in assertion pass rates from baseline to fully specified tasks, while some weaker configurations approach zero.
Specifically, the paper calls out convention based frameworks and ORMs as particularly problematic, and I'm using both. It feels good to see some evidence that I'm not crazy and incompetent.
birb_cromble
2hr ago
I was pretty autistic on that test - in the same range as @ToaKraka, who seems to be our gold standard here on TheMotte.
I was also diagnosed as an adult when seeking treatment for PTSD. Up until that point, I had no idea. Growing up, there were a lot of comments about how "that boy ain't right" and how I needed to "act normal". The idea that it might be something diagnosable or treatable didn't exist in those communities. Instead of an IEP, I got my ass beat until I could fake it well enough to get by.
Now that I know, it doesn't really change much. Mostly it just informed the PTSD treatment. Well, it informs treatment and gives my partner a new way to playfully make jokes about my behavior at times.
TK: Please don't take the gold standard comparison as an insult. You're one of my favorite people on this forum. The weird shit you dig up and post brightens my day whenever I see it.
birb_cromble
2hr ago
Now that you mention it, when was the last time you even saw a minimum wage job? Even the convenience stores and McDonald's around me are offering well over the state minimums. It seems like it's one of those things that exists on paper but doesn't really come up anymore.
birb_cromble
2hr ago
Anecdotally, I'm seeing a lot more of that around me. Punk is making a raging comeback.
birb_cromble
2hr ago
It might be worth moving this over into the finance thread, but I am at least partially putting my money where my mouth is.
I'm of the opinion that the current LLM hype is starting to hit the second knee of the S-curve, both financially and technically.
Technically, exponential growth leads to exponential friction, and it looks to me like the real-world improvements in model capabilities are slowing down between generations. Anecdotally, it feels like the models are increasingly fungible and most of the ostensible improvements have come from harnesses, which are regular old software engineering. I think there's something there, but I think LLM tech represents a local maximum. I'm eagerly watching whatever Yann Lecun is cooking up at AMIL, because the general concept of a world model seems to map better to what we think of as "intelligence". His paper on energy models, specifically, is fascinating.
Financially, I think a lot about the "during a gold rush, sell shovels" aphorism. I also think about Buffett and Munger's rules of investing. Meta and Oracle are buying shovels, but using them to dig their own graves, so far as I can tell. I don't think Anthropic and OpenAI are ever going to be able to support their valuations, and per their S-1, xAI has already pretty much given up. If Google dies, it'll be for reasons other than AI spending. Nvidia has a good product and a good moat for now, but various specialized competitors are nipping at their heels while Chinese cards may develop into direct competitors.
In other words, I think the tech is going to continue developing, but I think a lot of the current players are in for a rude reminder of market on market fundamentals by 2H 2027 or so. I know you've bemoaned "financialization" in the past, but at the end or the day the economy is just people buying and selling things, and fuck me if it doesn't seem like some of these companies are trying to act like that's not true.
Where does that leave me? I'm moving down the stack. If the tech is going somewhere, it has to run on real things and interact with the real world eventually. Companies doing physical things are riskier to start than pure software, but they're less likely to get disrupted of they establish themselves. I've largely stopped investing in funds that hold significant amounts of meta, oracle, Tesla (because I think they may absorb SpaceX), and even Nvidia. On the other hand, I'm expanding my positions in funds that hold TSMC, ASML, and Lam Research. I am watching Cerebras, but I won't invest until I better understand how they're using software to get around defects on their enormous chips.
Don't flame out because they haven't followed what you consider the optimal path. Everybody has to start somewhere.
I'm not a mod, but please don't lead with accusations of bad faith. It's clear you have a lot of experience in this area, and I fear you're assuming that the layman's level of expertise is much higher than it is.
birb_cromble
15hr ago
past like 5+ years
2022 was an apocalyptically bad year for bonds. It was the worst year in history. The runner up was 1803. How does your four year look?
Beyond that, are you looking at share appreciation or total yield?
birb_cromble
16hr ago
You can pry my minus sign dash from my cold, dead hands.
birb_cromble
17hr ago
It could still be, but I'm too old to keep track of that shit anymore.
birb_cromble
17hr ago
People like MLID have been floating rumors about Intel killing their consumer GPUs since about ten seconds after ARC came out. At this point I take rumors of cancellation as evidence of an imminent release.
birb_cromble
17hr ago
For tax loss harvesting specifically there's a limit, but I'm going to go out on a limb and assume that somebody with Trump's net worth has access to tax avoidance schemes that don't even have names yet.
birb_cromble
20hr ago
How does the price of SLV and the NAV of SLV correlate to the spot price of physical silver? Reading the prospectus, it seems like it is actually holding a fair amount of physical silver in trust, but I'm out traveling for the weekend and I can't really dig.
birb_cromble
22hr ago
I think we might be talking past each other. I was talking about treasury inflation protected securities.
birb_cromble
22hr ago
Charitably, it could be tax loss harvesting. At high net with and income strategies like that can get pretty intense.
birb_cromble
22hr ago
I wish TIPS weren't taxed the way they are. It seems almost perfectly self-defeating
England declared war on France and nobody was happy about it.
More options
Context Copy link