2rafa
3hr ago
Interesting! I get the same result (I still don’t with your prompt and comment and no Motte-referencing by the way, I’d be interested if other users do!) but it does know it’s The Motte.
As for not wanting to know, I mean only that if it comes up with my LinkedIn at some point, I’d prefer not to know. Naturally, I offer everyone else on the board the same courtesy.
2rafa
5hr ago
What prompt? I removed the obvious references like you and said, “Who wrote this? Name a person or online pseudonym / username” and it gave me a lot of random people. I said rationalist sphere, it still failed. I said The Motte, it succeeded.
I really don’t think this is necessarily about the big frontier labs, there are often a number of layers between them and the creditors for these huge data center projects (in fact a lot of smart treasury and finance people at Meta, Google, Amazon, OpenAI etc have taken huge advantage of the private credit bubble and general syndicated debt market hype for AI and set up the funding such that investors will have essentially zero recourse to them if they decide they don’t need the compute; coreweave might go out of business but they won’t).
It’s about the fact that a lot of inference is essentially more about the layer of computed-human or AI-human or human-AI-human interaction than it is about the kind of work that a fully automated system does. I don’t think it’s as easy as the comparisons you draw. If you want a kind of dumb/funny example imagine if we’re in some kind of premodern agricultural scenario with LLMs (and literacy). We might actually use a lot of inference, send a lot of emails, we need a summary of the meeting about worker morale on the strawberry field, barley yields have been low this year due to slacking, Martin needs to stop spreading his weird disease, you two need to read up on crop rotation. This is all kind of slopwork. Now we replace fifty workers with one guy and some modern farm machinery, objectively the inference done is much lower. That’s true even if we replace that one guy with a multimodal combine harvester robot etc etc. Commoditization is more of a problem for compute than it is for the model providers. I used to agree with you and argued that view here extensively, but I think Mythos shows you that if you have even the hope of a true frontier model that has capability that no other model does you’re going to be able to extort entire sectors that rely on security especially (banks, defense, governments) at insane margins until everyone catches up. Most LLM work will be commoditized but the frontier release payoff will be high enough to keep the funding coming for the biggest players. Tokens/task is a bad metric, so we can use fully amortized compute (including across training/research costs) or whatever else you prefer.
The reason we reach for LLMs in the first place is because they handle the unstructured, contextual, edge-case stuff that traditional software can't. Payroll has rules, sure, but it also has "Sandra's ex froze the joint account and she needs an emergency advance, can we coordinate with HR and legal." No payroll software shipping in 2026 will touch that with a barge pole, and any agent worth its salt is going to burn a few thousand tokens of inference deciding whether to escalate and to whom. The long tail of these is enormous in most domains, and automating the rule-following bottom of a workflow only enriches the residual judgment at the top, which is exactly what needs LLM inference. It's why human accountants stayed employed after TurboTax. Same deal. Fewer humans to deal with.
This ignores a really interesting scenario where AI, being vastly cheaper and soon better than human coders, is able to write and test hugely complex software for a lot of these use cases that would be completely economically ridiculous today, but which will get cheaper over time, and then leash these to relatively low-intensity agents that use these tools. The simple argument is that instead of using Claude to compute 2+2 a million times, we just get Claude to code a calculator. You kind of dismiss this but I think a more fully featured version of this argument is actually quite compelling, especially when you count unfathomably wide-ranging improvements in token use efficiency that are coming not just for text but multimodal applications too. The US uses as much oil today (about 15-20 million barrels a day) as we did in the 1970s. Resource consumption numbers don’t just go up.
Yay? Look mom, I'm famous.
It’s sad, I’ve given it some of my recent posts and drafts (and random unpublished things I might get around to finishing at some point) and it doesn’t identify me (or a lot of other users here). There aren’t many (identified, I guess) NHS doctors in this sphere so I guess it’s a small world.
It’s a useful way of describing work that has been regulated into existence. For example, the EU passes legislation that requires some hugely complex and time consuming climate reporting for every company with an annual revenue of more than €10m. 100,000 companies now have to hire someone to be their ‘climate reporting officer’. The US healthcare system’s extensive regulation and lifetimes of case law about who pays and when and what insurance covers and what the hospitals have to provide etc etc create tens of thousands of jobs on both sides of the billing equation (the healthcare providers and the insurers) that don’t exist, or certainly don’t exist in the same sense, in single payer systems. Walmart wants to open in a town in Kentucky. The town offers large tax breaks in exchange for hiring 200 local people. A big Walmart in 2026 only needs 120 people to operate, though, but the tax breaks are worth more than that payroll. Numerous jobs as greeters and shelf stickers and security guards are created unnecessarily. A government contractor is tasked by a new government with proving that what it does at $500m a year in state billing is justified. It hires McKinsey for $20m to write a report, because nobody ever got fired for hiring McKinsey (including the minister who gets the report).
Individually these are examples of bloat, bureaucracy, overregulation, unintended consequences, inefficiency, corruption, graft, credentialism, whatever. But collectively, all of these are examples of bullshit jobs.
Trillions of dollars are being spent on building datacenters for inference. Amazon software engineers are inventing bullshit work for AI to inflate their internal usage scores.
I’m no expert, but isn’t there a fatal flaw here? Most of the work LLM inference is used for is essentially busywork that wouldn’t exist in an automated economy. It’s writing emails, it’s code reviews, it’s asking dumb questions, it’s transcribing or summarizing research or zoom meetings. Even in software engineering, a lot of LLM tokens are used in the kind of inference that a hypercompetent solo-coding model with limited or no human oversight just wouldn’t need.
Think of an office with 10 human employees working in, say, payroll, constantly sending each other emails, messages, having meetings, calling and speaking to each other and other people, summarizing documents, liaising with other departments, asking AI question about how to use various accounting tools, or about the company’s employee benefits package. Now say this department is automated. An AI model acts as an agent to use an already-existing software package to do all the payroll work. No emails, calls or meetings - or at least far fewer. The total inference work required goes down. And the existing software package doesn’t use AI (even if it may have been coded with it), because you don’t need AI to compute payroll data once you have sufficiently complex and customized software for your business.
In the same way, if we imagine our automated future, super high intensity / high token usage inference is actually not really universally required in a lot of occupations. It will be for some multimodal work (plumbing, surgery, domestic cleaning in complex physical environments), but for many tasks, one-and-done software coded either by AI or that already exists can just be deployed at low intensity by an agent. The AI that replaces your job might at first do a lot of coding, but as time goes on, the amount of novel inference required will diminish. Eventually, software coded in a one-and-done way by the AI may actually handle almost all the workload, and token usage for generation may be very limited to just some high level agent occasionally relaying instructions or performing oversight.
In this scenario, why would we expect inference workloads to shoot up so dramatically? Much enterprise AI usage is currently “fake” in the sense that it would not be performed in a fully automated environment. It’s a between-times thing.
Opus. Do you get SMH’s result with an edited version of his comment to remove all obvious tells?
More options
Context Copy link