This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Does anyone else feel like we're heading for a good old-fashioned, 2008-tier, financial crash? I recall people bringing up the possibility ever since Covid bucks started rolling out, but even though we were due for one, and even though the money printer was going brrrrr, the crash has so far failed to materialize.
At the time, I was of two minds about this. On one hand, all the libertarian theory I used to subscribe to said money-printing => boom, boom => bust. On the other hand, the problem for me was it never felt like a boom, and I think this is changing now. A key feature of the pre-crash boom is "malinvestment"; capital going into often downright deranged projects, that are later abandoned half-finished. Well, I feel like the datacenter craze qualifies, and with the wave of AI IPOs that are coming just as major indices are changing their rules, to allow for these companies' near-instantaneous inclusion (an investment so good, you can't pass up on it. Literally, if you're American), it seems like we're solidly in the "irrational exuberance" phase.
Add this to the list of things I hope I'm wrong about, because if we get a proper crash, the political fallout is going to be massive. The script writes itself: Trump / tech bros / capitalism bad, even more gay race communism now.
To whitepillers: is there an argument for why I'm wrong that doesn't boil down to "you don't get it, chud, it's the New Economy! The Singularity is just around the corner! All the rules are obsolete!"? This argument is verboten, because this is pretty much what people say with every bubble.
To fellow blackpillers: any ideas on how to brace for impact? Any IT guys here old enough to make it through the dotcom bubble? How did you do it? Any advice you would have given your past self?
If there's a crash, I predict it'll be due to energy prices and the war in Iran, not AI.
Anthropic is nearly profitable, or supposed to have been profitable in Q2. Some say that's phoney accounting and Anthropic says profits may not be maintained. But it seems that major investments are paying off. Furthermore, AI models are getting continually better as with Fable most recently. This trend will continue, bigger and better models working for longer need more compute to run them and so justify further intensifying investment. I guess that argument is forbidden by your post. But if it is The Singularity, if it is a New Paradigm then presumably that's good for stocks!
Anyway, energy is more important to the economy than AI right now. You can't just shut off a huge amount of oil and gas production without ramifications! The 1973 oil shock is a useful precedent. The market seems to have been expecting peace talks to advance more smoothly than they actually have been. Bombing has just resumed. Iran has announced they're re-closing the straits and possibly the other straits in the Red Sea too.
I'm suspicious about that given stories like this.
Maybe they had a revenue-positive quarter, but if it was buoyed by various companies overshooting their AI budget, this might not be an actual trend.
I've been suggested this was even an intentional move by an Anthropic investor to goose the company's valuation by artificially increasing their ARR (claims, not substantiated).
Not quite. The U.S. has been a net energy exporter since 2019. So domestic energy production can more than satisfy our needs for the foreseeable future.
That's before we get into the possibility of nuclear coming online.
Yes, higher oil prices impact the U.S. too, but we're not going to be the first ones to tap out, in either a literal or metaphorical sense.
Remember we just secured a massive source of nearby oil.
One thing about markets, they respond to shifts in supply and demand.
How do you even do that, are we supposed to believe that some company with $500 million to spend didn't know how to use cost tracking? Cost tracking comes with the pleb-tier management tools I get!
And if they were trying to goose the valuation then why would they admit it was unintentional, it makes Anthropic sound expensive? (They are expensive). It sounds more like an anti-Anthropic story to me.
The story sounds greatly exaggerated or misleading IMO. Who is this mystery company too, what are the details on this? More likely some company just consciously spent a lot of money on Claude AI and then some reporter fluffed it up into a narrative we all hear instead of a boring article that goes nowhere.
Regarding energy, even though the US is a major producer and isn't as badly affected as other countries like Australia, Europe or poor countries, the world economy is global. Problems in Asia will spread to America. The US is busily exporting oil, including the strategic reserve releases, to take advantage of price gaps and stabilize markets. But this is a temporary fix. The price gaps will narrow. The invisible hand of the market will slowly but surely squeeze the US economy if this price pressure remains. Even if oil producers profit, much more of the US economy relies on truck transport, plastics, feedstock, jet fuel, lubricants and all the other chemicals which support industrial civilization.
Venezuelan oil is low quality and requires years of patient investment and capable administration to realize much net gains from. Crude oil production there is barely half of what it was in 2016 and shows no signs of making up for the current supply loss. Nobody is a bigger fan of nuclear than me but nuclear energy is not going to lower fuel prices in a matter of months.
https://www.ceicdata.com/en/indicator/venezuela/crude-oil-production
One of my tiny clients was billed over $10,000 recently because their customer-facing chatbot got locked in a loop with somebody's openclaw agent.
An acquaintance works for a small to mid sized private software company, and he alleges that they put restrictions on AI use after they already hit seven figures this month.
Overall, I don't know if that story is true, but what I'm seeing on the ground strongly suggests that executives really don't have the slightest fucking clue what's happening until the bill lands on their desk.
Why didn't they set a spend limit? Can't get tinier or more amateurish than my projects. I have a spend limit set up for precisely this situation, or if I get my API keys stolen. I'm pretty sure spend limits are on by default!
I guess there's just a huge gulf in discipline between businesses of similar sizes, the small companies I work with take pains to track all the dollars in their accounting.
I am less familiar with the AI space specifically, but cloud compute providers like AWS are famous for making setting up spending limits difficult generally. It's certainly convenient for them from a business perspective, but to be fair it is often unclear what "limit" would mean in the general case: nobody likely wants "we deleted all your data in S3 so you didn't exceed your spending limit on data storage", for example.
More options
Context Copy link
The small client didn't set a limit because another salesman/contractor set it up for them without a limit. They are not technical people.
The other company was pushing a tokenmaxxing scheme and got burned by their own choices.
That seems to represent severe negligence on the part of the person who set it up, with foreseeable consequences. The limits are AFAIK turned on by default so the salesman must have turned them off. There's a decent chance they could get their money back in the courts IMO. Especially if it was a salesman for a larger company not a contractor.
If you went to court the company would probably just say that it's their policy to have the limits on by default, which would of course be true. You would then have to prove that it's the salesman who turned off the limits instead of you. Good luck doing that.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
This raises the question- how much does a token cost, and how much does it do? I'm given to understand the standard software engineering productivity line is 1 line of code/hour, is a token more or less than that(and does it cost more than an hour of a software engineer's time)?
This discussion of tokens is sort of like 'the carpet costs 40,000 Kazakhi tenges(to pick random foreign money I'm not familiar with)'. OK, sounds like a lot, but how much is a tenge? If it's worth as much as yen, that's not a bad deal.
[apologies if I'm answering a rhetorical question]
A 'token' is the ML equivalent of a syllable: the word, portion of a word, or symbol that represents the smallest viable input or output unit of effort. The exact value (and cost) depends on model, as well as whether it's input or output.
I'm ... skeptical to endorse line-of-code as a measure of programmer time -- I've spent days planning out business-critical logic that ended up five lines of code and needed to be absolutely correct, and spat out thousands of lines of text in an hour before when it was just boilerplate -- but the output side you can give a pretty good average. Depends on the tokenizer and your output language, but I'd expect less an average of less than 20 tokens per line in C++ or TypeScript, and I'd get worried if a human coder was regularly writing >80 tokens in a single line.
((edit: less so in java.))
So pessimistically, 12.5k LoC per million tokens, more realistically 25k.
Input is the high-variance part. If you're writing from scratch, the input is a few paragraphs and some design documents, maybe some scribbled image files if you feel spicy and the model supports it. I've done a few personal projects like that where it's been <2k tokens to get 20k line-of-code. If you have an existing codebase you want the model to adjust to, or an API document you need the model to learn, that can burn through a lot of tokens fast; the only real restriction is context window size, and most of the corporate APIs obfuscate that (tbf, often because they have an automated store and search strategy). I've blown through 50k in a single search once (thanks, Atmel, love your manual layout too). Input is typically cheaper and there's some strategies to reduce the cost of repeated input hits with the same content, but they're complicated and pretty specialized.
For some examples:
For smaller or more efficient models, inference is pretty cheap: Qwen3.6 35B's probably the weakest coding model I'd use in a professional environment (and borders the point where it might be better to run it locally, if only for privacy/security reasons), but there's a lot you can do.
That said, all of that can go out of the window when you start getting agentic options involved. Someone made a fun experiment of trying to let a local model figure out a display protocol by hooking a camera, an LLM, and a microcontroller together, and they got it mostly there overnight, which is really cool. It also probably burned tens of millions of tokens on output for an interface code block that should have ended up in the <1.5k line-of-code level.
Hell, I'm at negative LoC for the year so far.
Deleting code feels a million times better than writing it
More options
Context Copy link
More options
Context Copy link
So TL;DR, tokens are cheaper than basically all white collar workers in the modern west, but they get overused for fun projects?
Largely (though Mythos approaches white collar wages in terms of dollars per hour at API rates). But it's not like all tokens go straight to code written. Tokens are more like measuring thought.
When I give Mythos/Fable instructions it first goes 'I'll explore the codebase' and so it searches for relevant things (those search commands are output tokens). Then it reads files which have the relevant data, more input tokens. Then it thinks for a while (that's output tokens). Then it makes its to do list. Then it reads some more, thinks some more. There are pages and pages of just reading and thinking before it goes 'i have a full picture'. Then it starts editing code!
Then it'll try and test if it actually works, often writing some test cases, so that's more code. Then it tells me everything it did in summary and adds stuff to its memory files.
So a lot of thought is happening even if it only adds a few pieces here and there for a new feature.
More options
Context Copy link
More that they're cheaper than a code monkey, only weakly expensive if you're doing something hard or novel (or novel-to-you), and they can get ludicrously expensive if you just start firing the slop cannons, either to solve a problem by volume or by producing a lot of useless or specialized lines-of-code.
More options
Context Copy link
And because they allow brute-forcing problems in ways that weren't possible before, or tackling new problems. Or because the user cocks up, as in the case where they fail to notice their two bots getting into an infinite loop.
More options
Context Copy link
The problem is that a token is cheap, but the amount of tokens you need to do useful things can be very high.
For agentic programming, the agent needs to hold a non trivial amount of the codebase in the context. That can easily be millions of tokens. Then you have whatever pile of "skills" (read: markdown files) you use, then add the various layers of prompts, then add reasoning chains. It adds up very quickly.
Once you start adding parallel agents and loops, it can get insane.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
A token is broadly a word. 'Tokenisation' is what happens because AI is fundamentally mathematical, and so it only works on numbers, so we have to turn language into numbers.
So if a line of code looks like:
Then that's
where each | is a split between tokens.
So that line is 11 tokens: one for each common word, one for each element of punctuation. In practice there's a big table with each word and each punctuation assigned to different numbers, and it looks up the numbers, so that line of code gets turned into the numbers (tokens)
and each of those numbers costs a certain amount to process. Each new token, i.e. each new word and punctuation mark it produces costs another (considerably larger) amount. There's a certain amount of complexity around how to represent numbers for maths, and some fairly commonsense rules about how to split up words that are uncommon, so "unfireable" might become the three tokens for un|fire|able and "garbleflarg" might have to be spelled out letter by letter with each letter being a separate token.
(Note: this is how the little guys like me do it. Tokenisation for Anthropic might be something much more advanced.)
How much a token is worth depends on what it's doing for the customer. If it's a word in my romance novel then it's
pricelessnot worth much, if it's a word in the code for my startup, it's either worthless or very valuable depending on how that startup turns out. Or if you like you can decide it's worth what it would cost to get a suitable skilled human to do it, which is generally how the big companies value it.More options
Context Copy link
A good mental model for a token is 3-4 characters. I'm assuming somebody will come in with an example of how it's wrong, but it's not a terrible heuristic.
A line of code is usually between 1 and 200 characters, with a fat part of the curve sitting around 80-100 characters.
More options
Context Copy link
More options
Context Copy link
I'm sorry for the company because that kind of money can hurt a small company, but this is just too perfect. Do away with even the callcentre human element for the sake of "save money, more efficiency" and you end up with no real humans in the loop, just bots jabbering at one another.
These are the days of progress towards the Singularity, just believe in and hold fast to the shining vision of paradise on Earth!
A whole lot of people died due to boiler explosions in the early days of the industrial revolution, but in the end it all turned out quite awesome. (In the old sense of being grand and powerful. Don't @ me, Uncle Ted fans.)
This pattern of argument is not convincing -- where people bring up anecdotes about most spectacular fuck-ups and then use them as basis for sarcastically mocking the idea that anything useful could come from this.
The sarcastic mocking was more about "this is where the grand dreams of artificial minds that are better than human has ended up, money-scamming machines communicating mindlessly with each other to burn time and money". This is what we do, we turn visions into "how can I scrape pennies out of this?"
Great things may come later, but we'll still be trying to scrape pennies.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Anecdote from a previous job that you just gave me a PTSD flashback to, back when I worked on the help desk and sysadmin side of things instead of development:
The company I was working at got acquired by another company (most Americans here would probably know the acquiring company but I won't get any more specific than that). Our accounting department was mostly laid off and we now had to send our invoices to accounting in the acquiring company to get them paid.
My boss starts sending them our invoices for the phone company (and internet, and a bunch of other important things) up to corporate accounting every single day because accounting isn't responding to him and isn't paying our bills. He's also calling them multiple times a week, but no one is answering. Also our long distance phone service is separate from our main phone service (this will be important) for complicated reasons I never bothered to learn because I was a help desk grunt at the time.
This goes on for months, and the phone company is getting pissy and threatening to cut off our phone service. My manager is forwarding the service cut off threats to accounting too. Finally long distance service actually gets cut off (but local phone service still works), and me and the other help desk grunt got flooded with about 200 calls from pissed off users that day.
This causes enough of a stink that corporate catches wind of it and ask my manager why he wasn't paying the phone bill. After all, they showed him how to send invoices to accounting, etc., how could he be so irresponsible? My manager whips out 60+ emails and his phone call logs and corporate immediately apologizes and presumably goes to bite off someone's head in accounting.
Perhaps this is a stupid question, but why couldn't your manager escalate to his boss until it gets to someone who can do something about it?
Yeah, I assumed reading the story that something must have been communicated up the chain because without that, it just looks a lot like malicious compliance.
If I recall correctly, my manager's new manager was someone in IT at the corporate HQ and they got some blowback over it too because my manager had reached out to them several times about accounting not responding.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I believe your war story, because I've seen the likes of it in a previous job. Being as vague as possible, two separate entities were amalgamated into One Big Happy Family for the sake of efficiency and cost-cutting and other fun management notions. It was all going to be peachy, the matters handled by both halves would now all be handled in the post-amalgamation blob and this would mean Better Customer Service and More Responsiveness and the other buzz words.
First thing to blow up was the annual Christmas party (I hadn't worked there long enough to have gone to previous parties). Before, both places had their own parties and management of both places threw a few bob in the respective funds for it. Afterwards, entity A (based in the city) would not come down to our town, and entity B (based in our town) would not go to the city to host it, because transport (everyone wanted to get blotto on free-ish booze because it was the Christmas party, nobody wanted to have to be sober enough to drive home). Plans to arranging hire of private buses (so people would be collected and then dropped back home or near enough) foundered on "yeah, but where will we hold it?" There wasn't any compromise "here's a nice restaurant or hotel halfway between both places", so it ended up no Christmas party at all for anyone.
That was the kind of co-operation and mutual understanding which developed between the two halves, which post-amalgamation continued on with "we do our stuff, you do your stuff, we don't interact or co-operate any more than absolutely necessary".
I imagine the reason for the accounting snafu was a combination of acquiring company accounting department going "Well nobody told us we were supposed to pay their bills" so the bills coming in got shoved into a pile on somebody's desk and ignored, and "who the hell are these guys, never mind, go to the bottom of the heap while we deal with the really important payments for our main office".
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I agree that the story is very vague, some mysterious unnamed company ran up that much of a bill? But the plausibility lies in if the company didn't understand what they were buying - they thought the licence worked like the usual software-as-a-service subscription and you just buy a licence for everyone who is going to use the product. They didn't realise they were being charged for token usage or how tokens worked.
As a cautionary tale, this hit my own workplace where unexpected heavy usage of photocopying meant big charges for excess monthly, to the utter surprise of the boss who thought we were paying for all that with the monthly service charge anyway. Nope, this is why you always read the fine print (because the buggers selling you the crap are always going to present the best face of it then hit you with the "and also if you do this, this, or that, which you will do because that's what you need to do, we charge you an excess" after you've signed the contract).
More options
Context Copy link
According to social media, it seems that several companies have ordered their employees to maximise token usage, thinking that this would launch their company into the future. I don't know about the $500M figure specifically, but it seems entirely plausible that this practice would lead to a ton of money being wasted.
More options
Context Copy link
I've not seen a single proper source for that 500m claim either, but everyone seems to just accept it at face value.
The one time I saw something like an original image for the claim, it was from an Indian subreddit (or twitter, I can't remember exactly) and had no currency attached. So if it were 500m rupees, that would only be 5m USD. That would make a lot more sense to me
Indians tend to use Lakhs and Crores though
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link