This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
One of my tiny clients was billed over $10,000 recently because their customer-facing chatbot got locked in a loop with somebody's openclaw agent.
An acquaintance works for a small to mid sized private software company, and he alleges that they put restrictions on AI use after they already hit seven figures this month.
Overall, I don't know if that story is true, but what I'm seeing on the ground strongly suggests that executives really don't have the slightest fucking clue what's happening until the bill lands on their desk.
Why didn't they set a spend limit? Can't get tinier or more amateurish than my projects. I have a spend limit set up for precisely this situation, or if I get my API keys stolen. I'm pretty sure spend limits are on by default!
I guess there's just a huge gulf in discipline between businesses of similar sizes, the small companies I work with take pains to track all the dollars in their accounting.
I am less familiar with the AI space specifically, but cloud compute providers like AWS are famous for making setting up spending limits difficult generally. It's certainly convenient for them from a business perspective, but to be fair it is often unclear what "limit" would mean in the general case: nobody likely wants "we deleted all your data in S3 so you didn't exceed your spending limit on data storage", for example.
More options
Context Copy link
The small client didn't set a limit because another salesman/contractor set it up for them without a limit. They are not technical people.
The other company was pushing a tokenmaxxing scheme and got burned by their own choices.
That seems to represent severe negligence on the part of the person who set it up, with foreseeable consequences. The limits are AFAIK turned on by default so the salesman must have turned them off. There's a decent chance they could get their money back in the courts IMO. Especially if it was a salesman for a larger company not a contractor.
If you went to court the company would probably just say that it's their policy to have the limits on by default, which would of course be true. You would then have to prove that it's the salesman who turned off the limits instead of you. Good luck doing that.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
This raises the question- how much does a token cost, and how much does it do? I'm given to understand the standard software engineering productivity line is 1 line of code/hour, is a token more or less than that(and does it cost more than an hour of a software engineer's time)?
This discussion of tokens is sort of like 'the carpet costs 40,000 Kazakhi tenges(to pick random foreign money I'm not familiar with)'. OK, sounds like a lot, but how much is a tenge? If it's worth as much as yen, that's not a bad deal.
[apologies if I'm answering a rhetorical question]
A 'token' is the ML equivalent of a syllable: the word, portion of a word, or symbol that represents the smallest viable input or output unit of effort. The exact value (and cost) depends on model, as well as whether it's input or output.
I'm ... skeptical to endorse line-of-code as a measure of programmer time -- I've spent days planning out business-critical logic that ended up five lines of code and needed to be absolutely correct, and spat out thousands of lines of text in an hour before when it was just boilerplate -- but the output side you can give a pretty good average. Depends on the tokenizer and your output language, but I'd expect less an average of less than 20 tokens per line in C++ or TypeScript, and I'd get worried if a human coder was regularly writing >80 tokens in a single line.
((edit: less so in java.))
So pessimistically, 12.5k LoC per million tokens, more realistically 25k.
Input is the high-variance part. If you're writing from scratch, the input is a few paragraphs and some design documents, maybe some scribbled image files if you feel spicy and the model supports it. I've done a few personal projects like that where it's been <2k tokens to get 20k line-of-code. If you have an existing codebase you want the model to adjust to, or an API document you need the model to learn, that can burn through a lot of tokens fast; the only real restriction is context window size, and most of the corporate APIs obfuscate that (tbf, often because they have an automated store and search strategy). I've blown through 50k in a single search once (thanks, Atmel, love your manual layout too). Input is typically cheaper and there's some strategies to reduce the cost of repeated input hits with the same content, but they're complicated and pretty specialized.
For some examples:
For smaller or more efficient models, inference is pretty cheap: Qwen3.6 35B's probably the weakest coding model I'd use in a professional environment (and borders the point where it might be better to run it locally, if only for privacy/security reasons), but there's a lot you can do.
That said, all of that can go out of the window when you start getting agentic options involved. Someone made a fun experiment of trying to let a local model figure out a display protocol by hooking a camera, an LLM, and a microcontroller together, and they got it mostly there overnight, which is really cool. It also probably burned tens of millions of tokens on output for an interface code block that should have ended up in the <1.5k line-of-code level.
Hell, I'm at negative LoC for the year so far.
Deleting code feels a million times better than writing it
More options
Context Copy link
More options
Context Copy link
So TL;DR, tokens are cheaper than basically all white collar workers in the modern west, but they get overused for fun projects?
Largely (though Mythos approaches white collar wages in terms of dollars per hour at API rates). But it's not like all tokens go straight to code written. Tokens are more like measuring thought.
When I give Mythos/Fable instructions it first goes 'I'll explore the codebase' and so it searches for relevant things (those search commands are output tokens). Then it reads files which have the relevant data, more input tokens. Then it thinks for a while (that's output tokens). Then it makes its to do list. Then it reads some more, thinks some more. There are pages and pages of just reading and thinking before it goes 'i have a full picture'. Then it starts editing code!
Then it'll try and test if it actually works, often writing some test cases, so that's more code. Then it tells me everything it did in summary and adds stuff to its memory files.
So a lot of thought is happening even if it only adds a few pieces here and there for a new feature.
Yeah, the "thinking" process itself also counts as output tokens. When you use a reasoning model, it's basically writing a long monologue about how it's going to solve your problem and then immediately throwing it away at the end. (Different providers have different policies about whether you're allowed to see this monologue, but it often significantly exceeds the length of the actual code or whatever that the AI is writing.)
So, I'm kind of clueless about this, but are reasoning models are actually different models, as in different neural net weights?
Like, do you get a reasoning model by running a single-pass model in a loop where you feed it prompts like: "first, understand the problem and make a plan for solving it, formatted like this", then "here's the plan you thought up before, try to execute point 1 now", and so on?
Or do you need a different model specially-trained for this kind of thing and it's a big secret black box how it all works?
Yes. Generally, reasoning models are trained to use special "start thinking" and "stop thinking" tokens, and to generate a specific kind of monologue in between those tokens. Similar to how RLHF biases models towards producing text that's appealing to human readers, reasoning models use techniques like RLVR to bias towards generating monologues that end in a correct solution to a problem.
Many reasoning models are trained in a way that lets you disable the reasoning by forcing them to never generate the "start thinking" token -- Claude Opus 4.8 probably uses the same weights regardless of whether you enable or disable thinking, for example -- but their weights are different from models that were never trained for reasoning in the first place.
With that being said, people used to use "chain of thought prompting" to get a similar kind of result out of regular LLMs. (I think reasoning models basically got started when AI companies saw the early success of chain-of-thought prompting and started baking it in at the training stage.)
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More that they're cheaper than a code monkey, only weakly expensive if you're doing something hard or novel (or novel-to-you), and they can get ludicrously expensive if you just start firing the slop cannons, either to solve a problem by volume or by producing a lot of useless or specialized lines-of-code.
More options
Context Copy link
And because they allow brute-forcing problems in ways that weren't possible before, or tackling new problems. Or because the user cocks up, as in the case where they fail to notice their two bots getting into an infinite loop.
More options
Context Copy link
The problem is that a token is cheap, but the amount of tokens you need to do useful things can be very high.
For agentic programming, the agent needs to hold a non trivial amount of the codebase in the context. That can easily be millions of tokens. Then you have whatever pile of "skills" (read: markdown files) you use, then add the various layers of prompts, then add reasoning chains. It adds up very quickly.
Once you start adding parallel agents and loops, it can get insane.
It is weird to think back to the time when bringing out a 4k token model was a massive deal. You could hold, like, paragraphs in context. Like, nearly a whole chapter of an actual book.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
A token is broadly a word. 'Tokenisation' is what happens because AI is fundamentally mathematical, and so it only works on numbers, so we have to turn language into numbers.
So if a line of code looks like:
Then that's
where each | is a split between tokens.
So that line is 11 tokens: one for each common word, one for each element of punctuation. In practice there's a big table with each word and each punctuation assigned to different numbers, and it looks up the numbers, so that line of code gets turned into the numbers (tokens)
and each of those numbers costs a certain amount to process. Each new token, i.e. each new word and punctuation mark it produces costs another (considerably larger) amount. There's a certain amount of complexity around how to represent numbers for maths, and some fairly commonsense rules about how to split up words that are uncommon, so "unfireable" might become the three tokens for un|fire|able and "garbleflarg" might have to be spelled out letter by letter with each letter being a separate token.
(Note: this is how the little guys like me do it. Tokenisation for Anthropic might be something much more advanced.)
How much a token is worth depends on what it's doing for the customer. If it's a word in my romance novel then it's
pricelessnot worth much, if it's a word in the code for my startup, it's either worthless or very valuable depending on how that startup turns out. Or if you like you can decide it's worth what it would cost to get a suitable skilled human to do it, which is generally how the big companies value it.More options
Context Copy link
A good mental model for a token is 3-4 characters. I'm assuming somebody will come in with an example of how it's wrong, but it's not a terrible heuristic.
A line of code is usually between 1 and 200 characters, with a fat part of the curve sitting around 80-100 characters.
More options
Context Copy link
More options
Context Copy link
I'm sorry for the company because that kind of money can hurt a small company, but this is just too perfect. Do away with even the callcentre human element for the sake of "save money, more efficiency" and you end up with no real humans in the loop, just bots jabbering at one another.
These are the days of progress towards the Singularity, just believe in and hold fast to the shining vision of paradise on Earth!
A whole lot of people died due to boiler explosions in the early days of the industrial revolution, but in the end it all turned out quite awesome. (In the old sense of being grand and powerful. Don't @ me, Uncle Ted fans.)
This pattern of argument is not convincing -- where people bring up anecdotes about most spectacular fuck-ups and then use them as basis for sarcastically mocking the idea that anything useful could come from this.
The sarcastic mocking was more about "this is where the grand dreams of artificial minds that are better than human has ended up, money-scamming machines communicating mindlessly with each other to burn time and money". This is what we do, we turn visions into "how can I scrape pennies out of this?"
Great things may come later, but we'll still be trying to scrape pennies.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
Anecdote from a previous job that you just gave me a PTSD flashback to, back when I worked on the help desk and sysadmin side of things instead of development:
The company I was working at got acquired by another company (most Americans here would probably know the acquiring company but I won't get any more specific than that). Our accounting department was mostly laid off and we now had to send our invoices to accounting in the acquiring company to get them paid.
My boss starts sending them our invoices for the phone company (and internet, and a bunch of other important things) up to corporate accounting every single day because accounting isn't responding to him and isn't paying our bills. He's also calling them multiple times a week, but no one is answering. Also our long distance phone service is separate from our main phone service (this will be important) for complicated reasons I never bothered to learn because I was a help desk grunt at the time.
This goes on for months, and the phone company is getting pissy and threatening to cut off our phone service. My manager is forwarding the service cut off threats to accounting too. Finally long distance service actually gets cut off (but local phone service still works), and me and the other help desk grunt got flooded with about 200 calls from pissed off users that day.
This causes enough of a stink that corporate catches wind of it and ask my manager why he wasn't paying the phone bill. After all, they showed him how to send invoices to accounting, etc., how could he be so irresponsible? My manager whips out 60+ emails and his phone call logs and corporate immediately apologizes and presumably goes to bite off someone's head in accounting.
Perhaps this is a stupid question, but why couldn't your manager escalate to his boss until it gets to someone who can do something about it?
Yeah, I assumed reading the story that something must have been communicated up the chain because without that, it just looks a lot like malicious compliance.
If I recall correctly, my manager's new manager was someone in IT at the corporate HQ and they got some blowback over it too because my manager had reached out to them several times about accounting not responding.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
I believe your war story, because I've seen the likes of it in a previous job. Being as vague as possible, two separate entities were amalgamated into One Big Happy Family for the sake of efficiency and cost-cutting and other fun management notions. It was all going to be peachy, the matters handled by both halves would now all be handled in the post-amalgamation blob and this would mean Better Customer Service and More Responsiveness and the other buzz words.
First thing to blow up was the annual Christmas party (I hadn't worked there long enough to have gone to previous parties). Before, both places had their own parties and management of both places threw a few bob in the respective funds for it. Afterwards, entity A (based in the city) would not come down to our town, and entity B (based in our town) would not go to the city to host it, because transport (everyone wanted to get blotto on free-ish booze because it was the Christmas party, nobody wanted to have to be sober enough to drive home). Plans to arranging hire of private buses (so people would be collected and then dropped back home or near enough) foundered on "yeah, but where will we hold it?" There wasn't any compromise "here's a nice restaurant or hotel halfway between both places", so it ended up no Christmas party at all for anyone.
That was the kind of co-operation and mutual understanding which developed between the two halves, which post-amalgamation continued on with "we do our stuff, you do your stuff, we don't interact or co-operate any more than absolutely necessary".
I imagine the reason for the accounting snafu was a combination of acquiring company accounting department going "Well nobody told us we were supposed to pay their bills" so the bills coming in got shoved into a pile on somebody's desk and ignored, and "who the hell are these guys, never mind, go to the bottom of the heap while we deal with the really important payments for our main office".
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link