This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
Alright, AI bros, follow-up from last week. I was able to secure access to Claude Opus 4.6 at my job, and I gave it the same prompt that I had given to Sonnet. It overlooked the authentication part of the HTTP client library completely this time in what it generated. In a follow-up I asked it to extract out the common logic for the authentication portions specifically. It didn't do that, instead it generated a class with two helper methods.
The first helper method was just a thin wrapper around System.Text.Json for deserializing the response. There's an optional flag to pass in for when case insensitive deserialization is needed, and nothing else.
The second helper method was something for actually making the HTTP calls. The strangest part with this one is that it has two delegates as parameters, one for deserializing successful responses, the other for handling (but not deserializing) error responses. It didn't do anything to split out handling of the 2 different ways to authenticate at all.
The issues with what was generated (for both the API client as a whole, and for the authentication part of the code specifically) are numerous, here are a small handful that I identified:
It assumes that an HTTP 200 code is the only successful response code, even though some endpoints return 202, 207, and more.
It assumes that all endpoints return plaintext or JSON content, even though several return binary data, CSV data, etc.
It didn't do null checking in several places. I assume it was mostly trained on C# code that either didn't do null checks correctly, and/or on code that doesn't use the nullable reference type feature that was added in C# 8 (back in 2019). Regardless, the null checks are missing/wrong regardless of whether nullable reference types are enabled or disabled. Also it always checks nulls with == or != null. This works 99% of the time, but best practice is to use "is null" and "is not null" for the rare cases where the equality operator is overloaded. Once again, I assume this is because most of the training data uses == and !=.
It doesn't handle url query parameters (nor path parameters), it assumes everything is going to use a JSON body for the request.
It uses the wrong logging templates for several of the logging calls. For example, the logs for an error response use the log template for logging the requests that are sent. Even more troubling is that it removed all the logic for stripping user secrets out of these logs.
There are quite a few more issues, but overall my experience with Opus was even worse than my experience with Sonnet, if anything. AI bros still in shambles. I definitely have zero fears that AI will replace me, though I'm still definitely fearful that retarded C-suite execs will think it can replace me.
My post from last week about using Claude Sonnet: https://www.themotte.org/post/3654/culture-war-roundup-for-the-week/426666?context=8#context
Edit: Just saw a very relevant post over on Orange Reddit about this very topic: https://news.ycombinator.com/item?id=47660925
Yeah I've been having some issues with Opus too recently as compared to Sonnet, probably they downgraded everyone since a new model is coming out soonish and they want more compute for that.
But I'm still getting good work done with it, working on a 100K LOC strategy game, which is not exactly boilerplate webdev slop. It is eminently possible to do pretty complex things with AI coding.
Why don't you tell it what exactly you've been finding that's wrong, update the memory file, have a correction pass go over it or do something a little more sophisticated in your orchestration instead of bitching about it online?
This 'I was able to secure access to Claude Opus 4.6 at my job, and I gave it the same prompt that I had given to Sonnet' sounds like you're just trying to one-shot it. You try, review, adjust, try again, have it go in from another angle and then it works.
You don't see me complaining about bugs it gives me, units chasing eachother such that they exit the map. I see a bug, I note it down, I fix it and try and work on a way to avoid similar things happening again.
There's a cavernous gulf between 'lived experiences' here, ironically this is what the motte is kind of for. It's self-evident to me that AI coding is great and effective, whereas it's self-evident to you that it isn't.
That sounds like more work than just writing the code yourself. It sounds like training an intern.
With the big difference that an intern will learn (at least ideally) and will need less handholding over time. The LLM does not learn, at least not directly from you showing it things.
Well I can't actually write code, so this way is a lot easier for me! I can also give high level observations and hypotheses, tell it to investigate further and get it to handle the nitty gritty stuff autonomously. Here is what I mean by 'adjust, try again, review':
The AI does actually learn, it has a memory file for the project that it autonomously made. Whether that actually helps much is open to debate. But I'm pretty satisfied with its performance. Occasionally it just fails and I say 'try again, think outside the box' and after a couple of times that usually works.
Also it's just a lot of code to manage. 3 MB of code is a fair bit I think, that's a lot for anyone to write.
Fair enough. I've had AI do things for me that I don't know how to do. I wouldn't stand behind the results, especially if I can't even judge them, but then again, that differs per use case. E.g., I'll use it for translating something from a foreign language if I want to read it, but I wouldn't use it to translate something I'd send out under my own name if I can't at least check it.
I note that people (and I've caught myself doing this too) tend to be bullish on AI when it comes to fields they don't know well, but bearish when it comes to fields they do.
That's only a little less than my employer's entire biggish, legacy enterprisey SAAS product with 20 years of history, if you're counting just the code. (Edit: at least the back-end. Probably about double if you count the front-end as well, but still.)
Are you counting assets such as graphics or really just the code itself? Perhaps the AI has simply generated really inefficient code. But hey, if it works.
Yeah, absolutely there are errors when it comes one-shotting things. It'll oneshot the bulk of what I want but there will be these little niggling flaws in integration or how it interacts with other features. I rely a lot on judging the outcome, trying and improving and AI troubleshooting.
There aren't really any graphics, just placeholders at this point. There's one fancy Rocaille-style procedural domain warping effect to create an FTL warp bubble animation but I count that as code since it's just maths.
I added one actual image icon for steel and that's in the assets folder, not the src folder that I count as the game.
It really is just 3 MB of code at heart, counting backend and frontend (a bunch of tabs and panels). There's a lot of stuff that it has to do, the ingame AI factions have to do their own war planning, province management, research, colonization and trade... I don't claim that they do these things particularly well, or that my game is bugfree. It absolutely isn't, I'm still making it. So perhaps your SAAS is more precise code. Nevertheless my game does work, it's totally possible to do large projects just via AI, without any coding skills. This used to be impossible, I tried back in the GPT-4 era and it was just a joke. But today it works out.
I've also been doing my own more boring SAAS projects for business use and they work too.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link