This weekly roundup thread is intended for all culture war posts. 'Culture war' is vaguely defined, but it basically means controversial issues that fall along set tribal lines. Arguments over culture war issues generate a lot of heat and little light, and few deeply entrenched people ever change their minds. This thread is for voicing opinions and analyzing the state of the discussion while trying to optimize for light over heat.
Optimistically, we think that engaging with people you disagree with is worth your time, and so is being nice! Pessimistically, there are many dynamics that can lead discussions on Culture War topics to become unproductive. There's a human tendency to divide along tribal lines, praising your ingroup and vilifying your outgroup - and if you think you find it easy to criticize your ingroup, then it may be that your outgroup is not who you think it is. Extremists with opposing positions can feed off each other, highlighting each other's worst points to justify their own angry rhetoric, which becomes in turn a new example of bad behavior for the other side to highlight.
We would like to avoid these negative dynamics. Accordingly, we ask that you do not use this thread for waging the Culture War. Examples of waging the Culture War:
-
Shaming.
-
Attempting to 'build consensus' or enforce ideological conformity.
-
Making sweeping generalizations to vilify a group you dislike.
-
Recruiting for a cause.
-
Posting links that could be summarized as 'Boo outgroup!' Basically, if your content is 'Can you believe what Those People did this week?' then you should either refrain from posting, or do some very patient work to contextualize and/or steel-man the relevant viewpoint.
In general, you should argue to understand, not to win. This thread is not territory to be claimed by one group or another; indeed, the aim is to have many different viewpoints represented here. Thus, we also ask that you follow some guidelines:
-
Speak plainly. Avoid sarcasm and mockery. When disagreeing with someone, state your objections explicitly.
-
Be as precise and charitable as you can. Don't paraphrase unflatteringly.
-
Don't imply that someone said something they did not say, even if you think it follows from what they said.
-
Write like everyone is reading and you want them to be included in the discussion.
On an ad hoc basis, the mods will try to compile a list of the best posts/comments from the previous week, posted in Quality Contribution threads and archived at /r/TheThread. You may nominate a comment for this list by clicking on 'report' at the bottom of the post and typing 'Actually a quality contribution' as the report reason.
Jump in the discussion.
No email address required.
Notes -
I've never encountered these issues and have had LLMs help me diagnose tons of computer issues and fix them. I pay for Claude Max. I've used Claude for coding and it has always delivered workable code b/c I still use the ol' fashioned software development life cycle i.e. plan, code, test & iterate. I test all the code I can (same as I do as a product manager with my human coders). I force Claude to explain new code and how it works and add it to the documentation which I review. Occasionally I do code reviews where I force Claude to make diagrams of how the code is interacting and then either add it to documentation or ask Claude to revise the code. Again... all stuff I've had to do as a PM with human coders b/c I'm not a great coder and left to their own devices human coders will go more off the range than even the dumbest LLM.
I think the difference as soon as I encounter something unhelpful or hallucinated I simply start a new chat with a summary of what I've tried so far. I never compact context but simply start a new chat.
LLMs should be compared with customer service agents... if an agent was unhelpful would you stay on the line or just hang up and call a new one? I hang up until I get someone helpful... LLM sessions should be used similarly.
Frankly whenever I hear about these problems with LLMs I just think you have to treat it like a person... would you as an engineering manager just let your coders go out and code and never check in on them again? Would you continue using an unhelpful human agent instead of someone who was helpful? Would you just let some random person control your life?
Seems pretty simple to me...
@P-Necromancer I think I'd like to bundle these two, as they're getting at a similar thing.
I agree with what you both say. Plenty of humans will come up with ridiculous things to do, or even just things that might make sense but have problems, and if you're not supervising them appropriately, they may just do their things. But that's like, the essence of technical debt?
For the example of fixing some OS issue, imagine I didn't have really any technical knowledge of how things work (say, I don't really even know what the registry is unless a tech/LLM tells me something about it). Maybe I'd take my computer to a human tech. Could even be a corporate IT guy. Perhaps, knowing that I don't have a clue, I just give it to him. "Here's my problem; please fix it
RalphRufus."Who knows what he'll get up to? What stuff he'll mess with along the way. Things he'll try just because, and then maybe leave it in a changed state, even though it didn't progress toward a solution to the actual problem. This cruft can build up. After years of having this corporate IT guy and that corporate IT guy and the other corporate IT guy just doing who knows what, maybe at some point, things get bizarre enough that the next one says, "Dude, stuff is wild here; we probably should just wipe it and clean install."
That makes sense, and it's utterly routine in the world with humans. I hear my wife tell me about weird stuff that's broken on her work computer... and even weirder stuff that whatever IT guy she talked to did. She doesn't have a clue what's going on. I get it.
I also agree that as of right now1, the best is when you know enough about what's going on that you can get it to explain things and are able to then understand it, yourself. Get it to document things fully, provide a suite of tests, have a back-and-forth. It can provide tons of utility!2
...but, if you genuinely lack enough knowledge to be a competent participant of that back-and-forth, it still may let you "just do stuff". There can still be tons of utility here, as it may still get things right a lot, and folks who have had some problem that they've wanted to fix for ages and could never get the time with a competent human and certainly couldn't figure it out on their own will be able to fix many of those problems, and it will be wonderful. It may also, occasionally, along the way, build up technical debt.
Note that I'm not saying that this is some unique problem that is fundamentally different from dealing with humans. Instead, I'm now conceptualizing it in the same way that I conceptualize human-driven technical debt. I think that dovetails well with both of your descriptions. If there is a downside, it's probably that many folks who wouldn't have ever tried to fix that OS problem or make that code will now do it, and they might be building up technical debt while they're also accumulating utility. They may choose to do it a lot, and they may jump into it with both eyes shut. This may still be the right choice! They may still get more utility from all the wins than they lose from either discrete bad events or built-up cruft.
This is a conflict, a tension, which is why I said that I was, indeed, conflicted. I'm am still neither an "LLM good" or "LLM bad" person.
1 - I continue to take no position on the question of to what extent future progress will render this concern de minimis.
2 - To briefly respond to the 'shouldn't you just hang up on a human customer service agent who you can tell is going to be unhelpful', yes. Absolutely. I didn't bother with the specific issue of it getting hung up on deleting the registry value, because I was close enough that hearing it append its bad idea one more time wasn't important to me. I did mention that I used multiple LLMs, and that was part of it; I left out every twist and turn of the story, but yeah, I not only just scrapped the prior context; I even just jumped to different models. This is a useful skill to have, when dealing with humans and LLMs. Even when dealing with some human professionals, my life changed long ago when I realized that I could grasp some understanding of what their "box" of the world was, and once I realized that my situation was outside of their "box", I just moved on from them. But the concern here is that you have to have just enough knowledge about the thing to be able to gauge where their box is, when you're outside of it, or when they're going off the rails. There are a lot of people who don't have that with humans, and they're not going to have that with the many many more things that they're going to want to do with LLMs. I don't have that with all sorts of different humans or things that I might want to do with LLMs.
Sure, I don't disagree with anything here. Or really anything in the OP; just adding my two cents and offering a couple tips for making productive use of LLMs.
I think there are two different use cases here it makes sense to distinguish. This is an example of allowing the LLM to act 'directly' (not actually directly, there's a human in the loop, but it's giving you commands to execute, not writing a script) on a complex, persistent system. Which, yeah, that can absolutely build up cruft that's difficult or impossible to clear away without starting fresh. But even the most careless vibe coding has a serious advantage, in that the actual operations are recorded and auditable. If you put in a tiny bit of effort and use version control, you (or someone else, or another LLM) can even audit how the code changed over time. And, better, you can separate out tasks into different, independently tested scripts to be sure there isn't some complicated interdependence issue. It's the difference between manually tinkering with a machine and writing a dockerfile. It's still certainly possible to build up technical debt to the point you're better off starting fresh, but it's a lot harder. At least for small personal projects, which I hope are most of the things people do make this way.
Careless vibe coding carries real risks; I haven't caught a model trying to do anything dangerous (as opposed to dumb), but I believe the people who say they have. I'd be very leery of running code I can't understand at least well enough to tell if it's making web calls or deleting things it shouldn't be. (But I'd say the same for StackOverflow.) I double check the library names. I wouldn't let it touch anything security-critical, or any files I care about and don't have backed up. I haven't pushed any generated code to a public repo, but if I did, I'd be very careful to ensure there aren't any api keys or passwords or other secrets anywhere in history.
It is... concerning that same tools are available to people less cautious and knowledgeable than me, and I'm certain that will lead to problems. (On the other hand, I'm sure there are people who'd put me into that group.) Enough to make the whole endeavor net-negative? Hard to say, but I'm pretty sure the answer is 'no.' At least, I think someone smart enough to get Antigravity or Claude Code or whatever running ought to be smart enough to understand the big dangers and a few basic principles of good, maintainable code with a short crash course-- which, actually, the LLM is very capable of providing, even if it can't (perfectly) reliably avoid those pitfalls.
More options
Context Copy link
More options
Context Copy link
More options
Context Copy link