ZorbaTHut
19d ago
It's theoretically possible it's a bug with a specific browser; what browser are you on?
ZorbaTHut
27d ago
Yeah. I'm very glad I set things up in a way where I could just move the servers easily.
ZorbaTHut
27d ago
There's a bunch of problems here. Yes, China traffic spiked, but it didn't spike that much. Also, now that the analytics have caught up, it spiked a day or two before the server problems began, and everything's back to normal. So . . . was it bots? Eh, maybe, maybe not. And while I can get a long list of IPs, it's not clear that there was any significant pattern within that.
ZorbaTHut
27d ago
If you had an employee whose work you had to check every single time, you'd fire him.
In most of the programming jobs I've been in, code reviews are considered mandatory for all programmers. Everyone's work is checked every single time, and yet we don't all get fired. Humans make mistakes, and we've set up systems to better solve that issue. So do computers pretending to be humans. Nothing out of the ordinary here.
There's a lot of cases where figuring out how to solve a problem is far more complicated than verifying the solution, and those are cases that LLMs are fantastic with.
ZorbaTHut
27d ago
I don't know what you've tried, but:
- Claude is probably the best-integrated system out there; use the commandline tool because it's able to go research your codebase on its own.
- Make it a CLAUDE.md with general architecture. It can make a CLAUDE.md for you with the
/initcommand, but read over it by hand to make sure it's right. - Make sure thinking is enabled (hit "tab"). If you're asking it to do a really hard thing, use the "ultrathink" keyword, which will give it more space to think.
- If you have reference tests to use as a model, point it at those. Telling it something like "read existing tests and use the same general style" can help a lot; if you can be more specific, do so.
- Ask it to run the tests and fix problems.
- Expect the occasional mistake. Review its code.
ZorbaTHut
27d ago
I'm paraphrasing a bit; I ended up with a bunch of prompts, including "write tests to cover every endpoint" and "write tests to increase code coverage" and "stop skipping this specific file, I've seen you consider it six times now and you keep saying it'll be hard and going to do something else, knock it off, go do it for real" and at one point it started just writing tests that verified an endpoint returned a success code instead of actually checking the data and I had to tell it to go back and fix them.
But nevertheless, it was fundamentally "go write a bunch of tests kthx".
What kinds of tests does it write?
Pretty simple ones. In general it's either "call endpoints and verify that the right thing happened in the database", or "change the database and verify that the right thing happened from the endpoint".
How does it know what to expect or assert in the tests?
By reading the code. It's not a black box, it just goes and reads the code. Then it tries stuff.
Then if the stuff didn't work, it reads the error messages and fixes it.
Same way a programmer does it, except it does it while I'm in the kitchen making a snack.
ZorbaTHut
28d ago
Some of this might actually be fixed, or at least, in a position where it can be fixed. We had a problem for a while where updating the software was impossible because it caused crashes and largescale fixes were sketchy due to how badly tested it was. Thankfully, with the rise of AI, I can (and have) just point Claude at the codebase and say "write a ton of tests kthx" and now it's a lot less questionable to do work on.
It actually found a few bugs while writing the tests which was pretty funny.
ZorbaTHut
28d ago
Honestly, I'll take some credit, but the mods have been putting a lot more work into it lately than I have. Give them most of that credit :)
ZorbaTHut
29d ago
Nobody mentioned it to me until maybe twelve hours ago :V I think it's now fixed, I think the computer it was on was having problems and the solution ended up being "bump it to another computer" . . . after doing a few in-retrospect-unnecessary optimizations. But hey, at least the optimizations are done.
ZorbaTHut
29d ago
The thing that's weird about this, though, is that it ramped up gradually over a period of a week. And it's hard to believe that us specifically would get a massive increase in traffic over simultaneously such a long and a short period of time.
That said, I may have solved it; I think we were just on a crummy cloud computer, I recycled the node and performance is fine again. Gotta remember that for next time.
ZorbaTHut
29d ago
To be honest, I'm not sure. If it's a DDOS it's a weirdly terrible one that's spending a lot of effort looking like a misbehaving web crawler. On the other hand, if it's a misbehaving web crawler, it's a weirdly terrible one that's spending a lot of effort looking somewhat like a DDOS.
Either that or we've gotten incredibly popular in China over the last week.
It's not just you. Working on it. :)
(Should be better now, I'll let it sit for half an hour or so and see how it's going. More work may be needed.)
(Edit: Sigh, not solved. Still working on it.)
(This may now be fixed; I think we were just running on bad hardware.)
Yeah, this actually might be somewhat reverted now, I'm still messing with stuff.
More options
Context Copy link