Be advised: this thread is not for serious in-depth discussion of weighty topics (we have a link for that), this thread is not for anything Culture War related. This thread is for Fun. You got jokes? Share 'em. You got silly questions? Ask 'em.
- 123
- 1
Jump in the discussion.
No email address required.
Notes -
Well, it's happened. The pressures of our workload have driven a push to use AI to help with some of the bitchwork of coding. There is going to be a push to use AI to generate unit tests.
The toolchain involves OpenCode, so I figured I'd install it locally to get familiar with it before I start burning GPU time at work. Also, for reasons, we aren't allowed to use the GPUs. So last night I installed Ollama, OpenCode and the gemma4:e4b model on my humble RTX 4070 Super with 12 GB of VRAM. I tried to have it do the simplest of tasks. Create a Hello World project in dotnet10, and write a single unit test to verify it's output.
The first thing that happened was it created a new project. New projects actually begin with hello world output already. It then added a second hello world output. This poisoned the well, as now the AI was horribly confused about why there were two hello world lines. It never fully recovered. The project was generated without it's int main format which I prefer, so I tried to have it restructure the project to use that. After several missteps because it couldn't get over the fact that there were duplicate hello world statements, it finally figured it out.
Next came the unit tests. It created a unit test project, but then didn't actually populate the tests or link the projects. Then it wanted to refactor hello world, and pull in all sorts of abstraction frameworks, so it could test the output without redirecting stdout. I told it forget all that and redirect stdout. It had already done half the refactoring in a state that could not compile, and then never undid it, and then the whole project was totally broken and it couldn't figure out how to fix it.
I remind you, this was a "Hello world" and a single unit test. I told the AI it fucked everything up, it asked what it could do to fix it, and I told it that it could shutdown. It did. I think.
I know a lot of people reading this are AI evangelist. Where did I go wrong? What the fuck do people see in this shit?
In my experience, you really need to hold the AI's hand to get it to do anything remotely complicated. It's awful at designing things, and will constantly make incorrect assumptions. Last week I had it tell me that some map was wrong because it was keyed off of type information, and so if two types have the same layout there will be a collision. Not only is that not true in C++, I can't think of any language that does that.
That being said, it can be useful when doing things that can be automatically checked. I've had decent success getting it to write mock implementations for tests after denying it the ability to change the actual source code, or to change which libraries the test links. It'll still try to do both of those things, but will eventually give up and write the actual mock. It's also okay at writing the test case outlines, albeit with as few assertions as possible. It's probably still faster for me to add them afterwards than write the whole test by hand.
More options
Context Copy link
More options
Context Copy link