Got burned by LLM-based "AI"'s tendency to make shit up, yet again, but this time at work, even though I don't use them myself.

I closed a bug ticket for a problem that involved a regularly recurring set of alerts in a log service. There was, indeed, a problem, and it came from an error handling process that resubmitted failed records for reprocessing once every 24 hours. In this case, some of the records were just plain invalid, so they were causing a bunch of errors at about the same time each day for about a week.

The ticket also smelled like it had been written, at least in part, by ChatGPT or a similar word barfing technology.

The ticket stated that "Users are affected by the inability to properly utilize the [service] forms," when there was no actual end user report, and the culprit was actually the internal error handler described above. The ticket also included "Steps to Reproduce" that were not actually followed, since the steps stated to submit a form at a certain time and watch for errors. But the time given was for when the errors were "binned" in a report, which could be up to one hour different from when they actually occurred, and there were no errors logged at the time the "Steps to Reproduce" were supposedly carried out.

As a result of this incorrect information, I spent several hours minimum trying to research possible causes for this bug that didn't need to be seriously considered, and didn't find out the actual cause until someone else (who knew about the bad-record resubmission process) helpfully pointed it out.

It's probably preaching to the choir here on Fediverse, but this is yet another reason to avoid this stuff as much as possible. "Vibe coding" (or, in this case, probable "vibe error reporting") ends up costing a lot more time than you think -- and it's not very nice when the time in question isn't your own.