Kevlin HenneyApr 3, 2023

Code generated from LLMs is going to need more testing than code written by developers. This seems self-evident to me, but I suspect a lot of people are going to learn it (or ignore it) the hard way.

Given that most existing codebases are not well tested, and most developers don't test, this does not bode well.

Show threadKevlin HenneyApr 3, 2023
The practical consequence of using LLMs to generate code is that many developers will find they have unwittingly moved themselves into a role they were probably trying to avoid: they have automated the creation of legacy code and have redefined their job role as debugging and fixing such code.
Show threadMichael Critz

@kevlin @mogul I asked ChatGPT 4 to write some caching code. I saw some problems. Then, I had to follow up with, “identify problem areas in this”.

To give it credit, *when prompted* it knew what could go wrong. And, again, when prompted, could write mitigation strategies to avoid issues.

Overall, I give ChatGPT 4 “junior developer” status.

Apr 4, 2023 at 1:04amIvory for iOS
Show threadPeteApr 4, 2023

@pixelscience @kevlin @mogul

It didn't know what could go wrong. It gave the illusion of knowing. Maybe there's no difference.