Out of curiosity, I wanted to see if ChatGPT noticed the bug/mistake I mentioned here:

https://github.com/obsproject/obs-studio/pull/13198#issuecomment-4019545931

I pasted the relevant function in and... it nitpicked all sorts of irrelevant stuff about error handling (allocation failures etc), types, and API contracts, and completely missed the actual issue. It even mentioned "the use of CLOEXEC is reasonable" and the fact it conflicts with some logic branches was completely missed.

This is a function using standard POSIX APIs. This isn't even obscure code, it's doing something (spawning a process with a pipe) that has been done a million times before.

It just didn't catch this bug because this bug / a similar shaped bug wasn't in its training data. Because AI isn't intelligent, it's still just fancy autocomplete and can't think.

("But you didn't catch it either" - I wasn't reviewing the whole function when I wrote that PR, I was fixing a different bug and had some tunnel vision since I was just normalizing/extending/fixing a pattern that was already there, so I didn't notice the use of CLOEXEC earlier. But I did ask ChatGPT to review the whole function, not just the PR change.)

Fix spurious fd closing (os_process and v4l2) by hoshinolina · Pull Request #13198 · obsproject/obs-studio

Description Fix two bugs causing the wrong file descriptors to be closed. The os_process one is harmless but was discovered while debugging this. The v4l2loopback one is actively causing crashes. M...

GitHub

Honestly it feels like AI is... something like optimizing for nitpicking "best practices" while completely missing the real issues.

Nobody writes code according to all "best practices". Error handling in the example above is an anti-pattern. This isn't a kernel. Littering your code with NULL checks for memory allocation (and functions which can only fail due to memory allocation failures behind the scenes, like fdopen) is a net negative because it achieves nothing (if small mallocs are failing you code is going to crash anyway) while making the code more obtuse and complex and hard to read, which means real bugs will slip through more often.

If you're the kind of person that cares about having more "strictly correct" code... you shouldn't be using C to begin with, but rather a language where doing that doesn't hurt readability (like Rust). But AI isn't going to give you that advice...

@lina

If you're the kind of person that cares about having more "strictly correct" code... you shouldn't be using C to begin with, but rather a language where doing that doesn't hurt readability (like Rust). But AI isn't going to give you that advice...

LLVM is ensloppifying and Rust relies on it so that's not great. (GCCRust is not ready.)

One could say Ada but Ada is not properly bootstrappable yet.

So the main thing left is formally verified C or formal checkers/proofers/etc with C export like Low*.

The KaRaMeL user manual and documentation — The KaRaMeL user manual documentation

@lispi314 @lina gcc-rust would've been ready eons ago if Rust was a properly designed language with a proper spec, but instead they have to reverse engineer the original implementation to figure out how to implement stuff