Mastodawn

ntfy.sh v2.18.0 was written by AI

New ntfy.sh v2.18.0 was written by AI - Lemmy.World

According to the release: >Adds experimental PostgreSQL support >The code was written by Cursor and Claude >14,997 added lines of code, and 10,202 lines removed >reviewed and heavily tested over 2-3 weeks This makes me a bit uneasy, especially as ntfy is an internet facing service. Am I overreacting or do you all share the same concern?

Show thread

henfredemars Mar 8

Definitely share your initial concern. Without strong review processes to ensure that every line of code follows the intent of the human developer, there’s no way of knowing what exactly is in there and the implications for the human users. And I’m not just talking about bugs.

They say it’s reviewed, but the temptation to blindly trust is there. In this case, developer appears to have taken some care.

The code was written by Cursor and Claude, but reviewed and heavily tested over 2-3 weeks by me. I created comparison documents, went through all queries multiple times and reviewed the logic over and over again. I also did load tests and manual regression tests, which took lots of evenings.

Let us hope so. Handle with care to ensure responsibility is not offloaded to a machine instead of a person.

Show thread

Jul (they/she)

Yeah, it could easily have added a couple of lines of code that sends everything to Northern Korean hackers because it found that in a bunch of repositories or just logging passwords to public logs or other things an experienced developer would never do. “AI” only replicates what it sees most often and as more spam and junk repos are added to its training data because “AI” companies are too concerned with profit to teach it properly, it could do tons of random stuff. It’s like training a developer by giving them random examples from the internet rather than specific ones. Of course they pick up bad habits. Even if it “works” it is almost never efficient or secure.