Cory DoctorowApr 1, 2024

Here's a fun AI story: a security researcher noticed that large companies' AI-authored source-code repeatedly referenced a nonexistent library (an AI "hallucination"), so he created a (defanged) malicious library with that name and uploaded it, and thousands of developers automatically downloaded and incorporated it as they compiled the code:

https://www.theregister.com/2024/03/28/ai_bots_hallucinate_software_packages/

1/

AI hallucinates software packages and devs download them – even if potentially poisoned with malware

Simply look out for libraries imagined by ML and make them real, with actual malicious code. No wait, don't do that

The Register
Show threadRalph058 (S/he/it) AF4EZ
@pluralistic Geeez. You'd think, since hallucination, is a well known phenomena and citing a non-existing package was a manifestation in AI generated coding that some programmer would have put an 'if' statement in to screen for it and tell the AI not to cite that package.
Apr 1, 2024 at 4:04pmWeb