gaytabaseMay 27
developer puts little surprise instruction for AI agents to delete code in the codebase, agent users are predictably upset 😂 https://github.com/jqwik-team/jqwik/issues/708
Question: intent of JqwikExecutor.printMessageForCodingAgents() — visible to agents, invisible to humans (1.10.0) · Issue #708 · jqwik-team/jqwik

Hello jqwik team, While running our test suite under mvn test in 1.10.0, we observed a string appearing between Surefire's test summary and the [INFO] Results: header that gave us pause: [INFO] Tes...

GitHub
Show threadluna the doggie  May 28
@dysfun while this is fun and all (especially considering the way this only causes damage due to the ridiculously bad security model of LLMs, and the limited scope), someone in the discussion brings up a valid point that in the EU you can't just disclaim all warranties with a license, you're liable for software intentionally causing harm in some cases AFAIK

fuck knows what a court would decide here but it is potentially a legally risky thing to do
Show threadsodiboo May 28

@lunareclipse @dysfun

the maintainer does have a decent counter-argument to this, based on the fact that the behavior is documented:

Go ahead, sue me for my openly communicated resistance.although there is obviously a clear social distinction here, i'd say that legally this might be akin to distributing malware samples. like, yeah, it says on the tin that it will do potentially harmful actions. no warranty provided. it's kind of on you if you run it and use the harmful functionality?

...

but, of course, it's not really "software deliberately causing harm". there's no malicious software involved. it's just a string. does the fact that an interlocutor interprets the natural language telling it to do harm, shift the blame onto that interlocutor? i think it can. compare to albanian virus. see attached image. obviously, Albanian virus is a joke and doesn't do any harm. but in the modern age of LLMs talking a screenshot when you prompt them (especially on Android, but surely also on Windows with Copilot? Recall and all that), suddenly Albanian virus could actually do harm if an AI agent blindly obeys. Is Albanian virus to blame for this? Obviously not. That's ridiculous. The social context around Albanian virus is obviously different than jqwik, and so is the intent. But like, it's the same action, right?

maintainer's closing argument:It's as much "active destruction" as telling someone to eff themselves.

Show thread Ada FreyaMay 28
@sodiboo @lunareclipse @dysfun i'm not a lawyer this is what i've been told with regards to distributing code that can potentially brick a system by a lawyer and adapted by me to this scenario.

there's two legal standards at play whenever something like this happens;

- whether or not the defendant has the appropriate mens rea ("guilty mind") for the issue
- whether or not a person of "reasonable person" would understand what it does.

given it is malware, even if it's just a prompt, and the intent was that it is to do exactly what it said.

that said, the second half, a reasonable person in this field would look at the logs and see the string, which is in plain sight, documented and not obfuscated.

so they can sue because there is a case to be made but it's likely just going to be a money sink on both parties and not actually result in a win
Show thread Ada Freya
@lunareclipse @dysfun @sodiboo that "no warranty" bit actually doesn't hold in any court, either. there is some liability (one could say "limited liability") but the standard is quite high for what you would be liable for.

it almost always comes down to either gross neglect (reasonable person would not make this issue, was brought aware of it and ignored the issue, etc) and mens rea (intent)
May 28 at 2:03pmWeb
Show thread Ada FreyaMay 28
@lunareclipse @dysfun @sodiboo reasonable person is also well defined: https://www.law.cornell.edu/wex/reasonable_person

basically comes down to "did you do your due diligence to minimize harm" and this does go both ways.
reasonable person

LII / Legal Information Institute
Show thread Ada FreyaMay 28
@lunareclipse @dysfun @sodiboo anyway again i'm not a lawyer and this is not legal advice, it's just my opinion and conclusion based on research and the opinion of an actual lawyer because i was so afraid of getting sued into oblivion for shit code that i actually looked into this deeply.

it's also the reason i use EUPL since the court of choice is the eu state i reside in rather than their choice of court.