Natasha 
🇪🇺I don't want to laugh at someone's real distress but this IS very funny ...
Frank JustonNo railguards in this chatbot 🤷♂️
mxk@Frank_Juston safeguarding AI is a myth. Fundamentally you can't prevent attacks like that against LLMs.
nikthechampiongr@mxk @Frank_Juston I mean if your goal here is to avoid having legal liability, you could just put up a giant banner on the chat window saying something along the lines "ANY DISCOUNT OR PRICE GIVEN TO YOU THROUGH THIS CHAT IS NOT VALID! PLEASE CONSULT THE STORE FOR A FINAL PRICE". Then attacks like this one don't matter.
The Penguin of Evil@nikthechampiongr @mxk @Frank_Juston I don't think it's a problem anyway. If someone sets out to scam you then they won't win in court. It's similar to cases where you buy an item that is mispriced. One of the tests applied is if you obviously knew it was mispriced.
So if you spent an hour fencing with a chatbot to trick it into a discount I don't think you'll win.
Raymond Russell@etchedpixels @nikthechampiongr @mxk @Frank_Juston
I would imagine that you cannot engage in a legal contract with an AI bot. So that adds to the other points that you made about not being winnable in court.
I would imagine that you cannot engage in a legal contract with an AI bot. So that adds to the other points that you made about not being winnable in court.
Carnildo@nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston That's a different situation. In the Air Canada situation, the user made a good-faith query of the LLM, and got a wrong answer, much like they'd have to honor a human-produced error on the website. In this case, the user spent considerable effort to make an LLM produce a wrong answer, knowing in advance that it was wrong.
Jeff Grigg@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
In the United States, courts have ruled that when you automate business process with computers, you are authorizing the computers to act as your agents. And, as such, they can enter legally binding contracts that you must honor.
…
@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
I read about this years ago, with DEC PDP computers, costing hundreds of thousands of dollars. Their order form computed discounts and the total in the browser, in JavaScript.
…
@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
A student "tricked" the system into accepting an order for a computer system for $1 total, by changing the data in the HTTP submission form sent to the server. The web site did not verify or reject the bad total. The system was delivered, and the student paid the $1.
Then DEC caught the "error" and demanded full expected payment or return of the system. They took it to court, and lost.
…
@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
The court ruled that the student made an offer, and the company's agent had accepted it. It's a binding contract.
…
@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
Now in this case, the chatbot apparently made the offer and entered the order with the invalid discount code, which was rejected by the server.
The customer entered the discount code, claiming the 80% discount when they paid the deposit.
Would the text of the chat and accepting the deposit count as an "negotiated agreement" in court?
Maybe.
…
@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
I don't think that the business owner's assertion that "[the] chatbot isn't supposed to be making financial decisions." would count for anything in court. They did authorize it to "log orders."
…
@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
They did support and authorize it to "chat" with customers. This might reasonably be interpreted as "negotiation."
Their chatbot did "take/accept" the order. They did accept the deposit.
Is that not acceptance of a business deal?
…
@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
Honestly, I'd expect a reasonable court to "toss" this out.
But, at least in the United States, the customer could sue. (One can always sue. There's nothing to stop it.)
…
@carnildo @nikthechampiongr @raymierussell @etchedpixels @mxk @Frank_Juston
And with the industry pushing use of LLMs as "fully autonomous agents," I'm sure that really serious problems, with no legal defence, are inevitable.
And that this is most likely to have catastrophically devastating effects on customers more that businesses, as services offer customers LLM autonomous agents to do tedious drudge work for them.
✕
@JeffGrigg @carnildo @nikthechampiongr @raymierussell @mxk @Frank_Juston in the UK at least there is a distinction between maliciously tricking someone's website and taking up an offer you reasonably believed was real. One is potentially fraud the other is generally tough shit for the seller depending upon other things the site clearly says.
I suspect similar is true in most places where malice is involved.
I suspect similar is true in most places where malice is involved.