kepanoNov 3

When you email issues to Obsidian Entertainment (the video game company) their AI support hallucinates and tells you to email Obsidian (the note-taking company) instead.

The perils of trusting an LLM with your customer support.

Show threadAires

@kepano RIP your mail server 🪦

Seriously though, you'd think they'd check things like this before pushing them to production.

Nov 3 at 10:23pmWeb
Show threadWenzel MassagNov 4
@aires @kepano no. They’re willing to have an AI reply to emails, they clearly don’t care to check on things…
Show threadRyan J. YoderNov 6
@aires @kepano
Seems like this would be really hard to catch in QA, since there'd be no way to reproduce it reliably.
Show threadHelix Nov 6

@ryanjyoder
Which is why you probably should not entrust AI with something like this; but:

*it's actually easy to catch in QA.*

E.g.: write a filter with an allow list of mail addresses support is authorised to hand out.

It just shows that they don't understand their own systems. Probably done by a contractor after all, so you can blame them once it backfires 😅
@aires @kepano

Show threadRyan J. YoderNov 6
@helix @aires @kepano
A filter of email addresses might catch this specific edge case, but are you confident you'd be able to think of all similar class of bugs and write tests to catch them?
Show threadHelix Nov 9
@ryanjyoder
No, but then again, I'm not writing a support chat bot for them here. I wouldn't allow free form responses, but rather predefined text blocks.
@aires @kepano