kepano

When you email issues to Obsidian Entertainment (the video game company) their AI support hallucinates and tells you to email Obsidian (the note-taking company) instead.

The perils of trusting an LLM with your customer support.

Nov 3 at 10:08pmElk
Show threadAiresNov 3

@kepano RIP your mail server 🪦

Seriously though, you'd think they'd check things like this before pushing them to production.

Show threadWenzel (stairjoke)Nov 4
@aires @kepano no. They’re willing to have an AI reply to emails, they clearly don’t care to check on things…
Show threadRyan J. YoderNov 6
@aires @kepano
Seems like this would be really hard to catch in QA, since there'd be no way to reproduce it reliably.
Show threadHelix Nov 6

@ryanjyoder
Which is why you probably should not entrust AI with something like this; but:

*it's actually easy to catch in QA.*

E.g.: write a filter with an allow list of mail addresses support is authorised to hand out.

It just shows that they don't understand their own systems. Probably done by a contractor after all, so you can blame them once it backfires 😅
@aires @kepano

Show threadRyan J. YoderNov 6
@helix @aires @kepano
A filter of email addresses might catch this specific edge case, but are you confident you'd be able to think of all similar class of bugs and write tests to catch them?
Show threadHelix Nov 9
@ryanjyoder
No, but then again, I'm not writing a support chat bot for them here. I wouldn't allow free form responses, but rather predefined text blocks.
@aires @kepano
Show threadMichael GaleNov 3
@kepano i think you go ahead and send the email. make the two obsidians duel it out.
Show threadMatthias KrämerNov 4
@miclgael If both have an AI agent writing E-Mails, that could go really fast with really many answer-E-Mails until the servers explode. @kepano
Show threadMandar Vaze (desipenguin)Nov 5
@Kraemer_HB If both are AI agents, they won't email each other, they'd switch to Gibberlink ? 😉
Show threadVladimirNov 3
@kepano nobody gets fired for integrating AI
Show thread ShalienNov 4
@kepano since they got acquired by Microsoft this was to be expected.
Show threaddean36963Nov 4

@kepano "You're absolutely correct! I've muddled two similarly named companies: I promise I will never muddle two similarly named things again!

As we can't email support to solve your account issue we recommend you play 'Outer Wilds', the prequel to 'Outer Worlds 2', also made by us, but doesn't require an account to play!"

Show threadRRBNov 4
@kepano feature, not a bug. AI meant to be a roach motel where users are put in a maze of twisty passages all alike to die and leave the company in peace. All companies now hate their users. Use FOSS.
Show threadHelix Nov 6
@rrb
I do not want to exclusively play FOSS games currently.
The current games, music, movie and show distribution systems are not allowing FOSS to thrive. This means that most of those media are closed source.
IMHO, not everything needs to be FOSS. Art doesn't need to be, as modifications can also be done with remixes and modding tools.
I think you meant that we should only play games from responsible and/or respectable indie developers? This is something I could get behind.
@kepano
Show threadRRBNov 6
@helix @kepano just tired of dark patterns and being tortured by bad user interfaces. The companies put a ton of effort in making interfaces to screw their users and inflict clanker slop on the rest of it
Show threadHelix Nov 9
@rrb
You can have better games without them being Open Source, is my point.
I feel the same, AAA gaming is annoying.
@kepano
Show threadIvan G.Nov 4
@kepano I think that the main issue nowadays is that if you reach customers support, most likely you are already paying for it, so the already monetized on you. So...
They don' t care about you. This is modern day corporations in my opinions.
Is all about getting customers, and if they lose an existing customer, then... there are going to be more coming. They will started to care only when number of cancellation will be higher than new subscriptions...
Show threadwithoutclassNov 4
@kepano Microsoft owns them, terribly unsurprising
Show threadDevin Prater :blind:Nov 4
@kepano Mmm, sounds like Gemini.
Show threadWesDymNov 4

@kepano Obsidian satellite company enters the chat.

I'm guessing there some other Obsidians out there. This could get hilarious.

Show threadhallunke23 🇺🇦Nov 4

For instance, there is Obsidian Shell, a metal group from Hungary.
(According to AI, Hungary is also known as Holland.)

https://www.jamendo.com/artist/349991/obsidian-shell

@wesdym @kepano

Obsidian Shell | Jamendo Music | Free music downloads

Check out our official website www.obsidianshell.com We appreciate your contribution by Donating via Paypal

Jamendo Music
Show threadWesDymNov 4
@kepano Oh, man, this was RIPE for a "dense" joke, and I didn't think about it.
Show threadoldguycrustyNov 4

@kepano

I think you are missing the point.

Obsidian Entertainment doesnt WANT your email. Clearly.

So - I would posit that their AI is performing perfectly.

Show threadSupermoosieNov 4

@oldguycrusty @kepano

"And on this graph, we can see that we have gone from 10,00 issues handled by our human customer service, down to just 2 post AI. We can therefore conclude that both the quality of our software we shipped has improved and any issues that did arise were easily handled by our powerful AI agent. Hence customers are experiencing a leap in satisfaction.

Reality:
Customer: I have a issue.
AI Agent: Piss off

Show threadoldguycrustyNov 5

@SuperMoosie @kepano

Yep. You get it exactly.

Show threadTim HergertNov 4

@kepano Although - kinda on brand for the Outer World's universe.

♫ It's not the best choice - its AI choice! ♪

Show threadMaarten De BalNov 4
@kepano I was actually planning on getting the game but automating support in such way off putting. At least get someone to screen emails.
Show threadjiub   Nov 4
@kepano fucking clown shit how is this garbage technology used for anything

(rhetorical question)
Show threadDerek Caelin is writing a bookNov 4
@kepano oh dear.
Show threadSashinNov 4
@kepano it hurts you that someone else uses it and thinking about that hurts me
Show threadCegorachNov 4
@kepano that's a feature! It reduces effort for reading support mails.

/s
Show threadPongolynNov 4
@kepano There has to be a zettelkasten or "second brain" joke in here somewhere
Show threaderi :this-user-is-a-fox:Nov 4
@kepano forward this exact email to obsidian (note app) i wanna watch them sue eachother
Show threadAfif R.Nov 5
@kepano May I suggest a complete name change? Is Skibidi taken?
Show threadRandagodronNov 5
@kepano prompt : "get rid of the customer's requests"
🤖
💡
-> send them to another company with the same name, nobody will notice 👍🏻✅
Show threadD.C.U.K.Nov 6
@randagodron @kepano Ever since we introduced the LLM, open tickets are way down!
Show threadRandagodronNov 6
@dutch_connection_uk @kepano "success" I guess ? 🤷🏻‍♂️
'effing KPIs ...
Show threadhusjonNov 5
@kepano Next shareholder meeting: "After introducing AI to our support force we reduced support tickets by 90% .... by sending them to another company" 🤦‍♂️
Show threadMatt Hardy 3.11 for WorkgroupsNov 5
@kepano Only yesterday, I sent an email to the wrong supportdesk@ address. I don't need AI's help on this. I'm already Abysmally Incompetent.
Show threadsalguod, man 🍂🍁 lazysupperNov 6

@kepano
You should email Obsidian.md and see what they think. 🙃

(Great app BTW)

Show threadcultdevNov 6
@kepano dang that’s embarrassing. what dipshits
Show threadSanskritFritzNov 6
@kepano
Not so serious but annoying: I'm subscibed to the #obsidian tag and thus also receive posts dealing with Obsidian Entertainment. No way to separate the two.
Show threadThomas MathoiNov 6
@SanskritFritz use #ObsidianMD instead or additional // CC @kepano
Show threadSanskritFritzNov 6
@thomas_mathoi @kepano
I use #obsidianmd but many posts only contain the #obsidian tag. Pity.
Show threadThe Peter Pan of Nerdery™ 🇦🇺Nov 6
@SanskritFritz Similarly the woman who makes jewelery out of obsidian rock..
Show threadSanskritFritzNov 6
@dhry
I haven't encountered that yet but yeah 🤭
Show threadScott KingeryNov 7
@SanskritFritz @dhry we could be use obsidianmd instead but that ship has sailed
Show threadThe Peter Pan of Nerdery™ 🇦🇺Nov 7
@techlifeweb @SanskritFritz Also, Obsidian isn’t connected with Maryland in any meaningful way I’m aware of..
Show threadScott KingeryNov 7
@dhry @SanskritFritz the file format we use is markdown plus the URL https://obsidian.md/
Obsidian - Sharpen your thinking

The free and flexible app for your private thoughts.

Obsidian
Show threadThe Peter Pan of Nerdery™ 🇦🇺Nov 7
@techlifeweb @SanskritFritz Nor does Obsidian hold a medical doctorate as far as I’m aware.. (touches button on top of chess clock lightly with pinky finger)
Show threadJen Kuntz 🇨🇦Nov 7
@SanskritFritz @kepano I have so many gaming hashtags blocked that typically appear with #Obsidian …
Show threadSanskritFritzNov 7
@jenk @kepano
That is a good idea indeed. Blocking associated tags could be a way to filter, thanks!
Show threadKristoffer GarsholNov 8
@kepano extremely bad QA from Obsidian Games. That mail could have easily been a boilerplate template in stead of an LLM. Wow.