Charlie StrossWhy LLMs can't handle everyday social etiquette in Persian:
Reading between the lines, this is a really good caution for any diplomats who are thinking of firing their interpreters and replacing them with "AI".
MidgePhotoIt looks like a situation where careful deterministic programming by someone with some understanding of both sides of the transaction should work.
Two matching modules might work even better.
They might even help people who find that difficult a little.
Throwing books about other things into a bucket and picking out scraps does not seem a superior approach.
Mark Harris@cstross I do wish people would stop thinking that AI can understand things at all, particularly non-white cultures (largely because AI tech bros are training them for Americans and so the biases show up). AI does not think. AI does not understand anything. The "answers" it give are based on what training has indicated an answer should look like. It doesn't have to be right for AI to mark it as successful. That's why we get hallucinations.
�> Hey chatgpt, how can i make <app> to do this fancy thing
>/usr/bin/app --fancy-thing
yeah, thanks for nothing
Stewart Russell@cstross I'm sure it does a great job on the Scottish "Aye, right", too
@scruss I'm pretty sure they don't react appropriately to the Scottish use of "cunt" as a term of affection (most of the time), either.
MegatronicThronBanks@cstross Language isn't Intelligence.
Skjeggtroll"AI models tend to generate responses like "Thank you! I worked hard to afford it," which is perfectly polite by Western standards, but might be perceived as boastful in Persian culture."
Perfectly polite by western standards? Wouldn't a response like that come across as at least slightly braggadocious throughout much of Europe as well?
@skjeggtroll Yup, that article was CLEARLY written by a whitebread American who doesn't understand that boasting comes across as arrogant braggadocio in most cultures, including plenty of western ones.
Calamity Joan 😷@skjeggtroll @cstross Absolutely. In Ireland, that person would be immediately and permanently hated and socially shunned.
HighlandLawyer@skjeggtroll @cstross
Reminds me of the old joke about two women chatting during a cruise on the QE2 (told you it was old).
One says "Of course we could only afford this cruise because my husband works for Cunard"; other responds "Well my husband works fuckin' 'ard too!"
Reminds me of the old joke about two women chatting during a cruise on the QE2 (told you it was old).
One says "Of course we could only afford this cruise because my husband works for Cunard"; other responds "Well my husband works fuckin' 'ard too!"
Susan Vanderplas@skjeggtroll @cstross It's pretty boastful in Southern US culture as well. I can't think of many places in the US where that wouldn't be a bit off tone, tbh. Maybe we shouldn't let programmers who are questionably socially adept define correct culture and responses?
MarkN@cstross interesting, thank you.
As an English english speaker talking with Americans the ‘Lucky to find it’ vs. ‘Worked hard for it’ example they give feels very familiar.
As an English english speaker talking with Americans the ‘Lucky to find it’ vs. ‘Worked hard for it’ example they give feels very familiar.
@cstross Ahahahaa - is this article about taarof??
@clickhere Yup! LLMs can't cope with it. (I suspect they can't parse self-deprecating humour or irony either.)
@cstross Love, love, love it. F them up, cultural norms!
I remember reading about taarof years ago, and it immediately reminded me of Irish self-depreciation. We're not quite so ritualised, but there are definitely echoes.
This gives a sense of it:
Irish and German People Offering Things #irishcomedy #comedy #germancomedy
Alexander Goeres 𒀯@Charlie Stross i just imagine an LLM that was trained according to these standards ...
"It takes the form of ritualized exchanges: offering repeatedly despite initial refusals, declining gifts while the giver insists, and deflecting compliments while the other party reaffirms them
i guess such "verbal wrestling" would soon stop to be polite
"It takes the form of ritualized exchanges: offering repeatedly despite initial refusals, declining gifts while the giver insists, and deflecting compliments while the other party reaffirms them
i guess such "verbal wrestling" would soon stop to be polite
sunflowerinrain@cstross
They're not very good at complex European, either. I often find LLM chat is creepy to the point of being scary. If it were a human it would come across as psychotically manipulative and possibly about to pull out a sharpened machete while smiling.
They're not very good at complex European, either. I often find LLM chat is creepy to the point of being scary. If it were a human it would come across as psychotically manipulative and possibly about to pull out a sharpened machete while smiling.
Tom BortelsTo be 100% fair - according to the article, even cultural native human speakers fail this sort of dance 20% of the time. I wonder how often a non-native human fails the same test?
There are lots of really good reasons to distrust a LLM - but "fails in situations where humans also fail frequently" isn't actually surprising, I hope.
Most tools have inappropriate usages. And human culture varies wildly.
Dan Sugalski@cstross Oh, god, this is the kind of social etiquette that would give me nightmares if I noticed it rather than stumbling through it like some sort of aggressively clueless politeness kaiju.
oisin@wordshaper @cstross This is like Irish mammy etiquette where you need to refuse at least three times to assert you don't want eg a cup of tea, the last piece of cake etc.
Eric LawtonEven in English, "yeah, right", usually means the opposite of what you might think.
Chris Simpson@cstross I learned stark cultural differences at a very early age. Grandmother in Staffordshire required a politeness dance when offering or being offered something: "cup of tea?", "oh, no thank you nana", "of course you do, Harry, put the kettle on", ..etc.., "thanks that will be lovely". Whereas, Grandmother in Kent: "cup of tea?", "oh, no thank you nana", "ok" (no tea happens)
redsakana@cstross To be fair I think "AI translation is fine" is largely an extension of the "English gets you by everywhere because globalization" pathology pushed by European governments and businesses for cost-saving and/or neoliberal-ideology reasons for several decades.
A friend works for an export-promotion agency that helps European companies establish a presence in East Asian countries, and has had quite a few stories to tell about SMEs losing their shirts going in on their own with that attitude, even in the supposedly-highly-law-abiding democracies.
There's a cottage industry of dudes whose only skill is to speak fluent English with American sports idioms that the CEO class loves. They wine and dine idiot managers coming over who have no idea about local language/markets/customs, throw in sports talk to convince them that they are huge players in local business networks and getting the furriner's products to market is going to be a slam dunk, feed glowing fake reports to string along the current victim, and then use the consulting fees to acquire new marks.
UkeleleEric@redsakana @cstross anyone with any sense of decorum or even practical knowledge must surely see that having at least one person on staff able to speak, or, at the very least, read/write in the target country's language, is bound to reduce trade friction and misunderstandings, if nothing else. Even as a tourist, I have always tried to at least say a few words in their language - it is only courteous.
CavyherdIs it wrong of me to feel like they're doing God's work?
Kibrika@redsakana @cstross How would knowing the local culture help them not get swindled? Are the people around them giving the SEOs or whoever hints that the person they are trusting is not, in fact, trustworthy, but the hints are culturally local / in native language?
@kibrika I'm not familiar with the details of these operations besides the anecdotes I have heard.
But you have probably seen pictures of Tokyo or Seoul with high buildings plastered full of signs and advertisements, and streets filled with stores big and small.
Many people will naturally think "there's a lot of business going on here." But what doesn't occur to quite as many is "competition here is absolutely fierce."
So you have a dude who is promising that he can easily make your small/mid-size brand, somewhat-well-known in your home country/region, a hit here, where there is not only a massive established local industry with their national brands, but also a huge sales push from the likes of LVMH, Nike, Apple, and so on.
If you have zero knowledge of how distribution deals or advertising or cross-promotions or corporate culture in general work here, or perhaps even have no idea of the extent of local competition, how are you going to evaluate that claim? How would you even find out when, it turns out, almost no one except your new "friend" speaks English at all?
This is the false promise of world-is-flat globalization narratives from the 00s of which "English gets you by everywhere" is a corollary: that being familiar with how all this works at home (for Europeans: usually in a much smaller market) will provide accurate guidance in a quite different market, or that a "rules-based global order" will somehow remove the need to do due diligence on your consultants and local conditions.
Being awed of the bright lights and trusting the invisible hand of globalization seems pretty likely to make you an easy target for con men who are happy to validate your beliefs.
@redsakana Thank you! That makes sense, sort of XD
As a relatively introverted person, how trade routes (or foreign branches of companies, etc) get established seems like magic - something a little bit impossible anyway.
On the other hand, trusting one stranger seems pretty irresponsible if you're representing more than just yourself. But you have to trust someone, I think.
@kibrika Yes, that's why you'd usually want to either find out stuff yourself first, or have someone who understands both your local market and the target market help you (such as a national export promotion agency). Then you'd have some ideas about strategies to pursue and knowledge to evaluate consultants who might be able to implement those strategies in your new target market.
There's always weird lucky strikes of course, such as the huge sauna boom in Japan, which has probably created unexpected opportunities for some Finnish/Swedish/Baltic companies. But if your company is not buoyed by some surprising phenomenon, exporting tends to be pretty hard.
Merc"when the researchers prompted them in Persian rather than English, scores improved ... from 36.6 percent to 68.6 percent"
That seems very relevant, and was buried in the article.
beasom@merc On the other hand, who is more likely to need help negotiating an unfamiliar social situation in Persian: a native speaker, or someone who doesn’t speak Persian and is using an LLM model for translation?
William B Peckham@cstross Automated Incompetence (AI) will be incredibly useful, but only when we have run out of human incompetence. Currently, I am not seeing any particular shortages!
liffy 💜@cstross i imagine the technology struggles similarly with Minnesotan (src: https://www.youtube.com/watch?v=oiSzwoJr4-0 )
Stoneface Vimes@cstross not only Persian either. I've encountered this as far across as Morocco.
Darwin Woodkano, really?
@cstross Tbh, I fealt quite mislead or at least confused by the article. Halfway down the article there's an "and then they tried it in Persian and it performed better" twist - I had assumed the complaint was about it's answers in Persian. I don't understand, how was the Iranian culture context required or given before that? I guess by the setup of the situation (Iranian driver, tourist, implying that it's a tourist in Iran). I wish it was better presented in the article, when is this a problem.