@davidrevoy If you’re interested in a detailed investigation of just these dangers (in AI translation from last year):
https://www.draketo.de/software/ai-translation-evaluated#completely-changed
Subtle changes (⇒ hard to spot) that completely changed the meaning of parts of the text; in some cases in critical ways.
⇒ you’re spot on. Except it could have been worse: "… but they offer 5 gold coins for your family".
Unless of course the AI complete fucks up.
Challenge: "Please translate the first article of the German constitution into Klingon."
The first part of said constitution (found correctly) is:
"Die Würde des Menschen ist unantastbar." (Human dignity is inviolable)
ChatGPT translates that explainingly into
"Human'oy' yItlhutlh"
=
"Drink human pain!" (imperative/command)
The second half of the "translation" is even more garbage, making no sense or form at all.
@vampirdaddy But that translation *is* surprisingly good.
Not in conveying what the words mean to humans, but in choosing similar concepts.
Dignity is an essential concept for humans.
Pain is an essential concept for Klingons.
The AI chose related concepts to translate the sentence. But it did not express what humans meant.
My takeaway from my experiment is that you can’t convey thoughts with an AI translation that are very uncommon in the target language.
Yours matches that.
@davidrevoy
@ArneBab @davidrevoy
I‘d like to object.
Pain is not an essential concept/goal, but an obstacle to overcome.
The closest in Klingon ethics to „dignity“ probably is „honor“.
„drink!“ is very much not „is inviolable“, not even concept-wise.
But there even was a literal translation:
„not Human nur wem“
(never human‘s dignity is-violated)
@vampirdaddy then my background on klingons is far too rusty.
⇒ I stand corrected. And AI fails badly at klingon.
@vampirdaddy And I guess that this is a point where it’s far too easy to interpret reason into LLM outputs.
Humans are good at seeing patterns in white noise …
@gunchleoc yes -- and for AI it is known that this breaks the models (⇒ model collapse).
Also see https://rollenspiel.social/@ArneBab/116016558057569807
For machines to actually provide something useful by mimicking humans, they need purely human input.
More general: machines need data that’s free from their own outputs to avoid accumulating their own priors.
@vampirdaddy @davidrevoy
When I did inverse modelling in my PhD we always made sure that we know what’s prior model info and what’s actual data. You had to know which evaluation includes prior data in addition to measurement data, because assimilating multiple sources with the same prior could cause the prior to become dominant. Todays LLMs repeatedly ingest their own outputs -- outputs that contain their own priors. It’s obvious that preserving the model quality requires pure human input. https://en.wikipedia.org/wiki/Model_collapse#Disagreement_over_real-world_impact
David Revoy
ArneBab
vampirdaddy
lp0 on fire 
GunChleoc