@noybeu the concept of "correctness" is antithetical to how LLMs function, so what the person is asking for is fundamentally impossible. LLMs do not associate a specific value of a response to a particular question; they have a pool of "values that are responses to requests in the form of X" and pick one from that pool. It might be the right one, it might not.

However, this does not mean the person is wrong for asking; it means that OpenAI and peers are falsely advertising what LLMs do.

@noybeu

Writing a Cloud / AI strategy atm. Thank you for supporting me with arguments.

@noybeu This is an interesting one!

When you use your phone's next word prediction and use it to write about someone, do you expect the text to reflect reality?

And can you then go to Google and tell them the text from next word prediction needs to be changed according to GDPR?

How is ChatGPT different?

Their marketing doesn't change that fundamentally it's just doing next word prediction.

@noybeu interesting. I suspect that the defence would be around them not holding records that "relate to an individual" and this is evidenced by the fact they can't correct it. You can't look an individual's "record" up in the backend.

They'll argue it's no worse than having a news article in which someone's date of birth is inaccurate.

@noybeu

“Lying” about people and making money from it. They should be sued out of existence.

@noybeu It's a language model. It's not a source of data.

It's like trying to sue the English language because it's possible to use it to say something defamatory.

ChatGPT's ONLY job is to simulate the *structure* of, e.g., a conversation. Managing the *content* of that conversation is beyond its scope.

Substitute "conversation" for letter, essay, lecture, etc.

@noybeu

ChatGPT says my website Fedi.tips is some kind of cryptocurrency website. I have no way of correcting it, and it is still spreading these lies.

No idea where this comes from, no human would make this mistake. The only mention fedi.tips actually makes of blockchain and cryptocurrency is to tell people not to use it as it's a big scam:

https://fedi.tips/does-mastodon-or-the-fediverse-use-ads-or-trackers-or-algorithms-or-blockchain-or-cryptocurrency-or-anything-annoying-like-that/

ChatGPT is garbage, no one should be relying on it for accurate information.

@noybeu reading the article I couldn't help but wonder if some folk deliberately misunderstand what AI is. In particular the I part. What part of an encyclopaedia is intelligent? It's a basic store of information. It can be read, edited corrected where wing and so on.

But a AI is thus named as it is loosely modelled on the imagined workings of a brain. A neutral network ... It's also a word used in the arena for similar reasons.

And the goal is for a similar outcome. That is, learning, which captures information in a form that permits recollection only by reconstruction (much like the brain) and in responding to a prompt or request, to assemble a truly judged most likely to please, an idea governer by a button of familiarity and being the kind of reply we're confident we've heard to similar questions etc...

But you get my drift... Open AIb is not being evasive or inconsiderate. They are working on AI, they can't easily predict is responses nor modify them etc, any more than you can your colleagues brain. The very thing that is evening AI is the ability to handle an abundance of abstract data in real time ... Stored as weights abstracted from learning inputs.

It is no more likely to spit out actuate facts than, wait for it, the brains it is being modelled on. The winner point of the I in AI is being able to reassemble learnings in novel ways and to respond to prompts with diverse goals etc.

In a nutshell, who is surprised that it can and does say untrue things... That would seem to be an inherent property of the endeavour.

@noybeu Da darf man sehr gespannt sein, wie erfolgreich @noybeu hier sein wird.

Das ist übrigens auch eine spannende Frage für manches Kultusministerium, das ChatGPT & Co in die Schule bringt und dort „halluzinierte“ bzw. schlicht falsche und von Betroffenen nicht korrigierbare Personendaten verarbeitet.

P.S. In (digitalen) Fotos pixeln wir (auch automatisiert) personenbeziehbare Merkmale von Menschen. Sollte man sowas vielleicht auch den KIs beibringen - bevor man sie von der Leine lässt?

@noybeu This is exactly why I maintain a list of utterly nonsensical fictional bios because if you are going to run a bot that uses my content, your output is going to be quirky, to say the least.

I think it is quite funny to read but I would do, I wrote it: https://matrixdreams.com/un-realistic-biographies/

@noybeu Well, of course. This is pertinent to machine learning based models - nothing to do with OpenAI or ChatGPT in particular. All LLM-based chatbots have this problem.

Once you train a ML model and it starts generating output, you generally cannot (a) tell why it has generated this particular output, (b) which particular part of the training data resulted in this output, and (c) change substantially its behavior without full examination of the training data and re-training of the model.

@noybeu

"Can't"? Clearly it comes from the training data, and if they don't have logs of where they scraped the training data from, then that's clearly a decision to avoid liability.