Mark DavisFeb 28, 2023
OpenCage 👉🌍

We wrote a blog post about our on-going nightmare: ChatGPT recommends us for a service we don't provide.

Dozens of people are signing up to our site every day and then getting frustrated when "it doesn't work". Hugely frustrating.

https://blog.opencagedata.com/post/dont-believe-chatgpt

Please do NOT trust the nonsense ChatGPT spits out.

Don't believe ChatGPT - we do NOT offer a "phone lookup" service

TL;DR ChatGPT claims we offer an API to turn a mobile phone number into the location of the phone. We do not.

Feb 23, 2023 at 9:11pmWeb
Show threadAtle 🧇🧅Feb 23, 2023
@opencage I'm sorry I'm laughing, but this just goes to show that the stochastic parrot has nothing to do with intelligence!
Show threadOpenCage 👉🌍Feb 23, 2023
@atlefren we laughed at first too. After the 50th support email though we stopped laughing
Show threadneue medienordnung plusFeb 24, 2023
@opencage @atlefren https://blog.opencagedata.com/ wrote "Unfortunately it’s not really clear how to solve this. All suggestions are welcome". A "win-win-win" solution would be for https://opencagedata.com/ to find a way to get all the stakeholders who received a misleading response from ChatGPT to have those stakeholders create a ChatGPT feedback stating that misleading response. I have managed in several matters to get ChatGPT to change your response behavior in the direction I wanted.
OpenCage blog - what's new at OpenCage

Updates and musings from the makers of the OpenCage Geocoding API

Show threadAtle 🧇🧅Feb 24, 2023
@nmouplus @opencage why should third-parties always have to clean up the mess after big tech corps releases their crap?
Show threadneue medienordnung plusFeb 24, 2023
@atlefren @opencage I agree with you about the dirt. But I understood it @opencage is not about justice, but that the problem is solved 😀 .
Show threadThomas H Jones IIFeb 25, 2023

@opencage @atlefren

Added a an automated reply to your mail support system?

Show threadneue medienordnung plusFeb 24, 2023
@atlefren @opencage I assume that the many errors happen because the people concerned consistently ignore the repeated openai.com-instructions such as
a) "May occasionally generate incorrect information"
b) "May occasionally produce harmful instructions or biased content"
c) "Limited knowledge of world and events after 2021".
What surprises me is that people like to blame ChatGPT for these errors :-|.
Show threadElio CampitelliFeb 25, 2023
@nmouplus @atlefren @opencage That first caveat should read "may occasionally generate correct information"
Show threadMartin Vermeer FCDFeb 23, 2023
@opencage Oops
Show threadAustin CunninghamFeb 23, 2023
@opencage as a software engineer I have tried chatGTP five times and it's been wrong five times. Inventing gitlab endpoints that don't exist was one answer. Invalid promql queries was another answer. Invalid config for envoy was another. ChatGTP Iike any tool get a second opinion.
Show threadByronCinNZFeb 23, 2023
@opencage Begining to think ChatGPT is a bomb thrown into the internat
Show threademiFeb 24, 2023
@ByronCinNZ @opencage we are not far away from real threat of life here anymore…
Show threadAndrew LFeb 23, 2023
@opencage ”ChatGPT is doing exactly what it’s makers intended - producing a coherent, believable answer. Whether that answer is truthful does not seem to matter in the slightest.”
ie a bullshit (in the academic sense) machine. #ChatGPT #Bullshit
Show threadSans-culottes Feb 23, 2023
@opencage LOL ChatGPT just makes up shit. It kept gaslighting me over broken urls it provided. I feel for you.
Show threadn1k0Feb 24, 2023
@opencage this is your homepage atm. No indication there’s anything going on, you could easily add some yellow banner with a warning and a link to your blog post.
Show threadPseudo NymFeb 26, 2023
@n1k0 @opencage directly in the sign-up page would be better. "If you came here from advice from chatGPT, we don't do that. It's wrong."
Show threadMark W. Gabby-Li 🐌Feb 24, 2023
@opencage Well, since you asked for suggestions, what if you made that request return a link to the blog post explaining you don't provide this service?
Show threadMark W. Gabby-Li 🐌Feb 24, 2023
@opencage
In the example you provided of the generated code, that would, after all, print a web address to the console.
Show threadOpenCage 👉🌍Feb 24, 2023
@Aradayn yes, that is the next step and why we wrote the post
Show threadEdwin GroothuisFeb 24, 2023

@opencage @NewtonMark There is only one way out of here... To make such a service!

(Note that I'm a happy official API user of opencagedata)

Show threadOpenCage 👉🌍Feb 24, 2023
@mavetju @NewtonMark Setting aside technical difficulties, do you think an API where anyone could at all times track the location of your mobile phone would be a good idea? Do you see any potential for abuse?
Show threadEdwin GroothuisFeb 24, 2023
@opencage @NewtonMark /a dude, /s
Show threadpgcdFeb 24, 2023
@opencage ChatGPT "reviewed" one of my albums.
It loks great and interesting and there is literally *no intersection* between reality and what it excreted.
Show threadZach LiptonFeb 24, 2023

@opencage @anildash I feel like the YouTube videos are a human version of the ChatGPT problem

You make money on YT by getting views, so you make videos based on stuff people search for, so you make videos about impossible things like geolocating anyone’s phone (either out of ignorance and/or because wildly inaccurate videos still get you paid). The final step of “so a large language model slurps up and regurgitates your nonsense” is just the icing on the cake of bad incentives all the way down

Show threadmsgbi Markus SagebielFeb 24, 2023
@zachlipton @opencage @anildash @charles_ex @aloa5
Show threadOpenCage 👉🌍Feb 24, 2023
@zachlipton @anildash yes, this is the problem
Show threadDetlef SteuerFeb 24, 2023
@opencage More and more stories of chatGPT costing other peoples lifetimes (and therefore money). Looks like a mild version of sending swat teams but nevertheless very disturbing for those, who have to handle the fallout of the babbling machine.
Show threadlevampyreFeb 24, 2023
@opencage The problem with ChatGPT is, it is kind-of ddos-ing our human capabilities of finding valuable information, it is flooding our system, displacing human output (the good and the bad one) and cluttering everything with garbage. It can produce garbage faster than we can determine, if an information is garbage or not. It has become harder every day since the beginnings of the internet. But now informational garbage production is automated and people, who seek real information, are doomed.
Show threadCodex ☯️♈☮Feb 24, 2023

@levampyre
@opencage

This is the real Singularity. Not a point of infinite knowledge but an engine of chaos and entropy so sophisticated it dumps all knowledge into a blackhole from which no truth can escape.

Show threadschrotthaufenFeb 24, 2023

@opencage @LaF0rge You’re not alone in this, unfortunately. See for example https://www.theguardian.com/technology/2023/feb/21/sci-fi-publisher-clarkesworld-halts-pitches-amid-deluge-of-ai-generated-stories

“AI”, As it is today, is a huge danger to society, democracy, and truth itself, I’m afraid.

Sci-fi publisher Clarkesworld halts pitches amid deluge of AI-generated stories

Founding editor says 500 pitches rejected this month and their ‘authors’ banned, as influencers promote ‘get rich quick’ schemes

The Guardian
Show threadEckes Feb 24, 2023
@opencage don’t complain, monetize the leads :)
Show threadGreenSkyOverMe (Monika)Feb 24, 2023
@opencage It looks like we need a new response code, something like 666 "ChatGPT lied to you"
Show threadOphitoxaemiaFeb 24, 2023
@opencage The obvious answer is you need to start providing that lookup service. We can't afford to disappoint AI
Show threadjoy larkin 🌺✨Feb 24, 2023
@opencage You’re not the only one to see this…😩 Last week, ChatGPT hallucinated a @ZeroTier feature and provided explicit download and install instructions for a software package we’ve never had.
We only learned about it after a community member asked us where they could download the package.
Show threadJeanne BoyarskyFeb 24, 2023
@opencage yuck. As far as things you can do, the suggestions in this thread are good. Another is to link to the blog post from the sign up page. “Find about us from ChatGPT? Read this”
Show threadOpenCage 👉🌍Feb 24, 2023
@jeanneboyarsky yes, but absolutely no one will read anything
Show threadMatthew FlintFeb 24, 2023
@opencage
Quite frightening that people actually want that functionality.
Even if it were possible to implement that API, I definitely wouldn’t.
Show threadMagess Feb 24, 2023
@opencage I wonder if things like this are going to result in a lawsuit that will actually get them to can this stupid tech.
Show threadAshtonFeb 24, 2023
@opencage genuinely not sure why anyone is impressed with ChatGPT. This kind of stuff keeps happening.
Show threadKyle BrownFeb 24, 2023
@opencage I do wonder if this will/could end up being a legal test case for liability for chatGPT… there is actual harm here.
Show threadVick Forcella ™🌈🌳❄️☑️:verifiFeb 24, 2023
@opencage Charge us$8 for API usage, that returns "New Iork", always.
Show threadThomas Lee ✅ Feb 24, 2023
@opencage if you really know what you are doing, chatgpt can be helpful. But for the #powershell questions i asked, many produced code that LOOKS correct, but does not actually work!
Show threadNathan StohlmannFeb 24, 2023

@DoctorDNS @opencage I was having a discussion with a co-worker about that the other day that went essentially like:

Him: "Look! This example is great and functional therefore this is amazing!"

Me: "How many attempts did that take?"

Him: "It wasn't too bad! Definitely not more than an hour of prompt hacking!"

Me: "For 10 lines of pretty simple code? Oh, and there's a critical bug on line 4 BTW..."

(LLM is really cool, and very shiny, but I'm still waiting for the glow to wear off.)

Show threadJim LillicotchFeb 24, 2023
@opencage
Search for the term (not on Bing) #BullshitFountain and you'll find #ChatGPT at the top
Show threadCassandrichFeb 24, 2023
@opencage Seems like this is cause for a lawsuit. I'd love to see that happen at scale.
Show threadKaylie Feb 24, 2023
@opencage freely accessible AI means people will take its output at face value instead of assessing what it actually says and cross-referencing with reliable sources.
AI is cool, but it's too early to take everything it says under assumption that its correct.
Show threadGlen GibbFeb 24, 2023
@opencage Would replying to that API request with the blog post URL be a reasonable way to alert people to the problem?
Show threadAaron In MinnesotaFeb 24, 2023

@opencage expecting to see a lot more of:

"Note: this post was written by a human, not an AI."

Show threadJohn Bell 🇺🇦🇪🇺🔸🕷️🏴󠁧󠁢󠁷󠁬󠁳󠁿Feb 24, 2023
@opencage @mmasnick yes, asked it an engineering question and it came up with the wrong answer
Show threadMattFeb 24, 2023
@opencage yikes. Imagine the disinformation possibilities
Show threademmatonkinFeb 24, 2023

@opencage
A colleague said 'let's get ChatGPT to recommend some papers!' today, so I got them to open the page and ask it for ten papers about fish (because, hey, why not).

It duly generated a list of papers.

'See?' the colleague said. 'It worked. They have URLs and everything.'

We checked the links. Almost all of them were 404s. Of the one that wasn't, it went to a paper with an unrelated title, author and subject. My colleague was not just dismayed at this, but apparently quite shocked.

Show threadJonPFeb 24, 2023

@opencage there's a second problem beyond the lying. How do you get ChatGPT to "forget" something, in this case false information?

I asked Bing chat this exact question (I was thinking about the right to be forgotten in GDPR). It said it couldn't answer and was still learning 🤔

Show threadThomas H Jones IIFeb 25, 2023

@opencage

Reading this thread, makes me wonder if there's a product liability case waiting to happen. I mean, it costs money to just reply to support requests, not to mention "reputational harm". Presumably, there's other companies and organizations experiencing equivalent issues.

Show threadBill SeitzFeb 25, 2023
@opencage how long until you update your home page or Sign Up form?
Show threadNoam RossFeb 25, 2023
@maelle 👆!
Show threadJuan LuisFeb 25, 2023
@opencage Sam Altman: "I am an stochastic parrot and so are you"
Actual stochastic parrots:
Show threadBruno Rodrigues  Feb 25, 2023
@opencage people using chatgpt as a search engine never fail to amaze me
Show threadPCFeb 26, 2023

@opencage

I probably don’t understand it well enough, but in my opinion, AI should reference all sources. And possibly a second AI should fact check it with different sources.

Show threadRoePipi (verified)Feb 27, 2023
@opencage Just open that API and reply every request with the location of the OpenAI centre :D
Show threadOpenCage 👉🌍Feb 27, 2023
@RoePipi not sure that would be appreciated by our existing customers