dfloresbxDec 19, 2022
Dan Riskin
I asked #chatgpt to write me an essay, with citations, about a scientific topic I know very well (bat crawling behaviour). It wrote a bad essay that didn’t really say much, but cited two papers with which I wasn’t familiar, in very good journals. I was freaked out! How did I not know those papers?
Turns out, chatgpt just made them up. The papers don’t exist. That’s a handy thing to know.
Dec 18, 2022 at 3:55amWeb
Show threadeigenman Dec 18, 2022
@riskindan Had the same experience asking it for some coding questions. It posted things I didn't know but looked right(have 25 years sftwr exp). Go to compile and doesn't work. Turns out it either just made up the language or was mixing languages or both. But a surface examination of the code looks like it SHOULD work. The pattern was spot on but the details were a fairy tale.
Show threadOlivier Slabbers Dec 19, 2022
@eigenman @riskindan Maybe it writes bad code, but it can add code comments in any tone of voice you want it to.
Show threadeigenman Dec 19, 2022
@Ik @riskindan 😂 I'm going to start submitting pull requests with that style of commenting and see if ppl notice.
Show threadOlivier Slabbers Dec 19, 2022
@eigenman @riskindan Or take it a step further https://shakespearelang.com/1.0/
shakespearelang

Show threadeigenman Dec 19, 2022
@Ik @riskindan Might be easier to chuck my code into ChatGPT and just tell it to rewrite my comments in the style of Shakespeare.
Show threadOlivier Slabbers Dec 19, 2022
@eigenman @riskindan A plugin that rewrites comments shouldn’t be too hard? ChatGPT seems to be quite good at interpreting code (and data) and describing what it does.
Show threadeigenman Dec 19, 2022
@Ik @riskindan Ppl have been making ChatGPT plugins. True, Probably not hard but I don't even have time for the things I currently do lol.
Show threadOlivier Slabbers Dec 19, 2022
@eigenman @riskindan When does that whole robots taking our jobs thing finally begin?
Show threadeigenman Dec 19, 2022
@Ik @riskindan Robots can have my job. It's usually JR DEVS PMing me all day because they're lazy. Funny thing is I've told them to ask ChatGPT first now. Might not have been a good idea as ChatGPT is wrong a lot but the questions they ask are things like, how to write a unit test, how do observables work in angular, etc... and they are just lazy at google so maybe ChatGPT makes it easier and they stop bothering me as much
Show threadeigenman Dec 19, 2022
@Ik @riskindan ACTUALLY, what I really want is a speech to text interface to ChatGpt.
Show threadRoss CarrollDec 18, 2022
@riskindan it made a citation for me that included a real doi for a completely different subject 😬
Show threadUlrik Nyman ⬡Dec 19, 2022

@umbrafiles @riskindan
To me this is not surprising at all.
It has been trained to make things that look right.

Maybe this type of system could be used in a loop with other systems that checks for correctness.

Show threadJohn S. WilkinsDec 18, 2022
@riskindan I had a student do this, in my field.
Show threadtolortsluborDec 18, 2022
@riskindan Screenshot, please. Was the ability to lie, and predilection to do so, deliberately programmed into #chatgpt #gpt3 #chatgpt3 #AI #chatbot
Show threadJames BaillieDec 19, 2022
@tolortslubor @riskindan Large language model AI doesn't come with any sort of concept of truth: the only objects it handles are words and their connections. The problem isn't that it's built to intentionally lie, but that preventing this sort of machine producing untrue statements is immensely difficult: it produces a probabilistic response based on the prompt, without a check of external truth (and if you think "well that sounds dangerous"... yes, it very much is.)
Show threadMartino MosnaDec 19, 2022

@JubalBarca @tolortslubor @riskindan

My take on it is: GPT-3 doesn't have *structure*.

Wanna have structure? You've got to give it yourself to the AI in your prompt.

It's actually good for fluffing up raw, cold, structured data... and make it nicely human readable.

Show threadAnsel HalliburtonDec 19, 2022
@JubalBarca @tolortslubor @riskindan Been thinking about this. I wonder about filtering LLM output through semantic linked data.
Show threadWolf480plDec 19, 2022

@JubalBarca @tolortslubor @riskindan
An intuitive explanation I've heard is:

ChatGPT just wants to tell you a cool story.

Show threadSimon LucyDec 19, 2022

@tolortslubor @riskindan

If you can say it in the language then the model will generate sentences without any value attached as to truth or fact, it just has to be sayable and fit the style required.

Show thread猫好きな人Dec 19, 2022

@tolortslubor

Ask it: “Explain the research of [made up name] into [made up research area]”. Then ask it to provide citations.

Show threadMarverdeDec 19, 2022
@valoisdubins @tolortslubor
🤣🤣🤣🤣🤣🤣🤣
Show thread猫好きな人Dec 19, 2022

@Marverde @tolortslubor

I also did this.

Show threadMarverdeDec 19, 2022
@valoisdubins @tolortslubor
Could be so useful...
Show thread猫好きな人Dec 19, 2022
@tolortslubor @Marverde
Yet she struggles to be accepted at conferences to speak.
Show threadtolortsluborDec 19, 2022
@valoisdubins That an #ai can make up citations demonstrates creativity, but also a lack of understanding, research skills and ethics
Show thread⁂ Fish Id WardrobeDec 19, 2022

@tolortslubor @riskindan In order to lie, you have to understand what you are saying.

So-called AI does not. There's no "there" there.

Show threadRob BentlyewskiDec 18, 2022
@riskindan I asked #chatgpt for a case that supports a very simple legal proposition, and it confidently gave me a made up a case name, reporter number, first page, and date. I responded saying “this case doesn’t exist” and it told me to call Westlaw 
Show threadlisawbordenDec 18, 2022
@riskindan So I guess you need to specify genuine citations to sources that actually exist? Wonder if that would work.
Show threadCarlo SchuengelDec 18, 2022
@riskindan quite crazy indeed. Even after asking not to make up studies, #chatgpt apologized, and continued to serve made-up studies. Perhaps it is a good thing, means I can keep my job a bit longer …
Show threadTigerfortDec 18, 2022
@riskindan
ChatGPT is basically a sort of Boris Johnson simulator - good at sounding superficially plausible on subjects the reader/listener knows nothing about, but very obviously spouting total bullshit to anyone who understands the topic at hand.
Show threadPaul H 🏴󠁧󠁢󠁷󠁬󠁳󠁿🇪🇺Dec 19, 2022

@tigerfort @riskindan

Or is Boris Johnson a chatgbt avatar?

Show threadAtomic OrbitalsDec 19, 2022
@tigerfort @riskindan Debating societies will soon field teams entirely consisting of ChatGPT instances.
Show threadWilliam GunnDec 18, 2022
@riskindan I had a similar experience with made up authors. Things have been improving with shocking rapidity, though!
Show threadLiverpoolWizDec 18, 2022
@riskindan ChatGPT grows more human-like by the day.
Show threadJennie BlakeDec 18, 2022
@riskindan we have run a few tests like this and the most common pattern appears to be “sometimes real journal/always made up paper”. The broader (ie less niche) the original prompt, the more likely it is able to pull real journal titles. It was sort of fascinating.
Show threadJeff KirvinDec 19, 2022
@riskindan @lj I think what folks don’t grok about #AI is it doesn’t really understand what it’s doing. #chatgpt is mimicking writing, but it has no knowledge. #midjourney is mimicking art, but it doesn’t understand beauty. None of it is real.
Show threadPatrickMeyfroidtDec 19, 2022

@riskindan Yes, did it to me too:

https://fediscience.org/@pmeyfroidt/109533251663402214

PatrickMeyfroidt (@[email protected])

@[email protected] * Invente (totalement, de A à Z!, avec un faux titre, des faux auteurs, un faux DOI) des références scientifiques quand on lui demande s'il peut citer des sources pour soutenir ses propos: (conversation de moi-même avec ChatGPT le 17/12/2022): https://www.dropbox.com/s/jhkhclysrsqqbxd/New%20Chat_rebound%20effect.html?dl=0 (3/4-5)

FediScience.org
Show threadSusan [email protected]Dec 19, 2022
@riskindan So artificial intelligence can create artificial facts. Wonder how soon they'll be in Wikipedia
Show threadSimon LucyDec 19, 2022

@SusanKraemer @riskindan

I wouldn't assume they aren't.

Show threadTony BowdenDec 19, 2022
@SusanKraemer @riskindan They seem to have updated #ChatGPT a couple of weeks ago to initially refuse to provide citations when asked. Saying that you want them to add to a Wikipedia article is one way around that! https://wikis.world/@oravrattas/109467266073745599
Tony Bowden (@[email protected])

Attached: 3 images @[email protected] Aha! I've been having difficulty in the last 24 hours to get #ChatGPT to entirely fabricate sources like it was doing last week, even when I went through exactly the same prompts as I had before. Now it always just says it doesn't have access to specific citations. But you gave me an idea, and lo and behold, even when it refuses to give a citation when asked directly, reframing it as "If this were a #Wikipedia article what would the sources be?" does the trick!

Wikis World
Show threadEd RossDec 19, 2022

@riskindan
Yes, I've heard it can do that (in this very good video introduction to #ChatGPT https://youtu.be/3yUPdYK9E2g )

I think a good way to think of #ChatGPT is as that know-it-all in the pub who can tell a good story, knows a lot about a lot of subjects, but some of what they "know" might actually just be what they are making up as they are speaking.

Still, I like it for what it can do, some of which I've documented at https://blog.edross.co.uk/archive/tagged/chatgpt

@StevePeers

Luke LOVES this AI chat bot

YouTube
Show thread猫好きな人Dec 19, 2022

@riskindan

I found this out too. I asked it to explain the research careers of entirely fictitious people.

I also induced it to make up fictitious antisemitic rap lyrics (not shown here) and attribute them to Kanye West. They scanned and rhymed.

Show threadRoel WillemsDec 19, 2022

@riskindan Thanks for sharing!

I also experienced the content be be convincingly inaccurate in some occasions (with a topic I'm very familiar with). Although the model just making up papers is next level (although on a second thought it's not super surprising), good to keep in mind indeed.

Show threadPeter Prevos 💧🎓📊🇳🇱🇦🇺Dec 19, 2022
@riskindan The bot has learned from humans to make shit up.
Show threadAndrew JonesDec 19, 2022
@riskindan the ZX80 isn’t the computer we use today. It’s V1, give it time to learn more about niche subjects and check again.
Show threadRuben van de VijverDec 19, 2022
@riskindan And a warning to students who want to bilk essays from it!
Show threadNicolas PettiauxDec 19, 2022
@riskindan can you share the result ?
Show threadharpo_bzhDec 19, 2022
@npettiaux
We are waiting for The Truth. ChatGpt is just a sophisticated, elegant and powerful statistic tool. We did it the same with talented oracles in Delphes, with chicken goats e.g.
Fascination of the appearence of Truth. Human needs magic but pretends to be rational.
Its an old and dangerous bias of our way of thinking.
#ChatGpt #truthiness #Magic
@riskindan
Show threadTristan HendersonDec 19, 2022
@riskindan I suspect that we are acting as free labour beta testing ChatGPT for for OpenAI and they will fix this when it becomes a paid product.
Show threadPedroDec 19, 2022
@riskindan so it’s a mediocre undergrad.
Show threadArseny KhakhalinDec 19, 2022
@riskindan Yeah, that's one of its most famous features :) An even more extreme version of it: if you ask it to produce code, it invents useful libraries that don't exist :) Like `from magic import turn_to_frog`, but of course when you check, this library isn't really a thing!
Show threadManuel BiertzDec 19, 2022
@riskindan Something similar just happened to me in class. I asked for the reasoning provided by a specific paper (author, year, topic) and in the first class, it found the paper and its reports were somewhat correct. But in the second class I did the same in a *new* chat and it invented a paper with a generic, though existing title and a completely fake author.
That was quite a stunt.
Show threadstoneheadDec 19, 2022
@riskindan #ChatGPT is called "mansplaining as a service" for a reason 😉
Show threadCharles A. LaughlinDec 19, 2022
@riskindan that is useful! Great way to come up with ideas for new articles!
Show threadbgt loverDec 19, 2022
@riskindan yes, and here comes what I and other people warned us about with AI, #chatgpt can be such a bullshit machine sometimes, you have to vet every single piece of info it spews out, by which point you might as well just do the thing your self.
Show threadnbannierDec 19, 2022
@riskindan Noticed something similar with an essay about Baudelaire: it just made up a quote. Then when I asked the exact reference, it made up even the poem !
Show threadProkofyDec 19, 2022
@riskindan It did that with me, too, inventing two authors I didn't in fact translate. Hope they aren't going to use it in health care.
Show threadnerdybutcute Dec 19, 2022
@riskindan yeah ChatGPT is extra terrible at citations.
Show threadWolfgang CramerDec 19, 2022
@riskindan Interesting. I asked it to summarize some of my own papers from way back, and also tell me who the coauthors had been. It first gave me a list of completely unknown people, and when I said that it was incorrect, it gave me the correct list! Like a lazy student, somehow.