I asked ChatGPT about primes ending in 2 to make it prove a point and it proved the point far better than I could have hoped for.
Please do not be a fool who trusts ChatGPT with anything outside your field of expertise, and even then double or triple check what it tells you if you must use it.
I still have to laugh out loud about it. My favorite part has to be:
Let's consider the first 100 primes ending in 2
@_cnt0 @alinanorakari Ahh, but did you try in base 2?
(Ourari? Or just someone else with that avatar?)
@elithebearded
It's a nightmare when a two comes up in binary: https://m.youtube.com/watch?v=MOn_ySghN2Y&pp=ygUSYmVuZGVyIG5pZ2h0bWFyZSAy
(someone else with that avatar)
@elithebearded @alinanorakari Wait a minute.... *THE* Eli the Bearded? Holy smokes! You're the guy whose name is on all the FAQs, Moria, procmail, pbm, and, of course, rec.arts.erotica!
Have you written a book yet? I think I wouldn't be the only one curious to hear your take on how modern "social media" compares to the Usenet of yore.
@alinanorakari
That's pretty funny. Lest anyone think this was an unfair test of ChatGPT, mathematics is one of its core domains of expertise. (At least that's what ChatGPT told me, but maybe I should know better than to believe a pathological liar twice.)
A while ago I asked it for the 100th digit of π and it was hilariously aggressive that there is no 100th digit. It seemed to be basing that on the fact that π doesn't repeat and there are less than 100 distinct digits, but I think I broke it when I asked about base 100. It eventually informed me that there isn't even a first digit of pi, either.
I will note that the answers it gave you are both (a) shorter and (b) less arrogant sounding. ChatGPT previously was incredibly rude, unable to admit, much less contemplate, possibly being wrong.
I think the problem was that they trained it on transcripts from very smart people. It learned to mimic their charmless assertions and condescending style, but with none of their knowledge.
@abananabag @alinanorakari this is a good point to make, though I'm in disagreement:
ChatGPT's area of expertise is *conversation* and nothing else. Everything else is incidental to it's design (though they keep working to improve the quality of it's output). To be precise, it's focus is on creating what a reply would look like.
This is why it gets a reputation at time for being argumentative, because if the response looks upset it thinks it's looking at the start of an argument so it thinks the reply would be argumentative.
If you ask it for prime numbers, it knows the response looks like a bunch of numbers.
It does well with programming because code is just another sort of language pattern.
Likewise with answering questions about general information because the best looking response is an accurate one.
But that's also why it hallucinates (makes up false information) because "I don't know" is not considered a good response in the system.
@shiri @alinanorakari Thank you for your insights. I'm curious how you know that "the best looking response is an accurate one".
Also, "code is just another language pattern" seems questionable to me. I'm coming from a computer science perspective where "code" is considered to be closely akin to mathematics and rather different from any natural language.
@abananabag
Mathematics is also considered a language. Not that it can do alright with explaining or creating basic formula, it just does poorly at executing it.
As far as "best looking response", that's part of the training process and why they routinely talk about improving accuracy. It's "motivation" is high training scores, during training it gets given a higher score when the information is accurate. This doesn't mean it's always accurate, just that it favors acuracy.
@alinanorakari
@abananabag @alinanorakari They are somewhat black box entities in which we train their responses by feeding in large quantities of data and temper those responses by manual training, in the manual training especially we set a tone for what's expected of the AI.
The scoring system creates a sort of implicit motivation for the AI, it's designed to "want" a higher score and learns from the training what answers give it higher scores and learns patterns from that.
The nature of the models is that they're generating what looks like human responses. It's a whole different beast (of which this would be just a component) to make an AI that's specifically broad general knowledge expertise, especially when said AI needs to also have a conversation with the user.
And as far as things like mathematics being a language, it's well established and it's also why their programming and math expertise are accidental. They didn't design these models initially for that, they just fed it craploads of data and it incidentally picked up those skills from the training data set. Both math and programming statements are just a form of instructions, just rigid ones. And because it's rigid, it's actually probably easier for LLMs to understand.
It's similar to how they have a wide array of language understanding, because they're not natively english or such... they had a wide array of languages thrown at it, and it's understanding of any language is learned from the dataset rather than being hard programming. So it knows English and German much the same as it knows Python and C++.
Attached: 1 image #OpenAI 's ChatGPT https://openai.com/blog/chatgpt/ ..........??
@alinanorakari I tried to use it to get a hint on a regex I got stuck with. It gave me 4 correctly looking wrong answers I had to refuse, then a correct one, but not working in subset of regex used in sed, and when repeatedly told that it doesn't work in sed, it got into a loop of "Oh, I'm sorry, you are correct, this won't work. The correct regex is: repeats the same regex
To be fair, at the end, it got unstuck and even provided explanation for why isn't it working, but yea... Whoever uses it for something they don't already know enough to verify the result is setting themselves up for a failure.
@alinanorakari
This is the best example that shows how those text generators work and fail, that I've seen yet.
Thanks for that.
@alinanorakari also i sometimes imagine that they secretly give me ChatGPT 4, presumably because they discovered my questions are such interesting training material. just because of how surprisingly good the answers seem sometimes.
i'm preeetty sure that's nonsense, but i guess i like to feel special. :P
@alinanorakari In my job, I do communications on complex topics such as banking, accounting, auditing and regulations.
FWIW, whenever I asked ChatGPT about something I was currently working on, most of the answers I got were clearly wrong but worded very confidently.
Therefore, I can only reinforce this warning!
@alinanorakari Anything about digits or letters is super hard for ChatGPT. It sees our messages and all the data it was trained on translated into a different (huge) alphabet. Its alphabet would write 74815 as just two tokens, one for 748 and one for 15. It's useful when "sleep" is one token and "ing" is another. But it sucks for numbers. Models trained with per-digit tokenization do better on arithmetic. (https://arxiv.org/abs//2305.14201)
Anyway, I don't mean to make excuses for ChatGPT. 😅
We introduce Goat, a fine-tuned LLaMA model that significantly outperforms GPT-4 on a range of arithmetic tasks. Fine-tuned on a synthetically generated dataset, Goat achieves state-of-the-art performance on BIG-bench arithmetic sub-task. In particular, the zero-shot Goat-7B matches or even surpasses the accuracy achieved by the few-shot PaLM-540B. Surprisingly, Goat can achieve near-perfect accuracy on large-number addition and subtraction through supervised fine-tuning only, which is almost impossible with previous pretrained language models, such as Bloom, OPT, GPT-NeoX, etc. We attribute Goat's exceptional performance to LLaMA's consistent tokenization of numbers. To tackle more challenging tasks like large-number multiplication and division, we propose an approach that classifies tasks based on their learnability, and subsequently decomposes unlearnable tasks, such as multi-digit multiplication and division, into a series of learnable tasks by leveraging basic arithmetic principles. We thoroughly examine the performance of our model, offering a comprehensive evaluation of the effectiveness of our proposed decomposition steps. Additionally, Goat-7B can be easily trained using LoRA on a 24GB VRAM GPU, facilitating reproducibility for other researchers. We release our model, dataset, and the Python script for dataset generation.
@alinanorakari @darabos
"a lot of people who think because it answers like an eloquent human it _must_ have basic human knowledge" ...
Suddenly I see a huge overlap here with voters.
@alinanorakari There was a video of a reporter who thought DAN was a super-hack, he asked DAN to give his own social security number and when it responded he acted really scared, then he read it... 987-65-4321. And he realized DAN could just make shit up too.
It’s at 9:33 in this video: https://youtu.be/RdAQnkDzGvc
Did it not understand the instructions? What happened?
Sea Cow
Alina Yossimouse
_cnt0
Eli the Bearded
Finley Ⓥ
abananabag
Deb has moved! (see profile) 🇨🇦
Allan Chow
maya
Andrej Shadura
okanogen VerminEnemyFromWithin
.morris
Shiri Bailem
Fabian ¯\_(ツ)_/¯
Natanael ⚠️
🅵
Shine
Aknorals⚑Ⓐ 🏴
Cegorach
sofia ☮️🏴
Möbius Moa🪶
yuki - queen of the snow
Stefanie Schulte
Daniel Darabos
Bornach
chuckadeus kummerer
Tom
Nick Gonzo
Klaus Alexander Seistrup
Andromeda Yelton
Cassandrich
Medea Vanamonde🏳️⚧️ ♀
Jonathan
VHasch