L4sBotJul 20, 2023

Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds. Researchers found wild fluctuations—called drift—in the technology’s abi...

https://lemmy.world/post/1855466

Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds. Researchers found wild fluctuations—called drift—in the technology’s abi... - Lemmy.world

Over just a few months, ChatGPT went from correctly answering a simple math problem 98% of the time to just 2%, study finds. Researchers found wild fluctuations—called drift—in the technology’s abi…::ChatGPT went from answering a simple math correctly 98% of the time to just 2%, over the course of a few months.

Show threadDominicHillsunJul 20, 2023

It seems rather suspicious how much ChatGPT has deteorated. Like with all software, they can roll back the previous, better versions of it, right? Here is my list of what I personally think is happening:

  • They are doing it on purpose to maximise profits from upcoming releases of ChatGPT.
  • They realized that the required computational power is too immense and trying to make it more efficient at the cost of being accurate.
  • They got actually scared of it’s capabilities and decided to backtrack in order to make proper evaluations of the impact it can make.
  • All of the above
    • Show threadSomeDudeJul 20, 2023
  • There’s a bug they haven’t found yet
    • Show threadkillerinstinct101Jul 20, 2023
    This is what was addressed at the start of the comment, you can just roll back to a previous version. It’s heavily ingrained in CS to keep every single version of your software forever.
    Show threadCaptainAnikiJul 20, 2023
    I don’t think it’s that easy. These are vLLMs that feed back on themselves to produce “better” results. These models don’t have single point release cycles. It’s a constantly evolving blob of memory and storage orchestrated across a vast number of disk arrays and cabinets of hardware.
    Show threadLazylazycat
    Exactly this, that’s why Loab exists forever now.
    Jul 20, 2023 at 12:48pmWeb