derpyunicornMar 27, 2023

@Riedl

LeCun's arguments about why LLMs are limited seem fine to me. However his last slide is just plain wrong. He claims that "almost nothing is learned through supervision or imitation". Babies/toddlers learn almost everything by imitation, and later of course comes a lot of supervision (schools). Animals also learn through imitation and some light supervision.

https://drive.google.com/file/d/1BU5bV3X5w65DwSMapKcsr0ZvrMRU_Nbi/view?usp=drivesdk

Show threadderpyunicornMar 27, 2023
Of course all living beings are also born with sets of behavioral patterns, so they are "pre-trained" in that sense. But it doesn't seem like LeCun is talking about those.
Show threadMark Riedl
@derpyunicorn There are some strange conclusion in that slide deck. I didn't dig through to the end because a lot of it is regurgitated from prior things I've seen.
Mar 28, 2023 at 2:28amWeb