Spring Jo 🥚 
🍀Apple did the research; LLMs cannot do formal reasoning. Results change by as much as 10% if something as basic as the names change.
https://garymarcus.substack.com/p/llms-dont-do-formal-reasoning-and
Anders Puck Nielsen@ShadowJonathan Why would we judge LLMs on their ability to solve complex tasks? The interesting thing is if they can solve simple tasks well enough to be useful.
Cassandrich@anderspuck @ShadowJonathan Which they also can't do.
@dalias @ShadowJonathan They can absolutely do certain things well enough to be useful. Create a fairly accurate transcript of a podcast, for example.
Jeff Grigg@anderspuck @dalias @ShadowJonathan
LLMs are NOT doing *speech to text* translation -- doing transcripts from audio (podcast). That's a different set of AI technologies.
The industry has been developing "AI" technologies since before I was born. Some are quite useful.
It's the "Generative AI" subset (which includes LLMs, chatbots) that is so misleading, mostly useless, and incredibly wasteful.
@JeffGrigg @dalias @ShadowJonathan True. I kind of bundled ChatGTP and Whisper in that statement.
I don’t find generative AI useless, though. There are many tasks for which it is very good, but probably not those flashy ones many people are thinking about. For example an LLM is much better at sentiment analysis than older methods.
I don’t find generative AI useless, though. There are many tasks for which it is very good, but probably not those flashy ones many people are thinking about. For example an LLM is much better at sentiment analysis than older methods.
@anderspuck @JeffGrigg @ShadowJonathan Are you sure about that? I'm pretty sure they do an extremely racist version of "sentiment analysis".
Amber 
@dalias @anderspuck @JeffGrigg @ShadowJonathan Let’s be clear, the LLM is not developing racism out of nowhere. It is just able to amplify racial bias in the dataset. The stuff used to train it was already racist. It’s extremely hard to filter that out. I still laugh at tip culture being ingrained into LLMs. Some would do "better work" than normal if you bribed it with a tip. Freaky stuff.