RE: https://mastodon.social/@gauthampai/113626318842111574
It's been exactly one year since I mentioned this and boy, have things progressed so much! I ended up building my own super app for feeds, bookmarks, notes and more. More details soon...
However, instead of leveraging traditional ML approaches where they still work well, many have jumped straight to LLMs for everything. In reality, traditional ML solutions can serve as an effective first-level filter - delivering solid results while keeping time and costs under control. I've found this approach to be incredibly practical, yet seems overlooked. Why aren’t more people talking about this?
Before the advent of LLMs, machine learning had already made significant progress. We had summarizers, classification/clustering algorithms, and other solutions that were about 80% effective. However, the lack of final-mile refinement often kept them from being production-ready. With LLMs now in the picture, many of these use cases are resurfacing.
This is the reason the "Semantic Web" had a concept of trustworthiness.
From the W3C site:
Not everything found from the Web is true and the Semantic Web does not change that in any way. Truth - or more pragmatically, trustworthiness - is evaluated by each application that processes the information on the Web. The applications decide what they trust by using the context of the statements; e.g. who said what and when and what credentials they had to say it.
Note that, this is not the classic LLM hallucination but a problem in how the AI handles misleading data.
https://simonwillison.net/2024/Dec/29/encanto-2/
You can check it yourself:
https://www.google.com/search?q=encanto+2
#GenerativeAI, #AIEthics, #LLMs
Google search hallucinates Encanto 2
Jason Schreier on Bluesky: > I was excited to tell my kids that there's a sequel to Encanto, only to scroll down and learn that Google's AI just completely made …
An interesting issue with #LLM training:
Jason Schreier found that Google's AI incorrectly generated a non-existent sequel to the movie Encanto. The AI misinterpreted a fan-generated idea from an idea wiki as a real movie, including a non-existent release date and misleading links. This shows a key limitation in LLM training where it lacks the ability to distinguish real v/s made up information.
Byte Latent Transformers : Patches Scale Better Than Tokens
A Deep-dive into the latest advancement in tokenization from Meta, which for the first time, matches tokenization-based LLM performance at scale with significant improvements in inference efficiency.
That said LLMs will continue to need a second brain, just like how we use a note taking software. Not everything can happen in the LLM's memory.
Right now we use Knowledge Graphs in RAG which is external to the LLM. The LLM's brain doesn't have this structure. Intuitively, it seems that LLMs can do with better understanding of the world. After all, we don't see the world in terms of "alphabetical or language tokens".
The work happening with things like Large Concept Models, byte latent transformers and fine tuning is giving me an impression that in the near future we will see more of structured semantic information getting into the core of neural networks.
