Miguel de Icaza ᯅ🍉Run your language models directly on your phone, courtesy of the ApacheTVM machine learning compiler:
My phone got warm, but it feels as if you were doing gpt4 on the cloud, except it is all local!
I knew upgrading my iPhone on launch day would paid off.
Sorry “Apple doesn’t innovate so I only upgrade my phone every six years” peeps.
Akshay | అక్షయ్@Migueldeicaza The implementation is both incredible and bad. Incredible that I’m running an LLM natively on my phone.
Bad because… it neither knows traffic laws in Barcelona nor MC Hammer’s unique style.
Choong Ng@cydonian @Migueldeicaza It's more useful to think of LLMs as tools for manipulating language rather than databases containing facts. Incidentally there is a lot of fact-like material in there but not in a way that you want to rely on.
@choong @Migueldeicaza You’re explaining LLM’s to me? 😀
My point is that _this_ LLM has worse outcomes than a cloud-based one perhaps because it has fewer hyper parameters and fewer training weights. Essentially: when you evaluate open-source LLMs, you must consider the possibility that they’re of even lesser quality than what you’re seeing with GPT 3.5 or GPT 4.
But that’s also okay. There’s an opportunity to train them the way you want them, evaluate them per your standards.
@cydonian @Migueldeicaza Oh yes, from what I can tell all the smaller models are much less capable than GPT-3.5. What I meant is testing capability with language manipulation tasks like question answering about a provided document or simple reasoning (like a lot of the standard benchmarks) seems to give a better signal than fact-based queries.