Soatok DreamseekerDec 24

Honestly, the thing that will probably kill LLMs the hardest is someone writing a small language model that fits in JavaScript in a browser and hits comparable benchmarks.

Why bother with all those GPUs and energy usage if your Raspberri Pi could get comparable results?

Show threadSoatok DreamseekerDec 24

Is this possible? I dunno. I'm not specialized in this.

But if I wanted to fuck the GenAI bubble over and had the relevant background experience? This is what I'd explore.

Show threadTommyTorty10Dec 24
@soatok about a year ago, a bunch of friends were trying to do this. Various Chinese companies and universities had just released a bunch of relatively efficient models, and my friends ran them on phones and pi's with a wait of 1-5 minutes for each response. Imo, that's too long to be really competitive, but it's real close. Idk where things are now, but Id guess that it's only a matter of time until someone makes a decent model that can run entirely on the gpu of a phone nice and fast.
Show threadftg
@TommyTorty10 @soatok
If RAM just didn't shoot up in price, more and more phones would have had more and more RAM to run more and more capable models.
Along with ML accelerators in silicon.
Dec 24 at 1:28amWeb