Soatok DreamseekerDec 24

Honestly, the thing that will probably kill LLMs the hardest is someone writing a small language model that fits in JavaScript in a browser and hits comparable benchmarks.

Why bother with all those GPUs and energy usage if your Raspberri Pi could get comparable results?

Show threadkasia  Dec 24
@soatok fine-tuning and distilling LLMs into small models that can run in very limited environments is already a thing, but I'm pretty sure that building tiny language models for very specific purposes is still relatively underexplored.

I'm not into LLMs though. And I have barely idea whether this makes much sense.
Show threadbluestarultor

@nullenvk @soatok It's totally buried now, but a couple years ago all the big players were shitting their pants because small language models were outperforming their LLMs (in general tasks, IIRC). Not even distilled; it just didn't take much. They all have their quiet small language models on the side now because of that. SLMs significantly outperform LLMs in their narrower areas, particularly in science, IIRC.

It's not a matter of whether it's possible; it's that pretending LLMs are the end-all-be-all is worth a LOT of money.

Dec 24 at 2:02amWeb