@n_dimension also, I think if we trim down to quality datasets - like Wikipedia and open-source books, I think we can build a smaller model that runs on lower spec hardware.

I run Qwen-3/Jan-code models on my RTX 2060 - no sweat for inference and I can use it for 80-90% of my work. Its like having an interactive encyclopedia offline. I love it.

Specific models for specific use-cases/communities might also be a good idea. Like an agri-trained llm for agriculture

@mahadevank

Google has just released a super tight, great local #LLM I had not had a chance to look at it yet

I really like your agri model idea.

I was thinking a basic medical (nurse level) one for the third world/post-collapse.

@n_dimension @mahadevank I recall there was some research a while back which showed that domain specific fine tuning really did not work well.

There was attempts at training astronomy specific models, and while they outperformed models of a similar size at questions like "describe the lightcurve of binary star mergers" they suffered from much higher hallucination rates, and performed worse at generalising outside of the specific documents they were fine tuned on.

Now admittedly, this was back in the Llama2 days so maybe "modern" architectures would behave differently. But it seems that a broad dataset is necessary for generalising, even within a specific domain