Michael LabbéI've been experimenting with programming while fully offline. Using local LLMs to ask questions has prevented me from needing to search for solutions where I'd usually go back online. Full music albums copied to the working computer's local storage, only come up for air when an album finishes. I can usually go a couple albums before needing to go online. Git fetch all repos and unplug.
Sylvain Meunier@michaellabbe Which LLM do you use ? How big is it on disk ?
@meuns llama3.2, 2GB via ollama. So far my questions are just double checks on whether something is UB, or things like that. Light usage.
@michaellabbe Sometimes I want to check CPP shenanigans without any good internet connection. It seems this is perfect for my use case.
@meuns That's an area it should do well in. Depending on how much VRAM you have, you could look at qwen3:30b-a3b for mixture of experts, or devstral 24b as higher end options to experiment with if you need more.
@michaellabbe I have some kind of AMD APU with 32GB of VRAM. I'm planning to develop things during vacation on my very little PC.
@meuns Probably the best thing to do in an environment like that is have a system prompt that consistently requests concise answers. But also, I think the productivity secret in this setup isn't the LLM -- it's playing an album at a time, and not coming up for air until it's done. Very boring tech-wise! https://gist.github.com/mlabbe/f6799275140976b42ea9073f32f066a3
