Michael LabbéI've been experimenting with programming while fully offline. Using local LLMs to ask questions has prevented me from needing to search for solutions where I'd usually go back online. Full music albums copied to the working computer's local storage, only come up for air when an album finishes. I can usually go a couple albums before needing to go online. Git fetch all repos and unplug.
Sylvain Meunier@michaellabbe Which LLM do you use ? How big is it on disk ?
@meuns llama3.2, 2GB via ollama. So far my questions are just double checks on whether something is UB, or things like that. Light usage.
@michaellabbe Sometimes I want to check CPP shenanigans without any good internet connection. It seems this is perfect for my use case.
@meuns That's an area it should do well in. Depending on how much VRAM you have, you could look at qwen3:30b-a3b for mixture of experts, or devstral 24b as higher end options to experiment with if you need more.
@michaellabbe I have some kind of AMD APU with 32GB of VRAM. I'm planning to develop things during vacation on my very little PC.
@meuns Probably the best thing to do in an environment like that is have a system prompt that consistently requests concise answers. But also, I think the productivity secret in this setup isn't the LLM -- it's playing an album at a time, and not coming up for air until it's done. Very boring tech-wise! https://gist.github.com/mlabbe/f6799275140976b42ea9073f32f066a3
Amin Mesbah@michaellabbe I think of this as the John Carmack approach to productivity: https://web.archive.org/web/20220911175639/http://bookofhook.blogspot.com/2013/03/smart-guy-productivity-pitfalls.html?_=1363670119317
> I remember Carmack talking about productivity measurement. While working he would play a CD, and if he was not being productive, he'd pause the CD player.
> He'd then measure his output for the day by how many times he played the CD (or something like that -- maybe it was how far he got down into his CD stack).
@aminom Great callback. Modern Internet is way more distracting than then, too, so the practice could be more valuable. Aim here is to build the focus muscle and see where we're at at 15, 30, 60 days.