Peter GrandstaffJun 1
I've been getting a lot of questions about my local LLM setup. Here are all the details, config files, and model choices. Anyone else having good success with Qwen3.6 or something else?
https://www.petergrandstaff.com/writing/2026-05-local-llm-setup/
Show threadPaul ZuradzkiJun 5
@grandstaff very good write up. Thanks for sharing.
Show threadPaul ZuradzkiJun 5
@grandstaff I got an AMD box. I’ve enjoyed local AI for private data processing on simpler tasks. On my todo list is to 1) set up sandboxed pi, 2) ollama->llama.cpp, 3) try MTP. My local AI use has been underwhelming for coding tasks compared to SOTA. It’s slow using opencode, but I suspect to get more mileage after the steps above. That said, nice to, say, let it loose on an Obsidian file tree. Also experimenting with self-hosted document processing (tagging, cleanup, retrieval).
Show threadPeter Grandstaff
@paulzuradzki I will also say pi hasn’t done anything scary on me yet. My sandboxing is overly paranoid, and now with some use it feels like overkill. But, you gotta do whatever makes you comfortable letting it just run!
Jun 5 at 10:01pmMastodon for iOS
Show threadPaul ZuradzkiJun 7
@grandstaff I’m on the paranoid side too, but I still often run Claude Code auto mode outside a container or VM. On pi, the sandboxing extension looks more like protection against accidental data loss than against exfiltration. With Claude, I mostly care about limiting what Anthropic sees; I trust the model not to do something irreversible. With local AI + pi, I’m a bit flipped: worry more about weaker model doing some big blast-radius thing or getting tricked after picking up bad docs.