Built for my offline AI workstation. Maybe it fits yours too.

Repo: https://github.com/llamastash/llamastash
Release blog: https://deepu.tech/introducing-llamastash
Benchmarks: https://deepu.tech/benchmarking-llamastash

⭐ welcome on the repo. Bugs welcome on the issue tracker. Feedback welcome here.

#Rust #LlamaCpp #LocalAI #LocalLLM #OpenSource #FOSS

Install LlamaStash:

curl -fsSL https://llamastash.dev/install.sh | sh

or

irm https://llamastash.dev/install.ps1 | iex

or

brew install llamastash/llamastash/llamastash

or

yay -S llamastash

or

cargo install llamastash

Then run `llamastash init` and you're chatting with a local model in a few minutes. Linux, macOS, and Windows 11 on day one.

On benchmarks: LlamaStash spawns llama-server unmodified, so the wrapper better not add overhead. I measured it across AMD APU, Apple Silicon, and NVIDIA.

LlamaStash ≡ raw llama-server within ≤1% on every cell, across 4 model sizes and 2 metrics.

Ollama 0.24 is 41–72% slower decode on AMD APU. RAG prefill is catastrophic (4 min cold prefill on a 31B model). Numbers + methodology in the next post.

The architecture is the punchline.

One binary, three personas: TUI, CLI, daemon. Invoked differently, same code.

The daemon spawns the *unmodified* upstream llama-server. Nothing patched. Nothing forked. Bearer-token loopback HTTP between TUI/CLI and daemon, same transport on Linux, macOS, and Windows.

Same primitives in your shell as in the UI. Anything a person can do, an agent can do via --json.

What you get out of the box:

• llamastash init — first-run wizard that detects your hardware, installs llama-server, picks a GGUF that fits your VRAM, downloads it, and smoke-launches.
• Full TUI with vim-style nav, chat/embed/rerank tabs, in-TUI HuggingFace browser.
• CLI with --json as a stable agent contract and documented exit codes per failure class.
• Multi-model concurrency, daemon-on-demand.

Local LLMs sit in an awkward gap. Raw llama-server is fast but tedious. Ollama and LM Studio wrap it in friendlier shells but hide too much and pay a real performance cost.

I wanted a launcher that stays out of llama.cpp's way and treats agents as first-class users. That's LlamaStash.