Taking #selfhosting to the next level 😂 Built a small server for locally hosted LLM, to be used as a brain for my #openclaw assistants. It might be not the smartest one amongst OpenAI and Anthropic models, but hey, self-hosting is not about getting the best experience, it's about freedom! So I challenged myself to squeeze as much as possible out of this setup with system prompts and instructions.
1/2
Same box is running all my self-hosted stack, which today includes: #mastodon, #NextCloud, #bluesky PDS, #Matrix (also serves as main channel in OpenClaw gateway), homepage web-server, and a backend for a few non-AI Matrix-bots. In total 15 containers. That's not including a Raspberry Pi 5 running #homeassistant and #mqtt broker.
2/2
Qwen3.6 27B. I still set things up and have Claude from my GH Pro subscription on a backup, but it looks very promising!
Oh, I only use the android app and web UI on PC to steer HA remotely. A couple of light bulbs and self-made weather station not really worth it to loop in any AI at this point.
Though I asked my OpenClaw bot to turn on the lights over HA API once just for fun, and it took it around 15-20 seconds too, but I think that's because it had to "think", find appropriate API endpoint, fire a request etc. If I'd write a skill with clear instructions it should've done it much faster.
@m3211n Thanks. We have built a decade long habit of using Alexa to run lights and stuff, and now I'm always looking to trade all the Amazon stuff for local, so that's my go-to benchmark.
Right now it looks like I'd be better served by adding more regexp patterns to the HA sentence parser instead of trying to optimize a giant LLM that's clearly unsuited to the task.
m3211n.dev
Lobster
John Deters