I finally got my #trunas server using the #gpu for #openwebui running the #deepseek coder v2. I got so close before with the system identifying the gpu and then not using it. I followed this guide and made only one change to put the open web ui portion into a custom app yaml so that it'd show up under the apps list, as doing docker run won't show up under the TruNas apps. It might with portainer or something else, but I don't have that.

Here's the guide that I followed: https://burakberk.dev/deploying-ollama-open-webui-self-hosted/

It wound up just being two docker commands. Now I can start cleaning up the previous 5 attempts lol.

Last night I was up until 2AM trying to get #trunas #amd drivers installed inside of a #docker #container so that #ollama would actually use the #gpu. I was so close. It sees the gpu, it sees it has 16GB of ram, then it uses the #cpu.

Trunas locks down the file system at the root level, so if you want to do much of anything, you have to do it inside of a container. So I made a container for the #rocm drivers, which btw comes to like 40GB in size.

It's detecting, but I don't know if the ollama container has some missing commands, ie rocm or rocm-info, that it may need.

Another alternative is one I don't really want, and that's to install either #debian or windows as a VM - windows because I did a test on the application that runs locally in windows on this machine before and it was super fast. It isn't ideal from RAM usage, but I may be able to run the models more easily with the #windows drivers than the #linux ones.

But anyway, last night was too much of #onemoreturn for a weeknight.

As your #homelab grows, we sometimes get bad about documenting what we did and how it's all working.

And then, sitting down in draw.io to diagram it all out can be a chore.

I think many people already use git repos for their homelab config (and if you're not, you should give it a try), so my pro-tip is:
diagram your setup with #json.

It's concise enough to be useful and quick to write. You can version control it easily. The schema is up to you. Make it make sense for you.

"But...that's not a diagram...?", i hear you asking.

And I reply to you: : "Yes it is!"
https://jsoncrack.com/editor

#selfhosting #selfhost #trunas #synology #SynologyNAS #smarthome

My #synology NAS has, honestly, been bulletproof for several years. It's getting long in the tooth, but still chugs away great.

That said, I'm at the point where, for philosophical reasons, I'm probably going to move to a #trunas #nextcloud combo.

In preparation for that upgrade, this morning I'm in the middle of doing a Volume/iSCSI LUN refactor on my #SynologyNAS for better data organization, and I'm in that nervous middle phase where all the data only exists on my external hard drives attached to my computer, while the storage pool gets an overhaul...

https://i.giphy.com/media/v1.Y2lkPTc5MGI3NjExa2trOXlkOHk5OTZobGkzMzY3b2htYTltNndvbTdqZGQzZjFpNnd1OSZlcD12MV9pbnRlcm5hbF9naWZfYnlfaWQmY3Q9Zw/f9ZLRWMBjKAbe07PtH/giphy.gif

#homelab #selfhost #selfhosting #nas