Luigi 💚
Hello #homelab and #selfhosted community! I want some basic orchestration for my homelab and in another toot some users told me they`re using #nomad.
How are you using nomad? My idea was using a single VM for server + client agent, but there's minimum resources about this deployment, I don't need enterprise high availibity as I'm running a homelab, but maybe I can run a VM for host and another for agent? So I have better allocated resources?
Jul 11, 2023 at 1:53amWeb
Show threadJimmy B. Jul 11, 2023
@luigi I almost used Nomad but just ended up going with Docker Swarm.
Show threadTupcakesJul 11, 2023
@luigi I’ve got 5 nomad nodes (2 old laptops and 3 raspberry pi 3s). Honestly it’s been really nice. I use gluster for the storage and I have consul running on all the nomad nodes. My setup is a bit overkill for what I’m doing, but you can certainly run nomad like you mentioned. You can even run it on a single node if you want to, but I wouldn’t do that in a production environment of course. :)
Show threadLuigi 💚Jul 11, 2023
@tupcakes Cool! Do you monitor nomad's resource usage from the server perspective? My homelab have some "production" services but it's not anything critical that needs HA, so I'm thinking running single node cluster just to save some resources of the computer. I have 2 VM one with 8GB and 6GB of RAM but I can spin up one VM with 2 or 3 GB of RAM and only install the agents on those 2 reamining, or nomad will need more resources? Thanks in advance!
Show threadItzTrainJul 11, 2023
@luigi @mmeier is a nomad user in #homelab! He's a great resource to ask!!
Show threadLuigi 💚Jul 11, 2023
@train @mmeier Thanks for tagging him! Hope to hear about how nomad is used by you @mmeier
Show threadMichaelJul 11, 2023

@luigi @train I've had Nomad running in two different configurations. The first one was what you describe, having one VM for the server agent (I used the same VM for Consul and Vault agents too), and then several additional VMs, each as a Nomad worker with a client agent.

Today, I'm running my Homelab (mostly) on a fleet of 13 Raspberry Pi's, including both the server and client agents for Nomad.

Show threadMichaelJul 11, 2023

@luigi @train But this is not necessary. I do it because I want high availability, mostly so that I can easily update each host, without having to take down the entire Homelab.

But Nomad has actually improved a lot recently for more resource-constgrained envs. Clients can now survive for a bit when there's no server reachable. And, more importantly, you can now run Nomad agents in both server and client mode at the same time, which is now officially supported.

2/3

Show threadMichaelJul 11, 2023

@luigi @train But, I must admit that I never did so. I initially planned to, to end up with three server agents, but I found enough other things to run on three "control plane" Pis that I decided to dedicate three Pis to the Control plane.

So now I have three Nomad server agents, and eight workers in my cluster.

But it also ran perfectly fine back when I only had one VM with the server, and one with the client agent.

If you want to keep it simple, I'd advise to just try the combined deploy.

Show threadLuigi 💚Jul 12, 2023
@mmeier @train thanks very much, helped me a lot and based on your experience and tips I'll start to install and migrate my services to nomad at my next free time :) I don't know if I'll keep this idea but with the server/client agent at same server thing, I'll try to get a hybrid of simplicity and high availability for fun using one small VM for server (maybe consul and vault aswell?) And one VM for server/agent where my services will stand.
Show threadMichaelJul 12, 2023

@luigi @train I'd recommend trying Consul and Vault as well! I found both pretty useful.

Although these days, you don't necessarily need them, as Nomad now has both secrets and service discovery build-in, which it didn't have when I set up my Homelab cluster.

Show threadLuigi 💚Jul 12, 2023
@mmeier @train I've already use vault in my work and it's pretty useful. I'm thinking on adding as well, about Consul from what I see I don't know If really is needed but I will try to implement as well just for fun :D yolo
Show threadaltoJul 11, 2023
@luigi you don't need much resources to run a #nomad #homelab client/server node. The system requirements stated on the nomad site are for massive deployments. I run client/server nodes on my homelab hosts which includes 2 servers and my desktop computer. I bet you could colocate them on an rpi without issues.
Show threadLuigi 💚Jul 12, 2023
@alto Thanks for your input! I'm curious about your desktop being part of your setup. Do you use to kinda fast test things? Or it runs some services and when you turn off the services goes to another host?
Show threadaltoJul 12, 2023
@luigi the desktop host only runs a few non-critical services which are brought up on boot. It's not really recommended to run a server node on a host that reboots more often, but I haven't really had any issues, the other two nodes take over the server responsibilities when the desktop host shuts down
Show threadLuigi 💚Jul 12, 2023
@alto that's cool to know, I'm still to read how nomad really works so I thought that we have like a k8s thing that the pods starts in another place if had the space.
Show threadaltoJul 12, 2023
@luigi you can do that if you want, I just don't have the jobs on my desktop host set up like that.
Show threadMakoto Itou   Jul 11, 2023
@luigi following. I have been curious about nomad as well…
Show threadLuigi 💚Jul 12, 2023
@makoto Hope this thread gave you some helpful thoughts, at my profile I'll keep updated with my progress as well :)