Paco HopeDec 9

I'm going to admit that I am doing something immature in my #homelab and I'm looking for opinions. I've got multiple #XCPng hosts, all using local storage. I have no NFS or iSCSI storage. That's kinda silly. Shared storage is super useful and I'm literally not using it.

Unless I go to some serious effort to make a high-performance SAN, I expect network storage performance to be so-so for VM storage, but maybe I'm too pessimistic. I currently only have copper gigabit in the rack. No fiber, no 2.5G copper or anything like that. I'm not sure if that's going to be viable for NFS or iSCSI.

I could dedicate a host to running TrueNAS Core with a bunch of storage. But what has always bugged me about this is that my storage host becomes a single point of failure for all the compute nodes. #TrueNAS is super reliable but everything has to reboot once in a while, and these stupid enterprise-grade servers take anywhere from 4-8 minutes to boot. If I had a single storage node, and I needed to reboot it for an OS upgrade, everything would hang for a while. That's no good. Not updating the OS on the storage system is also not good.

So what am I supposed to be doing for shared storage on a #Xen cluster? How do I avoid a storage host becoming a single point of failure? How do you update and reboot a storage node, without disrupting everything that depends on it?

#selfhosting #san #storage

Show threadNickDec 9
@paco How is local storage configured?
Show threadPaco HopeDec 9
@nicholasburns Each node has 5 drives in a ZFS pool. So each has about 7-10Tb of local storage. I just brought in a new 20Tb node, which I thought I might turn into a storage node. But, again, the 5 minute reboot duration is a real drag.
Show threadNickDec 9

@paco SATA or SAS drives? If the latter, that's a pretty resilient and performant local storage solution, and not 'immature' at all. I might be more worried about disaster recovery depending on how that looks.

I also think the point @unixorn makes about distinguising between storage location of virtual system disks versus non-system disks is important.

Next questions would be about what kind of non-system disk data specifically, you think you could be storing more effectively and why. Not to mention thorough benchmarking of current local storage to even coherently compare to non-local options. Otherwise a storage network for the sake of a storage network is just that.

Show threadPaco HopeDec 9

@nicholasburns Most of it is SAS. But where local storage bit me was in migrating from one host to another as I upgraded XCP 8.2.1 to 8.3. When you have shared storage, moving a node from one compute to another is trivial. When I have only local storage, migrating VMs has been really difficult. It's either time-consuming (20-120 minutes to move a big VM) or even impossible (version-to-version migration on XCP has been difficult with only local storage).

#homelab #selfhosting

@unixorn

Show threadClayton O'Neill
@paco @nicholasburns @unixorn Just for what it’s worth 10G networking makes moving live vms a lot faster and NAS based storage more viable. If you’re looking for a reason to upgrade the network 😂
Dec 10 at 2:34amWeb
Show threadUnixorn - 90% Snark by weightDec 13

@clayton_oneill @paco @nicholasburns

You're not helping me not buy new shiny.