At #work today I had an unexpectedly pleasant experience. With #kubernetes of all things.

I'm playing around with a sandpit #RKE2 cluster that I'm using to test how things are going to work when we need to deploy an actual working environment, and the matter of persistent storage came up.

A bit of poking around and I discovered #Longhorn. The requirements were trivial - all the required software was already present, just one service needed to be started. And so I installed it via #helm with just a handful of lines in the config.

And it just works. A fully distributed, clustered, read-write-many capable storage subsystem for Kubernetes, and it took me less than the time it is taking to write this to get it up and running.

I fired up a deployment that required multiple RWM PV's and ... it all just worked. I could even go into a management panel and see how the shards were distributed, and how busy everything was.

For a back-end all it needed was a filesystem on each node of the cluster, and even that could be managed with #LVM so it could be expanded at need.

Compared to the old in-tree VMWare CSI operator, this is a dream come true.

#Linux #SysadminLife

@rdm Pull the plug on a storage node and see how Longhorn responds.

Maybe it was just the very unbalanced nature of my homeprod cluster but when the biggest node fell over it took Longhorn out.

@zrail
That'll be the first thing I do on Monday!
@rdm @zrail And also try a Longhorn upgrade.

@rds @rdm @zrail I’ve not had problems with upgrades. But not sure read-write-many is longhorns strength, that’s just NFS right? There are better ways to serve NFS.

It’s great for RWO though.

@lloydw @rds @zrail
It is sort of NFS with a distributed data store, so not really? And from a K8s perspective, it is one of the easier ways of configuring it.
@rdm @rds @zrail replicated storage yeah, I use longhorn a lot for RWO. But still just use an external NAS or cloud NFS like EFS for RWM.