So it appears like, for the last ~months, my MTU configuration was REALLY wrong

Hinted by the immich longhorn replica not rebuilding, but I also didn't know, the extreme slowness of any service using a db cluster where the master node wasn't in the same region

I had put the slowness on the shoulders of packets hopping a lot between regions, but it turns out, it was just db requests maxing past the configured MTU value, silently dropping

Now that BOTH the wireguard and flannel MTU values are set properly, everything is so damn snappy

This feels like new skin

#homelab #selfhosted #selfhosting #wireguard #mtu #vpn #mesh #longhorn #immich #flannel #devops #linux #opensource #networking

For the last ~6 months, my immich Longhorn PVC wouldn't rebuild replicas across regions, and timeout instead

Today, I figured I had misplaced my MTU configuration for the Wireguard network under k3s...

So some packets were getting dropped silently...

Woops

#kubernetes #k3s #longhorn #network #networking #wireguard #wg #mesh #homelab #selfhosted #selfhosting #mtu

Using #django and #longhorn rwx volumes: when I run collect static and put static on an rwx volume it tries to write some 900 files in short succession and this kills the rwx volumes.
Also when creating folders via migrations the same happens. I get I/O errors.
I solved that with an RWX Django storage Class that allows for retries. One retry is enough and it works.

So am I doing it wrong?

PS: you could argue that static files should belong in an emptyDir. The underlying issue gives me headaches though

#kubernetes

Show scheduled on Sept. 11 has #SexPistols making explosive return to the #Longhorn Ballroom in #Dallas for first time since 1978, this time sans Johnny Rotten

https://www.dallasnews.com/arts-entertainment/music/2026/03/11/sex-pistols-return-to-longhorn-ballroom-dallas/

I've been a little rough and irresponsible with my #baremetal #Kubernetes cluster, especially when it comes to randomly rebooting nodes. Today I fixed that.

I'm running a bunch of somewhat delicate workloads, including database clusters with CSIs like #Longhorn and #OpenEBS. Checking if everything is in working order has been demanding task and often something I've skipped before rebooting or upgrading nodes - occasionally with horrific results.

Last night I finally took the time and wrote a pretty thorough script that checks that everything is working and healthy, before politely cordoning off a node, draining it and applying upgrades.

I felt so confident today that I tested it by running this new safe upgrade script for all the nodes in the cluster - and it worked! All nodes are now fully upgraded and running kernel 6.12.73 on Debian 13.

This also fixes the outstanding issue caused by #Hetzner no longer supporting obtaining IP addresses through DHCP.

#Linux #MSTDNDK #K8s

I accidentally had a QLC drive (that's the really shit kind) in an nvme raid array hosting a Longhorn cluster. Could not understand why it would regularly shit itself for longer than I'd care to admit. It didn't help that Longhorn 1.11.0 has a memory leak and OOMs were triggering replica rebuilds on a weekly basis.

#selfthosted #k8s #longhorn

Es gibt scheinbar keine einfache storage Lösung für docker swarm, die, ähnlich wie longhorn, replizierten storage zur Verfügung stellt...

#docker #kubernetes #dockerswarm #Longhorn

Da hatten ziemlich viele Menschen Bandshirts an am #BandshirtFriday auf dem #konzert von #tarja #tarjaturunen im #Longhorn in #stuttgart 🇫🇮🎤🎶🎸🎹🎸🥁❤️
Tarja hat übrigens Gesang in #karlsruhe an der #hfm #hochschulefurmusik "gelernt"…

https://de.wikipedia.org/wiki/Tarja_Turunen?wprov=sfti1#Entwicklung

We bought our little farm 7 years ago and there were various artifacts on the site. This one has been given a sacred space on our back porch. I am always thankful for animals that give sustenance (we get eggs and milk from ours) or companionship.

ART
https://fineartamerica.com/featured/rich-blue-longhorn-cow-skull-art-sharon-cummings.html

#cow #cows #cattle #bovine #beef #ranch #ranching #farm #farming #farmhouse #animal #animals #rustic #country #countryhumans #countrywestern #SharonCummingsArt #homestead #homesteading #longhorns #longhorn

What if #Microsoft prevented #Longhorn's collapse in 2003/2004 by merging Windows Server 2003 & XP's source codes & designs and released Longhorn/Longbridge as a gateway to Windows Vista & Windows 7?
It would serve 2004-2007 a good purpose.

Crafted on Copilot.