Mitex LeoFeb 24

I’m on a vacation, away from my homelab for almost 13 days now. On Feb 16, I suddenly lost all connection. No SSH, no monitoring, nothing. Most likely a power outage. My UPS can hold for ~40 minutes. After that, it’s game over.

(1/5)

#mastodon #fediverse #homelab #selfhosting #horror #mastoadmin

Show threadMitex LeoFeb 24

I live alone. The room was locked. There was literally no one who could check the server physically. I called my ISP thinking maybe a fiber cut. They didn't do anything despite I'm mentioning that last time one of their employee unplugged my cable and left it that way while working for another customer. So all I could do was wait… and wait.

(2/5)

Show threadMitex LeoFeb 24

Those 10 days were frustrating. You know something is wrong, you know it’s probably simple, but you’re powerless. This is the downside of running everything yourself from home.

(3/5)

Show threadMitex Leo

What saved me is backups. I had the database and files safely stored in S3. Without that, this would’ve been a full wipe. Today I finally got back, restored everything to a Contabo VPS, and brought the instance back online.

(4/5)

Feb 24 at 8:07pmWeb
Show threadMitex LeoFeb 24

Lesson: Backups and Redundancy aren't optional

#matodon #backups #linux #homelab #selfhosting

Show threadLasagneFeb 24

@ml

Everything can be a datacenter

Show threadMitex LeoFeb 24
@lasagne I didn't have enough resources to make the setup more redundant 😔