TW: goth girls, hot nuns, anti-christ, nutting
Incident report.
As you may have noticed, Stereophonica has been moved to a new server and then... it died for five days. Here's exactly what happened.
I bought components for this box in early November last year, the same day I was to attend a Halloween party. Not alone, of course. There was a hot goth girl dressed as a nun with me. I would totally upload her photo, but Stereophonica currently doesn't have media uploads working, this will be fixed soon.
Anyhow, after she put on the costume and the make-up of a possessed nun with a red inverted cross on her forehead, she asked if she could build this server for me. She's a gamer, so that's not really unexpected. I agreed and went to grab some cofe and ready myself for the party too. This was the short period this server was unattended and when the Anti-Christ must've possessed the nun and, consequently, the server. Also, she failed to properly tighten one of the nuts holding the CPU heatsink, which led to the following.
The server was tested, including prolonged full-load tests, all was good. However, that was during winter, as temperatures here are just a few degrees above freezing in November. And then my life went slightly sideways, so Pleroma migration was postponed.
Fast forward to two weeks ago, I finally gathered myself together and decided to move this instance to better hardware, successfully done overnight or so. All was good for a few days, until we were hit by summer and temperatures over +30C, which I totally loved, but the hardware didn't. Since the heatsink wasn't tightly held against the CPU on one of its corners, this caused small overheating, except it wasn't caught by sensors. Apparently, AMD did put temperature sensors somewhere else on the die, because the logs didn't show CPU going above +72C ever. Overheating caused issues with memory access, making software randomly segfault all over the place, usually after about an hour of operation. Segfaults also correlated with memory load, which sent me down investigating the wrong stuff, namely bad memory. Only after fully disassembling and reassembling the box I noticed that one of the nuts on the socket was looser than it ought to be.
Lessons have been learned: never allow possessed goth nuns to touch your future Pleroma box.
-
@newt
