lazykiq is a TUI for sidekiq, showing sidekiq activity in the terminal.

You can view jobs with graphs and see details + execution metrics. You can also filter, retry, schedule jobs and more.

Dmytro Shteflyuk (@kpumuk) made lazykiq using Bubble Tea by @charmcli and is Terminal Tool of the Week! ⭐️

https://terminaltrove.com/lazykiq/

A few days ago, a client’s data center (well, actually a server room) "vanished" overnight. My monitoring showed that all devices were unreachable. Not even the ISP routers responded, so I assumed a sudden connectivity drop. The strange part? Not even via 4G.

I then suspected a power failure, but the UPS should have sent an alert.

The office was closed for the holidays, but I contacted the IT manager anyway. He was home sick with a serious family issue, but he got moving.

To make a long story short: the company deals in gold and precious metals. They have an underground bunker with two-meter thick walls. They were targeted by a professional gang. They used a tactic seen in similar hits: they identify the main power line, tamper with it at night, and send a massive voltage spike through it.

The goal is to fry all alarm and surveillance systems. Even if battery-backed, they rarely survive a surge like that. Thieves count on the fact that during holidays, owners are away and fried systems can't send alerts. Monitoring companies often have reduced staff and might not notice the "silence" immediately.

That is exactly what happened here. But there is a "but": they didn't account for my Uptime Kuma instance monitoring their MikroTik router, installed just weeks ago. Since it is an external check, it flagged the lack of response from all IPs without needing an internal alert to be triggered from the inside.

The team rushed to the site and found the mess. Luckily, they found an emergency electrical crew to bypass the damage and restore the cameras and alarms. They swapped the fried server UPS with a spare and everything came back up.

The police warned that the chances of the crew returning the next night to "finish" the job were high, though seeing the systems back online would likely make them move on. They also warned that thieves sometimes break in just to destroy servers to wipe any video evidence.

Nothing happened in the end. But in the meantime, I had to sync all their data off-site (thankfully they have dual 1Gbps FTTH), set up an emergency cluster, and ensure everything was redundant.

Never rely only on internal monitoring. Never.

#IT #SysAdmin #HorrorStories #ITHorrorStories #Monitoring

I feel like there’s a certain type of personality that is drawn to Linux because they like solving software problems and Linux gives them lots of software problems to solve

but eventually they get too good at solving Linux problems and get bored, so they switch to Arch - which delivers new software problems to their computer all the time. but eventually they get too good at solving Arch problems so they switch to NixOS, which is an infinite wellspring of software problems to solve

💔 America’s betrayal of #Ukraine - selling out a nation bleeding for freedom - will echo as a moral failure long after Trump.

The world sees this shame. History will judge it harshly

When I keep saying that being on #streaming platforms is a pointless exercise, I get sometimes asked for numbers, or proofs.

Here you go. A single song of mine was played a MILLION times in 3 months, in 2022 (and, in later months, another 750k times), with a net gain of £33.

A million plays on streaming pays for a pizza night out for me and @silviamaggi

Took me FOUR sales on Bandcamp to reach the same amount.

So, ask me again, why quit giving out music to the techbros?

#IndependentMusic

After listening to about a dozen first-hand accounts, I’ve published what I know so far about the RubyGems takeover.

https://joel.drapper.me/p/rubygems-takeover/