I’ve had an IT Career for about as long as you. I’ve had 2 memorable backup failures and got real lucky both times.

The first was a ransomware incident, and the onsite backup was not hit, but it was corrupt. Thankfully, the client had been using a 3-2-1 strategy, and the off-site one was fine.

The second was a situation where a failed update rendered a client’s RDS unbootable. This time, they didn’t have an on-site backup and the off-site one was corrupt. This time I happened to get immensely lucky in that there was no real data on that RDS, so I was able to spin up a fresh one, and install their LOB app and all was good.

We now test that all backups are stable every 6 months.