Aurynn ShawGood DevOps practice is to build your systems with failure in mind.
What you build *will* break. That's inevitable. So what you build should be built to be quickly recoverable.
You should have good disaster recovery plans.
You should have backups.
Your deployment should be automated, and use infrastructure-as-code extensively.
Neale