If taking care of yourself means letting someone down, then let someone down. Always put your own physical and mental health first, and don't ever fucking apologize for it.