Avoiding conversations about race doesn't remove it from the relationship. It removes the couple's ability to talk about something already there.
Silence trains both partners that avoidance is safer. The problem isn't any single moment you let slide. It's the habit, until the gap between what you show and what you feel becomes structural.
The couples who succeed aren't the ones who never stumble. They're the ones who build a shared vocabulary before they need it.