As I awoke this morning from uneasy dreams I found that Google had replaced my authenticator app with an anus drawn by Kurt Vonnegut
When you sync an authenticator to a service, they do a secret exchange. It's a shared secret. If that secret is exposed, then yes, someone else can provide TOTP 2FA codes to the service. However, the secret in the phone should be encrypted and unlocked via your phones auth mechanisms (face ID, fingerprint, what ever). That mechanism is tied to the physical phone, those secrets / keys are not backed up, and not restored.
The risk is not zero but its low.
Risk is not zero: