I’m so glad (or not?) that I’m not alone here. Every couple weeks the remote just refuses to connect to the Apple TV until I reboot the box.

https://www.tomsguide.com/news/theres-an-apple-tv-4k-remote-bug-out-there-and-ive-got-it-too

There's an Apple TV 4K remote bug out there — and I've got it too

The Apple TV remote lost signal bug is real

Tom's Guide
@birchtree
This exact thing plus a whole lot of audio/soundbar issues has been the norm for me for the last year. We've been trying out the Shield TV and it definitely lacks the polish and speed of tvOS but the remote stays connected and we don't randomly lose audio while watching movies anymore.