The bedtime objection makes perfect sense. Cats evolved as hunters but not pack hunters, so they are careful to “share” territory by noticing when other cats are present, and avoiding the territory at that time. (Besides visual observation, they also smell the area and tell whether a foreign cat has rubbed up against the surfaces recently, as they can measure time by how much a scent has faded.). It’s likely that the nighttime is the cat’s “alone time” in the living room, and the humans are committing a faux pas by not appropriately sharing the territory.