Offhand thought - I wonder if part of why sci fi and fantasy are so huge now is that a lot of the genres that would've scratched the same escapist itch are now hard to do inoffensively. Like a lot of 19th century adventure literature is oriented around white adventurers visiting strange cultures, romancing hot locals, stealing stuff, and getting away. Setting it among aliens or elves or whatever allows the same story, theoretically minus the dehumanizing.
@ZachWeinersmith makes sense, but I'd add also that now those "otherness" can't be practically "othered" anyway.
Exotic places have been explored and colonized IRL, and thus normalized, people have entered (under cohercion) the Commonwealth and so on.
It's not just more polite, it would make much less sense even to the most ignorant and "anti-woke" Odysseus or Crusoe to explore whatever sea, island, or ocean.
Also, maybe much of the stories before the modern "realistic" novel have always been fantasy and sci-fi, in some way. Other genres are the actual exception. But mythology muddies the water...