Offhand thought - I wonder if part of why sci fi and fantasy are so huge now is that a lot of the genres that would've scratched the same escapist itch are now hard to do inoffensively. Like a lot of 19th century adventure literature is oriented around white adventurers visiting strange cultures, romancing hot locals, stealing stuff, and getting away. Setting it among aliens or elves or whatever allows the same story, theoretically minus the dehumanizing.
@ZachWeinersmith I've always thought the power of SF/F as a genre lies in how it allows an author to explore their topic of choice with as much or as little existing context as they choose. Ditching the baggage of the real world can make for much more interesting storytelling in the right hands!
@ZachWeinersmith But also you could argue that the 19th century was the aberration here, right? Throughout history people have used the fantastic to tell their stories (eg. the Pratchett argument that Fantasy is the oldest genre)