I figured this out recently! Visit all of the Board Games shops and card game shops in your town and find a group that meets publicly and plays board games.
If you don’t like Board Games, go do an outdoor hobby like disc golf and join other groups.
The board game meet ups have been fantastic because you can just focus on the board game and you don’t have to get a bunch of anxiety trying to come up with small talk. It can just happen organically overtime at your comfort level.
I think you’re approaching with the wrong approach: first that there is a place that one can just go to where there’s people you’ll like, and second that meeting people you don’t like meat be avoided. Is people turn out to be awful for you, don’t meet them again, you can’t always know beforehand.
As others have implied, going somewhere where people with similar interests go can be a good start.