I've been active on a big Fediverse server for a few years now. I like it here, and I enjoy my experience more in some ways *because* it's a big server.
Some people like it better on small servers. That's good too! I use smaller, more specialized forums in other contexts (though most of my favorites aren't Fediverse-enabled).
If you want more folks on small servers, I recommend inviting them to your favorites. I'm not sure "don't use a big Fediverse server" by itself is very helpful.