One of the issues with only following people you see in your federated time line is that you aren't growing your instance's federation. There is possible growth if you are not in the other person's federation as you might end up being introduced to their instance.

The community needs to figure out a way to aid discovery outside of the federation.