Noticing that people replying to other people I don't follow are showing up in my local timeline. So now I'm worrying: every time I have a conversation with someone, am I polluting everyone else's timeline with an unending series of replies?

That was one thing "the bird site" got right: you didn't see peoples' replies to people unless you followed both of them.