I was always told it’s a bit rude to use a language that not everyone present understands, since you’re basically excluding people from the conversation. Your example seems a bit silly though.

Talking to a person excludes all others by default. If I’m talking to you I’m not talking to the guy behind me. What does the guy behind me care what language we’re using? And why should I care about the one I’m NOT talking to.

Nah if it’s the workplace, I treat it like talking at a dinner table. I’m definitely using the common language and not excluding the coworkers around me. I’m being thoughtful towards the people around me.

I also get that speaking in a language your company doesn’t understand could make them uncomfortable. I speak a 2nd language. The very few times that I’ve used it at work, it was because a certain coworker switched to talk shit about another coworker to me. So yes, people absolutely do this to talk shit. It’s not paranoia, it happens.