Is it rude to interrogate downvoters?

I'm very curious about the reasoning different people use when deciding to downvote a post or comment. Often, when something gets "heavily" downvot…