Mob mentality aka Circlejerk effect.

I think the more downvotes something has, the more it causes people to read it specifically looking for the reason people downvoted, which in turn makes them more likely to downvote it than if they encountered the same thing with no votes on it. It happens with upvotes too, and it makes vote distributions tend to swing out to the poles more than they would if votes were solely “on the merits”, so to speak.