On multiple occasions a moderator has reached out to me and informed me an account existed purely to go through my comments and downvote me. I assume this is a common occurance and we could combat it better if we could see the votes.

I worry though that public votes will enable unstable people to stalk you. Vote the wrong way? Why, I’ll just create some bot accounts to constantly downvote you, and I’ll harass you with replies to every comment you make, reminding everyone that you vote against the grain and shouldn’t be trusted!