That’s… the wrong way to look at it.
Systems are often resiliant in the light of individual variation. Discarding voting entirely because of the actions of a few is like seeing the outliers in people clicking where they think a country is on a map. Sure, you’ll see a lot of dumb guesses that are radically incorrect, but the majority of clicks tend to be on the country.
In the same way, the voting tends to generally work, for a given understanding that voting was always theoretically (from reddit days) supposed to be upvoting good contributions and downvoting spam./trolls/etc, but voting is also or even more about what people agree with. So as long as you realize that’s what’s actually happening, voting is generally accurate enough.
Of course it’s infuriating when people agree with things that are wrong, but that is a wholly different issue.