Not everyone thinks rationally about certain subjects. I'm sure I've ranted on a topic before only to later realize my comment was poorly thought out. This is what occurred to me. People see a subject and comment based on a variety of reasons. Some want give an opposing view. Some want to just be a cheerleader for the original topic. Some see the trend and follow suit. Some try to logically (or sometimes illogically) debate the issues brought up by the original topic or to others' comments.
The problem is that good comments can sometimes be voted down simply because the comment expresses views that another user doesn't want to read. It's not that they think the commenter was wrong as much as it is sticking their heads in the sand. This isn't always conscious either. Emotion frequently plays into commentary systems and skews perspective. I know I have voted down certain comments before that perhaps I shouldn't have.
There are plenty of comments that fully deserve to be voted up or down. The system itself is not broke. As frustrating as it can be at times it's actually a pretty decent mirror of human behavior. In person we might not make the same comments to each other, but we'd be thinking them.