I think that's often (but not always) the case. That said, I don't know how to convince those who think otherwise. One person's weakly argued comment might be someone else's disagreement. Compare with the Political Detox Week experiment: there was quite a bit of expressed disagreement as to what "politics" means. Or what constitutes civility. Or what's off-topic. It's not only whether the down votes are due to disagreement; it's also whether the commenter perceives the down-votes as such (which is the case of the example I provided).

Edit to add: There will always be some percentage of members who won't agree, and they're more likely to be vocal. I'm not sure how to measure when the optimum is reached (most users feel the system is working as expected).