Well, explaining why you thought a point was wrong is a little different than trying to hide dislike or indifference towards a person. The former isn't personal.
The problem with explaining every down mod is that it causes a whole discussion to occur regarding moderation. To me, that seems like a tremendous waste of time.
In a democracy, not everyone is going to see things in the same light. That's just something that we all have to deal with and move on.