If no one ever downvotes what you say, you're probably not taking enough risks in your commentary. (But if you know most people will downvote it, especially if you know it's going to be flagged, you probably shouldn't say it. Not for any big reason, just because it's rude.)