The solution I came up with was just hiding the number of votes either way. Bump up good things and lower bad things, but otherwise keep it normal. It's not a perfect solution, but at least it stops the groupthink and leaves us with the dilution problem.
thesixtyone.com, a social music site, has a clever way of dealing with this: You can "bump" a song, but it costs you "points," which you see a return on if the song becomes popular. It turns upward momentum into a currency instead of something tossed around in large amounts.