Good people want to limit the speech of bad people in principle, sure, just as good people would endorse violence if there were a way to ensure it were directed only against bad people. But no-one can fairly judge that, so limiting speech is a bad idea in the same way that permitting violence is a bad idea. The marketplace of ideas works slowly, but it does work: it does, ultimately, find the truth; good ideas succeed while bad ideas die out. Whereas the "marketplace" of violence can't tell whether an idea is good or bad, and while the violent people might be on the side of good for now, there's no reliable way to keep it that way.