I suspect actually Nazis may be a rounding error, but Nazi believers are probably much more. And in some regions maybe even approaching double digit percentages.
If we have data let's go with the data. If all we have is opinions, let's go with mine.
What are we to do? One way of dealing with the problem would be to ask the question at census time: Do you believe in Nazi or Nazi like ideologies? and then round those people up and put them to death. This has it's own problems.
I agree that we shouldn't tolerate intolerance, right up until you said "round those people up and put them to death". Any state run mechanism for this will be abused and infringes on a whole bunch of other rights.
Doing things like clamping down speech that incites violence or otherwise directly harms people is working. More options are open for almost all members of minorities than ever before (in terms of jobs, opportunities, legal recourse, etc... ). The nazis are just being loud about it now because they think trump will somehow protect them.
Convict and arrest people advocating for the death's of others, financially fine those empowering that one specific kind of speech and the people will continue to get smaller. Without the violence hate speech cannot grow.