Pretty much. This isn't a court of law or a public gathering. This is a privately organized event. It's totally okay to exclude disruptive people, and to act on stuff you're reasonably sure is true.
Imagine there was some emacs guy who routinely went around harassing people who used vim. To the point where he got so offensive that the vim people felt they weren't being respected as professionals, even humiliated and demeaned. Some were hounded by him at offsite gatherings. The conference organizers had multiple reports of this occurring. Do the vim people need to "prove" something before it's appropriate for conference organizers to take action? Should their claims be more suspect than anyone else's?
I suggest that you re-examine why the 'unproven claims' thing is so important to you. If you start with the assumption that women and men, just like emacs and vim users, have equal interests in having a good technical conference, and are equally competent to determine when someone's being disruptive, why wouldn't you trust their reports?
Let's look at the flip side. Someone who does go around making life miserable for others by falsely accusing them of things. In a case where nobody knows both parties well, what are you supposed to do? I'm not saying what happened was OK. I'm saying the flip side is something we need to beware of also.
I think the OP is arguing that there should be a procedure in place, so you're not improvising, or making arbitrary rules that aren't fairly applied.
If nobody saw it happen and nobody knows either party well, I would say, record it and move on. If you have multiple reports, have a planned way to escalate, involving friendly warnings, going all the way up to removing the person and/or banning them.
Where it gets really bad - and I think everyone has seen this situation before - is when the guy is some sort of alpha geek, and may even be the kind of person that draws attendees. Everyone is inclined to bend the rules. I think that's why a standard procedure would be important.
That doesn't seem to be the case here, though. The organizer's sigh and admission that "he never should have allowed this guy to come to the conference" sounds like he had prior warning that this could happen.
I can't reply to Avenger42 as this thread has got too deep, but in response to him or her, you are supporting one unproven claim, by citing her, citing him, making another unproven claim. He could be lying. She could be lying. And if I didn't have access to the same blog post and you were telling me about it, you could by lying.
Moreover, she could be _mistaken_. Perhaps about the guy's name. He could also be _mistaken_, thinking she's talking about another guy, or getting names and faces mixed up.
Well, just imagine there was some emacs guy who routinely went around harassing people who used vim. He was offensive and yet they gave back words and didn't leave as he wanted. So he reported them on the next best thing that would get them baned.
One can take a stand against the kind of bad behavior without being prejudicial toward the accused.
E.g., the organizer could go to the culprit and say, "As you know, Joe, we take sexual harassment very seriously here. Could you tell me what happened last night at the bar?"
I could imagine a lot of ways it goes from there, but all of them seem compatible with taking a strong stand against jackassery without giving up a strong stand for fairness.