If you eliminate the chance of someone feeling uncomfortable you also eliminate a lot of good interactions.

If there are only a few people uncomfortable with this and there is no harassment, that seems like it's probably fine overall.