My take on it has always been that you wouldn't want to date someone who picks on physical features like that anyway, it's a giant red flag for an overall shitty personality.
Nearly everyone has physical dealbreakers, and height is a very socially acceptable one that the vast majority of women have. Most women also filter on somewhat less politically correct factors like race. The total hypocrisy when it comes to this topic is considered acceptable since women control the market.
You're not wrong though, both men and women love to make fun of a short guys.