"Just a different chromosome" can also give you Down's, if it doesn't outright kill you. It seems strange to use a diminutive to refer to a huge change in the genetic material that one's entire body is defined by.
It can be pretty straightforwardly interpreted as a reference to the probability of being born into one gender or the other, which is roughly equivalent to a coin flip (not taking into account the higher chance of being born male, hermaphrodites, etc.)
I have trouble grasping why this distinction matters.
My understanding was that when people make "If I were X" statements, they are referring to having a conscious experience different than what they are currently experiencing. They would still be themselves by virtue of being the conscious observer in that context.