Yes, there is a significant cost to being built differently regardless of perceived advantages (by one's self or others). For example, as an autistic, I have to cope with finding interaction with non-autistics quite difficult for me, even if detail-oriented thinking can make certain tasks seem easier to me.