"Privilege" has negative connotations and people do not want to be labeled privileged for that reason.

There were times when people tried to hide being gay, having Jewish ancestors or being atheist not because they were out of touch with their own reality, but because they knew that telling the truth would make them socially dirtier. Aside from a few celebrities, "privilege" is looked down on in contemporary discourse.