I use that term because its gender neutral and relationship neutral. I find it irrelevant to a discussion whether we are married or not, or a member of the LGBT community or not. However I don't hide my gender or my partner's gender (ie. I use he and she forms due to practical reasons).
I have always assumed "partner" either means "business partner" or that the person is signaling both (1) "I'm gay" without being overt and (2) "I'm resent society's prevailing idea of marriage." at the same time.
One can use "spouse" or "significant other" to avoid the perception of being passive-aggressive.
I'd encourage you to give others the benefit of the doubt. Without other evidence, that "perception of being passive-agressive" is only that, a perception.