Most people are already vulnerable to this because they use the same password everywhere. The fewer service providers there are holding a copy, the lower the risk of compromise.
This is not how it works. We don't hold any copies of users' passwords these days, there are hashes for that. Certainly, some old, or poor quality in-house software still do it, but then it won't offer you integration with whatever SSO service anyway. And single point of failure is very real, if you trying to operate world-wide: sign-in with X may suddenly become illegal, or inaccessible outside of the USA.
