if you share Ansible inventory files that are encrypted with ansible-vault, then this is not happening. But i still wouldn't want to have a public repository with the files and the metadata of servers that is clearly not meant for public consumption.
Let me brig another example for OSS projects that could need a private repository: branches for security fixes that are not public yet.
Passwords emphatically do not belong in git.
Your private repos should be maintained such that accessing them would not compromise your security.