I believe that sign-in-with-X implementations require actually authenticating with X in that context. So in order to sign in with Google to a website, I need to sign into Google itself in the same browser. So in that sense, you are enabling Google to track you in that browser - but no differently than if you just logged into gmail. You could sign back out of Google immediately afterwards.
But just the fact that an app uses Google authentication doesn't give Google any kind of privileged access to that app's data (beyond purely the data that they're logging in on this browser at this point in time) unless the app is pushing data actively back to Google (which they could do anyway for a gmail-managed email address, I guess) or you believe that Google will then subsequently forge authentication requests to that app and pull data out itself. Both of these things are hypothetical violations of your privacy enabled by using auth-with-Google, but for most use cases are rather unlikely, I would guess.
It's not that bad. The only thing Google/FB/Twitter gets to know is that a user Y is using app X. Nothing more, and not detailed usage stats, just the basic fact.
For that they handle the complete user registration, recovery & auth process for you, with all the work and pain attached to it.
Granted if your OAuth provider were really evil, they could log into the users App X account and access whatever data he has inside the app, so you have decide if that a concern or not.
Yes, but they get "only" that, but with it, they get, time/date of when you use the app(and perhaps how long, depending on how you/they handle logouts). Plus they get this for EVERY app that's used. You start aggregating this information and suddenly you can tell a LOT about a person. Plus this is all for ad dollars, FB/Google/etc can(will/do?) sell this information, to anyone willing to pay for it.
For a hello world app, no big. For a game app, what happens when your employer buys the data, and notices you are playing games on "company" time... Of course lots more privacy failures can be easily imagined here. I picked low privacy failures, but larger failures are very easy to imagine.. Especially when we know that most large governments also have this data, directly siphoned from Google/FB/etc.
But just the fact that an app uses Google authentication doesn't give Google any kind of privileged access to that app's data (beyond purely the data that they're logging in on this browser at this point in time) unless the app is pushing data actively back to Google (which they could do anyway for a gmail-managed email address, I guess) or you believe that Google will then subsequently forge authentication requests to that app and pull data out itself. Both of these things are hypothetical violations of your privacy enabled by using auth-with-Google, but for most use cases are rather unlikely, I would guess.