Seems a bit disingenuous to lump Segment in with ad pixels etc.? We don't use it, but I know of several other SaaS apps who use them to integrate their web app with support tools like Intercom etc. If you block Segment in these cases, sure the web app will work, but the in app chats etc. won't.
True, but segment provides many integrations and the user won't know where that data is getting sent or for what purpose. See the list of advertisers with integrations here: https://segment.com/catalog