Self-hosting a website means other people can use your website. That person is right. Let's Encrypt controls the web now, and we only put up with it because it's acting altruistically right now. But that's also how we used to treat Reddit and Twitter and Freenode and how we still treat Discord and Hacker News. Only in one of these cases did users manage to wrestle power back when the owners turned sour, and that was only possible because the ruler forgot to pay his keys, so they defected.
No one seems to realize that Lets Encrypt is a single point of failure now. You compromise them and you compromise a huge percentage of traffic on the web.
Their root CA can't really be revoked either unless you want to turn off most of the internet. Not that big CAs would ever suggest anything like that in the name of "security"