I didn't say that you should say no to everything. I said you should develop the ability to say no to things that you don't want to do because you have more important things to spend your time on. Trying to make everyone around you happy by giving up time you want to spend on important things is a much quicker route to burn out than declining some meetings you don't care about.
"declining some meetings you don't care about" isn't normally going to make your family angry and your friends drop you, so I don't think that's how people are interpreting your comment.
