Well, I agree with you. I think that pursuing our interests in mathematics, music, literature or whatever strikes our fancy is admirable. And I think it makes us happier, wiser and more humble as you say.

At the same time, I maintain that we shouldn't feel guilty if we aren't doing it that, for whatever reason. Sure, sometimes we actually want to pursuit some of these things, but don't. Maybe it's because we have a messy schedule, we can't organize ourselves to prioritize passions.

Feeling guilty does little to actually make you pursue your passions. You're better off learning about habits and how to pick ones that serve you.