As long as you don't email anyone who uses a gmail address or uses google's email hosting (which is the majority of internet users now?) then you can get away with that. I did too (also having run an MX since around 2001)... but a year or so ago google started filtering all email which didn't comply into spam.. so.. had no choice but to set it up.
Everyone I know who isn't tech-literate uses Gmail because (a) Gmail is/was better than their ISP's email in many ways, including UI and spam filtering, and (b) it's hard to switch off without dedicating a few hours to changing your email on a bunch of services (and there are a lot of services that don't offer email changes at all) - also, if someone hasn't use a password manager religiously, chances are they don't even have a convenient list of all the services they're signed up for.