In my experience, if you value your privacy, freedom etc... I'd say living like a millionaire with close to 100 (but not above) is much nicer than living above that. Below 100 million, you don't need bodyguards and private security, you can be as low key as you want.
Oh yeah of course, depends entirely of the country and the income disparity. I live in HK and regularly I see some members of the Li Ka-shing family go to one of my favourite restaurant (Chiu Tang, a nice but not too expensive fine dining place). They're always accompanied by 4-6 bodyguards and have to dine in a private room.