At a superficial level you would appear to be successful, as anyone who doesn't know you just sees the house and the car. As soon as anyone heard that you spend all your money on yourself and let your family eat scraps, I think most people would hate you. So the question is, do you care more about what some random stranger thinks about you or what people who actually know you think about you?
If your family ate foie gras from golden plates, people would hate you, too. Success and approval are two different things. Someone may think that you are successful, but using that success in an irresponsible manner.
I think it is just OPs wording that got me arguing this point: "a man's success in life". Not a man's success in gaining material wealth. The argument I was trying to make (rather poorly it seems) is that people don't normally measure life success solely by the number in your bank account when you die, there are other measures as well.