I think this is true, but it's also the case that businesses that cater to the very rich are doing very well. My wife works for a luxury retailer who's Vancouver location is seemingly supporting the entire company with it's sales. Revenue has basically tripled over the past 4-5 years.
My impression is that many of those buyers don't even set foot in Vancouver. It's just an investment for them.