Frankly speaking, I haven't handled this case yet. I guess most users are genuine. And so far no one has complained, so I wasn't really worried about it. But I see your point, and need to think about this - at least I need to have some plan in mind in case such thing starts happening.
just curious, how do you deal with spam so users don't send unwanted voicecalls to random numbers?