Really, what I want is the ability to shunt unknown callers off to a holding area where they have to listen to a message and press a button in order to be let through. If it's a legitimate caller, then the inconvenience is minor.
This works very well in my experience. I wrote a "captcha/graylist-esque" rule for my asterisk server which intercepts unknown incoming calls with "because we don't recognize your phone number, please press one now -- or leave a message after the tone". Pressing one transfers the call to the ring group and whitelists the caller id.
For grins, _failing_ to press one results in my office speakerphone playing "sad-trombone-sound.wav" -- how I love to hear that sound.
Most phone spammers wouldn't get through.