Something like that would certainly work - I was thinking a one-time code could be emailed to the email address on the account, but either way would be fine.
Given the relative simplicity, I wonder why nobody (at least that I know of) has implemented something like this?
This is how PayPal support works: you use the website to get a random six-digit "customer support PIN" which you then dial in when you connect to their system.
Media Temple operates this way, as well. I like this approach, as I can never remember those "4-digit account pins" (looking at you, AT&T), and I'm happier knowing any given pin is only usable for the next hour, anyway.
Given the relative simplicity, I wonder why nobody (at least that I know of) has implemented something like this?