If you can run g.711 for your codec, things should work ok, depending on your jitter... g.711 is the digital calling code from 1972 that telcos used for T1/E1. On voip, it's packetized into 20ms bursts rather than multiplexed one sample at a time.

If you use one of the newer predictive voice codecs, expect a lot less success, but maybe moving to older/slower modem speeds will help.