I don't know if there's a realistic way to get low latency video[1]. But the way to get low latency audio is to get as close to POTS as you can. All of the little delays here and there on modern internet audio paths add up. More so if you've got wifi on either side.

[1] I mean, ISDN video calling is probably low latency, but good luck getting an ISDN line installed these days, and video quality was trash.