I'm currently trying to get realtime audio streaming over WebSockets to work. I'm encoding audio using libspeex on the server side (an iOS device) and decoding it again on the client using speex.js[1].
Transmitting Speex "frames" (where each frame is a 20ms chunk of audio) over WebSockets and decoding in JS works beautifully. However, I have a really hard time to queue up these 20ms AudioBuffer nodes perfectly, without producing any glitches or pops. I'm not sure its even possible with the current API.
What I would like to have is an AudioBuffer node where I can dynamically append chunks while its playing. Since all(?) source nodes are for one time only use, the browser could free the data that was already played again.
An AudioBuffer that allows modification of the data while it's playing would work too - you could just loop it and use it as a ring buffer. However, my understanding of the spec is, that modifying an AudioBuffer's data is explicitly prohibited once its playing.
I can't use WebRTC yet because I want to have support for IE and especially Safari & Mobile Safari. I want to add audio for an iOS App[1] that's using jsmpeg[2] for video streaming.
IE doesn't support Web Audio either. The project I linked above does contain a Flash shim for IE, which you may find helpful.
Any proper realtime audio stack (which ogv.js is not: it targets moderate latency streaming) is going to require a jitter buffer to deal with network/processing latency variation. Writing one is a non-trivial exercise (though not impossibly so).
In any case, I'd recommend using Opus (the audio codec used by WebRTC) over Speex.
Transmitting Speex "frames" (where each frame is a 20ms chunk of audio) over WebSockets and decoding in JS works beautifully. However, I have a really hard time to queue up these 20ms AudioBuffer nodes perfectly, without producing any glitches or pops. I'm not sure its even possible with the current API.
What I would like to have is an AudioBuffer node where I can dynamically append chunks while its playing. Since all(?) source nodes are for one time only use, the browser could free the data that was already played again.
An AudioBuffer that allows modification of the data while it's playing would work too - you could just loop it and use it as a ring buffer. However, my understanding of the spec is, that modifying an AudioBuffer's data is explicitly prohibited once its playing.
[1] https://github.com/jpemartins/speex.js/