I've designed Talon (my voice / alt input project) from the ground up for cross-platform scripts/configs. Once I port the OS layer to Windows/Linux you should be able to reuse mostly everything (with some app/os-specific stuff overridden at whatever abstraction layer it makes sense).
What sort of CPU load are you seeing? I'm able to dictate on battery, peaks averaging ~20% cpu. (Dragon is a different beast, peaking to >80% anytime it's over the noise threshold, so you need a really good mic to get good battery life).
I think that if you made a module that did voice commands for VLC media player that it could be quite popular. I've often wished I could just yell out "VLC pause" instead of having to walk over to the keyboard or mouse.
What sort of CPU load are you seeing? I'm able to dictate on battery, peaks averaging ~20% cpu. (Dragon is a different beast, peaking to >80% anytime it's over the noise threshold, so you need a really good mic to get good battery life).