Software modular synth is really good for learning about sound and signal. I use Alsa Modular Synth[0], which has its faults, but has a simpler interface than VCV Rack. I use it some times even to filter my voice when role-playing monsters on our online dungeons & dragons game (I love how easy these things are to do with JACK on Linux, used to do something similar in PulseAudio with LADSPA filters, but it was much more of a pain).
That said, I got myself a Moog Mother-32[1] recently, which is a hardware semi-modular synth, and I found it much more musical than software alternatives. I could immediately create much more musical pieces than I could with the software – even though I have MIDI peripherals for the software and the software can create any module I can imagine.
Another good option is SunVox; it's part modular synth, part tracker in a really cool UI. It includes a bunch of simple examples that demonstrate how to use the synth modules.
Slightly off topic. I just added a link in that Wikipedia entry to the the Phone_connector_(audio) entry at the first occurrence of phone jack because I'd never seem that connector referred to by that exact phrase.
Back to the topic at hand. I've played with software gizmos, and recently picked up three secondhand units from the Korg Volca range. I'm not particularly musically talented, trying to change that over time, I find the physical controls way way way more intuitive.
That said, I got myself a Moog Mother-32[1] recently, which is a hardware semi-modular synth, and I found it much more musical than software alternatives. I could immediately create much more musical pieces than I could with the software – even though I have MIDI peripherals for the software and the software can create any module I can imagine.
[0] http://alsamodular.sourceforge.net/
[1] https://en.wikipedia.org/wiki/Mother-32