Speaking as someone who was self-taught, spent four years getting a formal CS education, and then has 16 years post-college professional experience, I don't think you probably missed much. I think the only really practical thing I learned in school was big-O notation, and you can probably pick up the essentials of that in about one day of moderate studying. I can't recall ever learning a data structure more complicated than an AVL tree in school, and I think that was in an extra directed-study class. (If you've not heard of an AVL tree, that's not because they're terribly tricky -- it's because they're now nearly obsolete.)
Mind you, a lot of the stuff I learned studying CS at college was interesting. It just hasn't been particularly relevant to my career. My math classes (I had a double major in CS and math) have actually been much more professionally useful.
I agree with you about the math class. As I mentioned elsewhere in the thread, I ended up doing a minor in CS (which comes out to about 6 courses). I focused these courses on the more theoretical/mathy options, and that has definitely served me well.
Mind you, a lot of the stuff I learned studying CS at college was interesting. It just hasn't been particularly relevant to my career. My math classes (I had a double major in CS and math) have actually been much more professionally useful.