Many companies with lots of designers will actually have designers that specialize in visual (or animation, or even sound) as well as interaction (who might not be trained in visuals at all). My wife, for instance, has a graphic design background, so is perfectly capable of visual design work, but focuses on interaction design (aka usability), and even here one can specialize in things like different kinds of input (touch, gesture, speech). Her old boss (and my former colleague) was a visual designer who specialized in...color...and was quite good at that. And then you have jack of all trade web designers that do a bit of visual, a bit of interaction, a bit of CSS/Javascript, and maybe even SEO. And then industrial, material, packaging, ..., designers, of course.

Designers are really analogous programmers in the sense that there are tons of specializations and skills to consider from within.

I personally self identify as a programming language designer...which is a completely different rabbit hole.