The skins were bitmaps and all the controls had a fixed size and mostly-fixed position, I think, so it wasn't a very sophisticated theming engine.

That's true of Winamp 2, which was the most popular major version. Its skins were, as you say, basically just bundles of image files that were used to override the built-in assets.

On the other hand, Winamp 3 was a complete rewrite, with a much more flexible theming system based on a custom-designed GUI framework, complete with its own XML-based layout engine and scripting language: https://en.wikipedia.org/wiki/Wasabi_(software)

Unfortunately, it never really caught on, mainly due to the combination of performance problems and a lack of feature parity.

That's true for winamp 2. For winamp 3 you could pick the design including button positions and the skins got way out of hand.

Win32 API.

You are like 18 years late. Check xmms and Audacious.