The Alpine Linux docker image is only a couple megabytes[0]. There are quite a few people (including myself) using it as a light-weight OS for their containers.
There is also my own 'minifs' https://github.com/buserror/minifs ; it's targeted at embedded, but it works fine on x86s too. Haven't ported it to musl yet (mostly due to the fact that musl wasn't yet merged into crosstool-NG) but that's high on my list.
One thing minifs does is that it has a 'cross linker' took that jettisons every piece of code/library that is not actively linked by executable; also, instead of installing everyone's crap everywhere, I build the distro the other way around, and 'pick and chose' the bits that packages need until they work.
There's also Void Linux, which has musl and glibc versions of packages and offers both musl-based and glibc-based download images: http://www.voidlinux.eu/