You can always opt for Alpine or Tiny Core or such if you're into minimalism. IIRC Debian and Ubuntu install lots of localizations by default, which takes up quite some space.
Minimalism? Minimalism is fitting the kernel and initrd on a single floppy ;) (to be fair, it was hard to get a full x11 in there.. Which was one reason the qnx demo floppy was so impressive)
