As far as I understand it, the ether was supposed to be matter, ultimately made out of some kind of particles that obey laws of motion.

Conversely, the electric field is just a potential. It's not made up of anything, it just describes how charged particles interact if they are in a particular place in space-time.