Thanks to the other comments, I now think it's not that hard to self-host it :-)

edit: still, self-hosted won't have the phone numbers to join the room