Well I basically used a a Linux distribution made precisely for this: it was called "RasPBX" back then, IIRC. The hardware was simple: a Pi 1, an external HDD hooked through USB to the Pi and Cisco VoIP phones. That Linux distro already took care of picking software components/versions all working fine together and it was tailor made for the Pi. It was pretty plug and play from what I remember.
I'm a software dev, not a sysadmin, but I can find my way around configure Linux machines.
I don't remember it as being particularly complicated. What I do remember for sure though is that once it worked, it worked flawlessly for years.