Sure, but in most systems you wouldn’t have much of a choice; if you’re using Debian or Ubuntu, you’re probably using binary packages built by someone else. It takes a lot more work to build everything you want from source in a traditional distro.

It seems like a pretty straightforward tradeoff: do you want to rely on prebuilt packages for speed, or do you want to rebuild it all (and accept the extra time that takes)? Nix and Guix at least make the latter pretty straightforward to do.