The rpath trick you mention is how Nix actually works, but the difficulty is in convincing build routines to obey it. I'd estimate less than a quarter of the packages in nixpkgs have "well-behaved" build routines where it's just a matter of telling the build system where to look for things. Ensuring that you're using all the correct corresponding versions of libraries, headers and auxiliary files for all dependencies requires some build environment trickery, unless you want to maintain extensive patches for half of your packages.

It also means we can allow most build systems to "auto-discover" their dependencies rather than extensively coach them with explicit locations of everything (i.e. tens of configure --with-foo=... flags)