I've found that a good practice for building systems is not to rely on the software that comes with the OS for the specific task the system is being built for. When I can I always build task specific software from source, not because the software with the OS is bad always, but because building from source gives you a lot more control over the software (compile time features, paths, etc). And you can also typically get a more recent release of the software when building from source since it doesn't have to go through the OS vendor.
This seems like a good idea on the surface, but it has some pretty serious negative consequences.
When you need to replicate your environment, you now have to build all of the custom bits exactly as they are on the production system, rather than simply running "yum install perl foo bar baz". Depending on the length of your dependency chain, and the dependency chain of all of those components, this could be incredibly time consuming, even if you don't make any mistakes in the building process. Building a binary tarball of all the stuff you need is an option, but then compatibility issues with existing system libs and such are bound to happen, and that's pretty ugly from a paths and upgrades perspective.
You also make your environment less standard. A new hire is going to have to learn not only your application, but also all the crazy town details about your particular and very specific deployment (and setup their own copy of it on their own system). If everything except your app comes from OS-standard packages, you can expect someone familiar with RHEL/CentOS or Debian/Ubuntu or whatever OS you use to know where most things are right off the bat.
You'll probably do more things wrong with your build than the OS vendor did with theirs. In my business, I see a lot of custom PHP builds, for example, and almost every single one of them is broken in more than minor ways (and we end up hearing about it, and trying to figure out what they did wrong in their build). Your OS vendor version has a lot of people banging on their builds and reporting bugs. I'd pretty much always bet that their build is better than yours from a reliability perspective.
It makes it harder to replicate your deployment, if something catastrophic happens to your production box. Packages are more resilient to library changes and such than a big ball of crud tarball of your binary builds. And you won't want to spend several hours rebuilding on the new target machine while you're offline. A complete system backup could be restored...I dunno if you've ever done that on a remote system before, I assure you it is non-trivial and stressful.
What I would instead recommend is to find out which components you need custom (I'm not denying that sometimes you really do need, for example, perl 5.10 and the OS has 5.8.8--it happens, and that's fine), and build new packages int he native format and dump them into a yum or apt repository. It takes an extra day or two, if you don't already know how to do it, but it'll save you many many times that amount of time in the future--and those hours in the future might be far more stressful hours than while you're first setting things up. Rebuilding a package from SRPM or a deb source bundle is usually pretty easy...bumping revisions in dramatic ways might not be trivial, but recompiling with specific options is no problem at all. And, one can usually find a source package of the latest and greatest in the devel branch of the OS, which makes even major revision bumps easy (though, because it is the devel branch, you're probably giving up some maturity in the package...far fewer testers on the devel versions).
I agree, if you don't know what you are doing then you can screw things up pretty good by building things from source. Although harder, you can still also screw things up installing software from OS vendors. Both approaches require care. I've found build/deployment automation and documentation to be the things that address most of the problems highlighted here. There's a lot of cool software out there in this area that helps. Building task specific software from source is definitely worth it, you've just got to know what you are doing.
