What do you mean by goes wrong? Either some package is installed or not. For me, chef manages which ones are. If something is really FUBAR, then wiping is exactly the path I'd take - or more accurately, take that server down for analysis of how exactly it got into that state (so we won't do that again) and bring a clean one up.
Mainly it's about incompatibilities. What if you have two apps that require different versions of a library? If you've installed it in site-packages, then you have little recourse. By separating them out with virtualenv the two apps will work just fine.
You roll version Y yourself and install it into an alternate prefix. If the server uses debian that means make a new deb and deploy it using the standard tools. apt-get/yum/etc. are very solid deployment tools.
