I've discussed the MPI-on-YARN thing with quite a few people from both communities and really only found one use-case: MPI jobs you want to run with data in your Hadoop cluster, where it will be done so rarely it's not worth moving the data from your cluster to your supercomputer. Even then, I can't name an instance of this type of use case I've actually seen in the field.