A compromised main UI device could also show the wrong account recipient, even if hardware key is used. The text could be changed on the screen when the user meant to send a small payment to someone else. Yubikey will be pressed like usual. Apple's standard prompt on the phone may not have the recipient shown.