Quoting from above: Finally, for these three reasons, passphrase-encrypted SSH keys are leaked in their encrypted form, but an attacker may attempt to crack the passphrase offline. On the other hand, SSH keys that are available only through an authentication agent are never leaked, in any form.
So if you use an agent, and follow the good advice to encrypt your private keys you should be safe(er).
https://www.qualys.com/2016/01/14/cve-2016-0777-cve-2016-077...
Quoting from above: Finally, for these three reasons, passphrase-encrypted SSH keys are leaked in their encrypted form, but an attacker may attempt to crack the passphrase offline. On the other hand, SSH keys that are available only through an authentication agent are never leaked, in any form.
So if you use an agent, and follow the good advice to encrypt your private keys you should be safe(er).
See for older client versions:
http://martin.kleppmann.com/2013/05/24/improving-security-of...
or better for newer clients:
http://www.tedunangst.com/flak/post/new-openssh-key-format-a...