Come on mate, just use a bit of imagination. You use the pam_usb module to login locally and an encrypted passkey stored on the same usb thumbdrive for remote SSH connections.