Personally, I like to have different keys that I treat with different levels of care/paranoia. I'm not particularly worried about leaving my github key 'added' to my ssh agent 24/7, but I don't do that with my work production key.