To paraphrase jwz: "Some people, when confronted with a
problem, think 'I know, I'll use PGP.' Now they have two problems."
More seriously, PGP is really hard. I think it has some potential for signing emails from your bank, Amazon, etc. but other than that ... it's just a pain. I'm not a crypto expert but a reasonably skilled IT professional, and even I struggle with it.
Yes, the message is encrypted with a key, you only get access to this key by decrypting it with your private key. Everyone on the email has the key for the message encrypted with their public key. Hence the privacy is not perfect but pretty good.
Encrypt it before sending.