Here's what works for small and medium organizations for data which needs to be encrypted at rest, but is not often accessed (so, backups):
1. Buy a bunch of Yubikeys, minimum of 2.
2. Create GPG keys and store them on YubiKeys. Follow this guide: https://github.com/drduh/YubiKey-Guide (if you want to, keep the secret keys, but in case of multiple YubiKeys I would not keep them anywhere). Remember to set the keys to always require touch.
3. Use GPG to encrypt your backups to multiple recipients (all of the YubiKeys).
4. Take care of the physical keys with proper storage and procedures. Do not store the keys together, have at least one in a really secure location, check if you have all the keys regularly, etc.
5. Test restores at least once per quarter, with a randomly selected key.
The advantages of this solution is that it is simple, works pretty well, and gets you a lot of mileage with relatively little inconvenience. You don't have the risk of keys being copied, and guarding physical keys is easier than digital ones.
You still have the problem of guarding the passphrases to the Yubikeys (if you use them), but that is much less of a problem than guarding the encryption keys. A passphrase without the physical key is useless.
This setup works for organization from size 1 up to fairly large ones.
Note that some recently fashionable security consultants crap on GPG from great height, but do not provide an alternative. It's a tool that while having multiple flaws, does many jobs better than anything else out there.
This is all generally good advice, but I think there's huge potential complexity lurking here:
> 4. Take care of the physical keys with proper storage and procedures. Do not store the keys together, have at least one in a really secure location, check if you have all the keys regularly, etc.
Would be great to see what folks think this concretely looks like for joe random startup in Capital City, Somewhere.
e.g. Does "really secure" mean "find a bank that still offers safety deposit boxes"? Does it mean paying for something like Iron Mountain (http://ironmountain.com/) or one of its competitors?
> Does "really secure" mean "find a bank that still offers safety deposit boxes"?
Realistically? Yes. This is what several of the companies I've done contract work for have done. You can still find at least one bank or self-storage place (look for the ones that don't have a nationally-advertised brand and don't look like they're made entirely out of corrugated metal) that do regular safety deposit boxes in pretty much any city. They may only be offered at a couple of locations and I've noticed credit unions bailing the hell out of this market as fast as they can decommission the vaults but boxes still exist.
Let's assume all variables work in the other way, though. If you can't find a safety deposit box and don't have somewhere that's not your office you can drill into a floor or wall and you're storing a small device like Yubikeys or USB sticks, buy the heftiest portable gun safe you can find, one with a steel cable that loops back into the device, and stick it under your bathroom sink with the cable wrapped firmly around the water supply or drain pipe.
One problem with a safe deposit box is what happens if things go to hell at 2am? The bank isn't going to open for another 6 or 7 hours, and meanwhile you're sitting there with possibly business-destroying downtime.
Well the idea is that the safety deposit box key is if everything falls apart, all the employees go missing and you dont have any other options. Ideally multiple employees would have keys as well that could respond in those crucial hours.
But if for instance, your whole security team got in a car accident and all the keys burned up, you'd have a way to recover the creds and save the business.
There's also the ability to just leave it at a lawyer/notary (they already handle deposits, they might even have a secure box at a bank, so you can piggy-back on them for this).
Directors/Founders of Random Co. should just make a few copies a few pieces of papers that contain the passphrase and store them at their own home and ask a few relatives to do the same for them. Depending on their recoverability/safety/accountability trade-off they can increase the number of copies, they can increase the separation between the parts (eg. keys and passprhases), and so on.
The big-big-big advantage of the yubikey approach is that it's a HSM, and you can't accidentally copy the key and leave it somewhere.
This depends on your company size and security requirements. I'd say use common sense for small companies, where the idea is mostly not to lose access to all of your decryption keys at once. As the company grows and you need to worry about trusting people, you are solving two problems: having an always-available fallback decryption key (that's the easy part, and indeed deposit boxes work just fine, but so does your parent's home in many cases :)), and restricting access to keys to those people who need it. The second problem is more difficult to solve.
My main point was that the use of hardware keys makes many things much easier, and you do not have to worry about your keys being copied and used without your knowledge. That's a big thing. Also, the often-ridiculed GnuPG is amazingly useful with Yubikeys (using the setup I linked to), because you can use the same keys for SSH, thus ensuring access to all resources as needed.
i think it's more useful to discuss the goal rather than the means; if the goal is resilience to theft and natural disaster, the means might range from "stick it in a fireproof safe in the boss' office" to "outsource to iron mountain" depending on threat model.
I'm very glad it's being developed, but it is in no way a GnuPG alternative.
GnuPG is a command-line tool that is omnipresent, keys can be stored on Yubikeys, can be used with ssh-agent, and I can use it to encrypt files in an automated fashion, using both symmetric and asymmetric crypto.
This is great advise, and depending on your organization and stack, best when coupled with Vault.
Vault stores all secrets needed by running services (ACL tokens, access keys, credentials for databases, PKI for certificates, what have you).
For the rest (Vault unseal keys/cert keys/operator token, other operator secrets), secure those with the GPG keys mentioned above and store them in some way that suits you (GNU Pass/git-secret/there are several alternatives).
Where you draw the line between what's stored in Vault and not will depend on your org and its needs.
This is similar to a system I have seen, other than the inclusion of a dense QR code as the backup, stored in a secure safe. And you need to test the entire process from scan to key resurrection. We saw that our offline signing laptop's camera was low enough quality that it was very hard (but possible) to read the key, because it was so dense.
1. Buy a bunch of Yubikeys, minimum of 2.
2. Create GPG keys and store them on YubiKeys. Follow this guide: https://github.com/drduh/YubiKey-Guide (if you want to, keep the secret keys, but in case of multiple YubiKeys I would not keep them anywhere). Remember to set the keys to always require touch.
3. Use GPG to encrypt your backups to multiple recipients (all of the YubiKeys).
4. Take care of the physical keys with proper storage and procedures. Do not store the keys together, have at least one in a really secure location, check if you have all the keys regularly, etc.
5. Test restores at least once per quarter, with a randomly selected key.
The advantages of this solution is that it is simple, works pretty well, and gets you a lot of mileage with relatively little inconvenience. You don't have the risk of keys being copied, and guarding physical keys is easier than digital ones.
You still have the problem of guarding the passphrases to the Yubikeys (if you use them), but that is much less of a problem than guarding the encryption keys. A passphrase without the physical key is useless.
This setup works for organization from size 1 up to fairly large ones.
Note that some recently fashionable security consultants crap on GPG from great height, but do not provide an alternative. It's a tool that while having multiple flaws, does many jobs better than anything else out there.