This thinking is why Heartbleed was such a disaster. Everyone left it to someone else, and the end result was OpenSSL being the only serious choice. Yet even the 'experts' got it wrong -- way wrong.
I'm not advocating everyone and their mother implement their own crypto. But some software diversity is a good thing. These algorithms aren't quite as scary or fickle as the documentation and existing implementations make them seem. Especially if you stick to good software like DJB's stuff.
For instance, using montgomery/edwards curves instead of weierstrass curves eliminates a lot of the difficulty in writing a constant-time implementation of ECC. And the 25519 implementation comes with a fast, constant-time implementation of a prime field type.
Yet even DJB's stuff can be simplified. You can knock off a good 80% of the scary code at a cost of a mere 10% of performance. To Google or Facebook, that may be unacceptable. But to me, that's entirely worth it. Now you have a tiny library that is easy to understand, and easy to audit.
I haven't benchmarked TweetNaCl's performance yet, but I think that goes way too far into making the code completely unreadable. The 80%/10% number I gave is against the ref10 implementation.
Please note that like BearSSL, my implementations are alpha-quality. Further, I'm not suggesting anyone use these in production. If I do so myself and it blows up in my face, it'll only have harmed me, and I'll only have myself to blame.
(Also, I'm really bad when it comes to source code comments, sorry. The why really needs you to read the research papers; the how is mostly self-evident. The remaining one-letter variable names were used to match the papers, and because I couldn't think of more descriptive terms.)
If you're using modern C++ you might consider using SaferCPlusPlus[1] for improved memory safety. Safe, fast compatible direct substitutes for C++'s unsafe elements (pointers, arrays, vectors, even references are technically unsafe). Small, easy to use, no dependency risk, and can be optionally "disabled" with a compile-time directive. It's the best answer to the Rust crowd who (justifiably) point out the inevitability of memory bugs in large enough C++ code bases.
I'm not advocating everyone and their mother implement their own crypto. But some software diversity is a good thing. These algorithms aren't quite as scary or fickle as the documentation and existing implementations make them seem. Especially if you stick to good software like DJB's stuff.
For instance, using montgomery/edwards curves instead of weierstrass curves eliminates a lot of the difficulty in writing a constant-time implementation of ECC. And the 25519 implementation comes with a fast, constant-time implementation of a prime field type.
Yet even DJB's stuff can be simplified. You can knock off a good 80% of the scary code at a cost of a mere 10% of performance. To Google or Facebook, that may be unacceptable. But to me, that's entirely worth it. Now you have a tiny library that is easy to understand, and easy to audit.