It's definitely not a rocket science to have the test examples with the crafted certificates that are not supposed to pass the tests and to have such tests run at every build.
Most CPAN Perl modules for almost two decades have such tests for much less critical stuff. There's no excuse for not having such methods for security critical code.
Tptacek is muddying the waters by claiming it's hard. It's not. We have more different implementations of the same protocols so it's even easier to cross-verify.
- Here's a library that claims to validate certificates.
- Here's a TLS httpd with a forged commonName.
- for ciphersuite in $supported; do
Completing this test is an exercise for the reader, yet we're waxing philosophical about the perils of Ruby on Rails in this thread for some reason. If I'm being told by a security expert that such a framework would not be helpful, that's concerning and makes me wonder how many bugs such a framework would uncover (given that this one remained untouched for a decade).
This is very close to some of our integration test setups. It's literally a pair of folders "good inputs" and "bad inputs". To run our integration tests, we just throw each input file in those folders at the application and check if it works or if it crashes.
Some of the stuff has assertion files, too, so you can check the way it fails or check some conditions the success, but that's not common.
Overall, this took like a day or two to setup and it catches a lot of errors already, especially because people can just drop errors into the bad input folder and be done with it until someone has time to handle it.
That would be a more compelling argument if any other TLS stack had ever had this particular validation bug in it. As it stands, the test case that you suppose should have existed is specific to exactly the code in GnuTLS, and applies to no other stack.
What is GnuTLS specific in "The failure may allow attackers using a self-signed certificate to pose as the cryptographically authenticated operator of a vulnerable website and to decrypt protected communications"? [1]
Apparently just a self signed cert. It was accepted as the "CA signed." Since 2005.
Most CPAN Perl modules for almost two decades have such tests for much less critical stuff. There's no excuse for not having such methods for security critical code.
Tptacek is muddying the waters by claiming it's hard. It's not. We have more different implementations of the same protocols so it's even easier to cross-verify.