So, if I'm writing a new application in 2012, should I use SHA-512, or keccak? If course SHA2 will have better library support... What other considerations are there?
That's out of context; Schneier said there isn't one that's significantly better (which he defined as "orders of magnitude") than SHA-2. But they're all strictly better than SHA-2.
If you're writing a new application, by all means use the new standard. If you have an existing application or have to interface with existing frameworks, use SHA-2.
The construction used in SHA-2 is in part inherited from SHA-1, which inherits from MD5, which inherits from MD4. The only hash in that list that doesn't have a scary research finding is SHA-2. There is a general unease about the family of hashes SHA-2 is in (but that doesn't necessitate unease about SHA-2 itself).
SHA-2 also has length-extension, which isn't a hash flaw per se, but is something you have to know about if you're building cryptosystems with the hash (as opposed to using something off-the-rack like HMAC).
If you are using HMAC, you really don't need to worry much about which hash you're using. There is to my knowledge no practical attack against HMAC-MD5.
Obvious solution would be to make the application capable of using both. I would default to SHA-512, but I don' see any reason why the hash function should be fixed.
Interesting footnote: because of concerns about the security of both SHA-1 and MD5, the original SSL3 protocol used both MD5 and SHA1 as its PRF. TLS 1.1 changes that with a PRF that relies solely on SHA-2.
Also curious about this. I'm writing something with a DAG storage scheme similar to git, currently using SHA-256 to hash blobs. Been thinking about switching to SHA-512, but there's no rush as I can easily switch hash functions anytime before initial public release. How much time/what indicators can I look for before considering a switch to SHA-3?
In fact, for regular hashing you're probably better off using a non-cryptographic hash in the first place, because they're much faster. Instead of security, they're designed for avalanche behaviour and uniformity of the hash space. Cryptographic hashes also cover these design constraints (otherwise they'd be vulnerable), but a hash does not need to be cryptographic strength to be a really good algo to generate hash keys from blobs.
Murmurhash and Cityhash are two excellent and fast non-crypto hashes. The FNV hash is also quite good, very fast, maybe not as good as Murmer and City, but has the advantage that if you like running your own, it's super-easy to implement (like two lines of C code easy).
Let's say I know a little bit about crypto in general, and wanted to write a software implementation of Keccak/SHA-3 as an academic endeavor. How hard would it be? I'm not looking to produce something that will be certifiably secure against side channel attacks, just something that is reasonably performant and can produce the correct results.
I find it interesting that the contributors work for semiconductor companies. I would've thought that something like this would be more likely to emerge from a team with a mathematics/CS/academic background more so than hardware. Perhaps these algorithms can (and will) be going strait into some new hardware cryptography IP cores?
I only tangentially follow the semiconductor companies...
However, I know several advanced algorithms in the realm of cryptography can be computationally expensive, and as demand for security grows, several of these semiconductor companies have implemented things in the hardware to optimize certain forms of encryption. (I can't remember if it was Intel or AMD that released a chip with some sort of built-in support for security).
This is just a theory, but if they're going down this road, it would make sense for them to develop a highly secure algorithm that they know can be easily implemented and optimized on the hardware level. If they're developing specialized algorithms AND hardware, they can probably take it into account during the design process and further optimize their boards.
Or, a bunch of semiconductor guys are also crypto-geeks. Which is pretty likely anyway :D.
All of the SHA-3 candidates were designed to be implementable in fast hardware. Here's a comparison of some of the hardware designs evaluated during the competition:
The C code appears to be public domain. Which might be a problem in regions in which public domain is not a thing but should otherwise be fine for use in most projects.
it's funny, we have new standard, so sha256sum, .., sha512sum utilities should start providing us different results - Keccak, instead of SHA-2 :) i'm wondering how the naming issue is going to be solved
