Apologies if this is overly simplified, not sure of your level of expertise (can give you the more complex one too if you want more information!). The dongle in this case is a USB diagnostic tool used by Sony employees and technicians to put a PS3 into service mode.
Essentially it's plugged in and the PS3 started up. The PS3 communicates with the dongle and swaps a set of 'secure' keys to authenticate that the dongle is a legitimate, then runs some code to give you access to all sorts of options you normally wouldn't be able to see/use.
What these guys have done is found the master key found in all PS3s that allows it to authenticate any/all service dongles. Using this information one can generate their own service ID and ultimately create their own dongle.
Basically this was possible to do because the protection mechanisms in place to protect the key relied on the rest of the system not being broken. Once the system was hacked it was simply a matter of time before this was decoded as well.
From my own experience, service mode on a lot of embedded devices typically exposes some diagnostics maybe let's you load some things you couldn't otherwise load, I assume it's the same on the PS3. I also assume this doesn't compromise Sony's ability to sign software or allow third parties to sign software, however in service mode you might not need "signed" software.
Heck, there are off the shelf solutions for this stuff, there are chips you can load a set of keys in to at manufacturing time and they contain all the crypto in the chip such that there is close to no way it could "leak" out. I'd assume IBM, Toshiba and Sony would use something like that and if they properly generate keys the only real way the "master key" could escape would be a rogue employee leaking it. They knew people would attack the platform.
Yep you're spot-on in this case, and as you say the software signing keys hasn't been compromised, though they aren't needed in service mode.
There are definitely ways the dongle keys could have been better protected (and I'm sure a few people are having some very serious talks about why they weren't), but have to give Sony kudos for having a system last 3 years without being compromised, and even now it's only easily broken at ring 2; the gameOS level of the system.
It's simple to protect the dongle keys better: sign the dongle ID. In this way, only the public key exists on the PS3, and the system is secure (if implemented properly). As it stands, their system is equivalent to having both the public and private key sitting on the PS3. No matter how well you protect this key, the system is still broken in theory.
Agreed with the way it could be done better though they didn't actually have the private key on the system. They really badly implemented their crypto so they might as well have though.
I'm referring purely to the dongle attack. When you use an HMAC in that way, your secret is your "private" key. It also just happens to be the "public" key as well. That's why it's a terrible design.
What is the dongle ID exactly?