Still doesn't fix the fundamental problem though.
Let's say you automatically install updates on a staging server and automatically deploy to production if all tests pass.
What do you do when you're faced with a choice of deploying an app with a few failed tests (for perhaps not totally clear reasons) or leaving an old version up with a vulnerability?
I missed this earlier. I think it's a good question and point about tests. More and more companies are relying on having good tests, but the situation you propose certainly can arise. It would be a technical and business decision at that point. What is the exposure? What is the risk? What is missing? How long would it take to replace it? What kind of PR do you want to send out. If I got a message that said -- hey, we know this feature is broken, but your security is more important to us, I probably would accept that as a customer.
Still doesn't fix the fundamental problem though. Let's say you automatically install updates on a staging server and automatically deploy to production if all tests pass.
What do you do when you're faced with a choice of deploying an app with a few failed tests (for perhaps not totally clear reasons) or leaving an old version up with a vulnerability?