Hacker News new | past | comments | ask | show | jobs | submit login

> How does it go unnoticed that S3 backups don't work for so long?

My uneducated guess (this one hit a friend of mine): expired/revoked AWS credentials combined with a backup script that doesn't exit(1) on failure and just writes the exception trace to stderr.




I bet it writes to a log file. It just doesn't alert anyone on failure so the log just grows and grows daily with the same error.


Or it alerts people, but on the same channel every other piece of infrastructure alerts them, and they have a severe case of false positives.

I've seen that many more times than I've seen the "no alert" option.


New guy: "Hey I see an alert that XYZ failed to run."

Existing team: "Yah don't worry about that. It does that every day. We'll get to it sometime soon."




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: