> So by definition, if a service is large enough to serve a zillion people, it is probably big and bloated and complex.
Totally agree with your take as well.
I think the unfortunate thing is that there can exist a "goldie locks zone" to this, where the service is capable of serving a zillion people AND is well architected. Unfortunately it can't seem to last forever.
I saw this in my career. More product SKUs were developed, new features/services defined by non-technical PMs, MBAs entered the chat, sales became the new focus over availability, and the engineering culture that made this possible eroded day by day.
The years I worked in this "goldie locks zone" I'd attribute to:
- strong technical leadership at the SVP+ level that strongly advocated for security, availability, then features (in that order).
- a strong operational culture. Incidents were exciting internally, post mortems shared at a company wide level, no matter how small.
- recognition for the engineers who chased ambulances and kept things running, beyond their normal job, this inspired others to follow in their footsteps.
Totally agree with your take as well.
I think the unfortunate thing is that there can exist a "goldie locks zone" to this, where the service is capable of serving a zillion people AND is well architected. Unfortunately it can't seem to last forever.
I saw this in my career. More product SKUs were developed, new features/services defined by non-technical PMs, MBAs entered the chat, sales became the new focus over availability, and the engineering culture that made this possible eroded day by day.
The years I worked in this "goldie locks zone" I'd attribute to:
- strong technical leadership at the SVP+ level that strongly advocated for security, availability, then features (in that order).
- a strong operational culture. Incidents were exciting internally, post mortems shared at a company wide level, no matter how small.
- recognition for the engineers who chased ambulances and kept things running, beyond their normal job, this inspired others to follow in their footsteps.