I thought this was going to be about system engineering, but it's just about web scale distributed computing. Disappointing, because I'm digging into systems engineering at the moment and got a bunch of books on it for Christmas, plus thumbing through the SEBoK wiki [1] and associated content.
Honestly I think it's a bit silly that such a broad term as "system" is often used to describe software and hardware administration. Reality is made of layered and networked systems so it feels odd to me to call somebody who manages Linux servers a "system administrator".
Sure thing (though it was my lovely wife that bought them, not me!):
- Systems Engineering Demystified by Jon Holt
- Essential Architecture and Principles of Systems Engineering by C. E. Dickerson & Siyuan Ji
- System Design & Management: An Introduction to Systems Engineering by Bob Parkinson
On a somewhat related note I also got Difference & Repetition by Deleuze and Process and Reality by Alfred North Whitehead, but they're more philosophical than practical.
It seems like all such resources are about web-scale distributed systems and the content/methods are quite standardized.
What about other large-scale systems? E.g. desktop applications - for instance, how to design a large-scale desktop system such as Adobe Photoshop or MS Office.
I doubt you could find two that look the same on the inside, too many talented coders are needed to get to that point; but there's some good stuff by Sean Parent of Adobe on YT.
Systems engineering is the discipline that got us to the moon, and builds advanced technology like military hardware.
For other large scale software projects, I think they tend to use enterprise software architecture patterns like service-oriented architecture (basically microservices) and ESB (basically kafka).
Architecture of open source applications, which has a free book version available on their site, has that info on some other large desktop applications .
Yup and I'm glad you know that because I wish it got more attention! (I'm a mod here, sorry that wasn't clear.) The reason I post those links to past related threads about specific stories is that many readers like to look up old comments on the same topics. Often there are surprises and gems in there.
Btw, the 'past' link in the top bar shows you (more or less) what the front page looked like on a particular day. To find threads about specific past stories, it's best to use HN Search at the bottom of the page.
That's the search box at the bottom of the page. I get that "hn.algolia.com" link when I search for the title of the article. Otherwise I don't see a "past" link. Maybe there is, I don't know.
What really matters I think is when you really need to go this route?
Wat what kind of traffic / transactions/s level do you need to (re)design for this?
Or even, by what metric would you opt for this solution?
I mean, just buying a bigger instance will get you a long way and if you offset that cost to development time / effort / lost opportunity, how will the scale be tipped?
I think it depends a lot on the application use cases and requirements.
Things like:
Will there be spikes in demand 50x higher than baseline usage? I.e. how fast and efficiently the system needs to scale up?
What level of reliability is needed? I.e. is it really a big problem if the system goes down for an hour a handful of times throughout a given time period?
Nah people need to go this route just to pass the interview and get the $600k job by answering the question made up for the interview - it always need to scale globally with at least 99.99 availability and development time /other non-tech dimensions almost always not in scope
1: https://www.sebokwiki.org/wiki/Guide_to_the_Systems_Engineer...