I have worked on security, databases, some distributed systems, a couple of video games, AR/VR and some ML systems, mostly connected to database and data pipelines. Knowing what the kernel is likely doing underneath has been useful in all those domains. More than that, being able to set up a production environment by yourself is hugely useful.
If you know the internals, the complexity of a lot of middleware kind of collapses, too. There are a million frameworks for doing threading, for example, but they basically all boil down to clone, a futex and some atomics. Similarly, you can figure out how almost anything works with strace. It demystifies so many things that you otherwise need years of experience to debug.
As a practical matter, you can always find a job as a competent DevOps type who can code and talk to SWEs.
Are there any books etc that you could recommend on Linux internals. I am mostly write c# code, and I didn't do computer science, I mostly self taught.
I recently set myself the task of learning some c as I was finding that I was hitting a brick wall when it came to trying to learn more lower level concepts.
I want to work my way through computer systems a programmers perspective :
I recommend consuming everything Brendan Gregg produces. His work is mostly around kernel performance profiling and tracing, but you can learn a lot from just doing that. Also, the vast majority of time when you need to dive into kernel internals it is for performance reasons.
If you prefer physical books “Systems Performance” by Brendan is good as well.
As the sister comment says, Linux Programming Interface is really good for learning linux system programming topics but it is mostly focused on user space.
For more userspace stuff I like Chris Wellons‘ blog at nullprogram.com
If you're the kind of guy who likes to work through one 1000+ page behemoth at a time, I can recommend The Linux Programming Interface specifically. As a nice secondary bonus, the code examples are all in good old fashioned C, so you can compile and run them to test things out as you please. :)
> Similarly, you can figure out how almost anything works with strace.
I recently had an example of this higher in the stack than anything you listed: A co-worker couldn't install node_modules because yarn appeared to be running an older version no matter what he tried to upgrade it - turned out there was a hidden config file in his home directory none of us knew about that could tell yarn to use a different version of itself.
