I read Code by Charles Petzold so I think I have at least some concept of how the operating system and hardware communicate. So now I guess I need a better understanding of how applications communicate with the operating system, as recommended by another reply, more for the sake of knowing rather than any practical reason.
I'm not sure if you'll see this late reply, but here's an idea that may be interesting (ie, accessible and hopefully not too challenging): write a Linux program that doesn't link a standard library (aka glibc). The result will be something that only uses kernel syscalls.
Your first such program probably wouldn't do much; low expectations would be ideal. Returning an arbitrary exit code could be one startpoint, or printing a fixed string could work too (you have write() to write bytes to the TTY, but not printf() (and %s, %d, etc!), as that's a stdlib function :D). This would be a "journey" exercise, not an "end goal" exercise.
You'll need to learn how to
- Disable linking the C runtime (crt0 et al - compile a .c file with 'gcc -v' and pour over the output and you'll spot several crt* things at the link stage), which will mean learning about '-nostdlib', '-nostartfiles' and related parameters
- Deal with the fact that you don't have a main() anymore, you're expected to define _start() instead
- Rummage through a standard library to figure out how to do, well, anything - I recommend poking a smaller C library, like musl or klibc, for ideas (it's mostly boring drudgery you won't need to extensively follow up on just to get started; and there are sufficient "I'm making a tiny stdlib!" projects floating around github that it may be a good idea to weigh up making a 6,983rd - that said, by all means take the opportunity to implement your own strlen(), things like that they're standard entry-level whiteboard challenges)
A good way to get your feet wet might be to first link with musl/klibc/similar, so you can get an idea of the fragility at play (mostly compilers' faults) without blowing everything up instantly, so you can learn how "actually don't use glibc please" is handled in practice. (Read: gcc works, but only within a very narrowly defined scope; clang may work, but may not have as much (easily-found) documentation as gcc. The IRC channel/mailinglist will probably be useful if you're sufficiently interested, but the responses you get may take a day to understand/unpack :) )
There's just enough documentation floating around on stackoverflow and related places that the process isn't overly terrible to start with, and this particular learning process will very clearly explain a heap of stuff.
If you like I can try digging out a bunch of links, but it might take a while (my bookmarks are a thoroughly under-categorized tangle :( ).