How is the command line more direct than a visual file manager showing a directory as a list of files ("ls"), displaying file metadata in a file properties window ("stat") and opening files in a graphical text editor/hex editor ("cat")? How is printf(1) an interface to the operating system? It's just a helper command for manipulating text, which in a graphical interface is unnecessary.
Also, remember that part of the reason the Unix command line matches the underlying operating system primitives so well is because many of them were in fact designed to provide better support for command-line interfaces. In a graphical environment, concepts like "the current directory of a process", or its "controlling terminal", or "session ID", or distinctions between SIGSTOP and SIGTSTP, are superfluous.
Your second paragraph largely answers your first. The command line and system calls are strongly aligned. GUIs are not, and present an entirely new metaphor.
Stephenson describes the problems of GUI interfaces in terms of "metaphor sheer", a beautiful phrase.
There's another essay on interfaces, which I cannot presently place, that describes the situation in another way -- that the GUI folder, and colour, and listing, and columns, aren't themselves representative of how the OS sees or organises its content, nor do the actions a user practices on the desktop correspond in large part to what's happening at the underlying level.
And it's not as if all GUIs are necessarily bad. Another reference I cannot place -- a conference presentation of programming concepts in which a simple game was developed and an algorithmic landscape with a tree, flowers, and moon designed, shows how GUIs can be tightly linked to the underlying code.
But the key again is that the metaphors at each layer must be aligned, in ways that most visual computer metaphors are not. I find myself, when using computers, constantly having to second-guess the system, and in particular, if I know how the plumbing is connected, think in terms of "well, I want X, and have Y, but it's wired through Z, so ..." Which actually makes my cognitive process more complicated. As well as removing much the power of textual tools to perform innate looping or mass operations.
Not really. The text-mode shell presents you with a hierarchical file system where "files" are organised in a tree of "directories" and can be opened in various programs --- and so does your typical GUI. The only new thing added by the GUI are interaction and composition paradigms; e.g. windows, letting you simultaneously operate several programs (directly), and the clipboard replacing pipes. The features mentioned in my previous post are mostly implementation details and could just as well be moved to userspace; they hardly demonstrate some inherent "directness" of the command line.
Quite simply, this concept stumps and frustrates me endlessly in GUIs, and I almost always resort to a commandline.
In a CLI, a filename is simply an argument to the command you offer, and your thinking is command-specific. "How do I process this data?"
In a GUI, the filename is front and center, and the logic is one of "I need to open file X", with the application being dictated by some attribute of the file -- and at worst, the file's extension, something which is highly subject to external manipulation.
There are cases where those bindings are fairly strict -- only a single program can meaningfully interact with a file's content. This is often the case in GUI contexts, examples being Microsoft Office file formats (see a related ongoing discussion on that topic), some project-related tools such as Autocad, Photoshop, project planning tools, or medical imaging systems. But that's a deliberate choice of tool designers.
In a workflow I'm familiar with, I edit text files in specific formats with various editors (GUI tools, vim, emacs), process those to produce desired conversion or end outputs (Pandoc, latex, etc.), and can view the finished results in any of numerous tools, or process those further.
The GUI doesn't entirely limit the user to a single application operating on a file, and it can make the process somewhat simpler in terms of "what do I use to open this file" (though there are CLI tools which achieve this as well, "open" on OSX and some equivalent commands, though I use them so infrequently I can't recall them at the moment, on Linux), through standard file associations. It is possible to override those defaults, but it's difficult and cumbersome, particularly when interacting with multiple files.
And what the GUI does not support except in the most tedious way is the concept of operating on many files with many programs in a systematic way.
You may not need this in your world. Most people may not. But for me, the construct of:
for file in $( list-generating command )
do
process1 | process2 | ... | process n > $file.out
done
is something that's very high on my list of tools. The more flexible in that I can supply arguments, filters, input and output files, and processing logic, on a single command line. That's a loop which might have a handful of arguments or millions, and I've used it across that spectrum (though generally with some form of parallelisation in the latter case, also fairly trivially supplied).
And I can develop (and test) the processing iteratively, in terms both of inputs (one or a small set of files at a time) and processing (adding to the set of manipulations) through my CLI editing capabilities (bash readline and command editing).
The process is powerful, adaptable, flexible, and quite literally allows me to do in seconds what would take hours on a GUI, or in a day or two of unattended operation what would take years of penny-ante error-prone interactive work.
That is where the GUI and CLI metaphors start departing rapidly, and it's why I find myself cursing and screaming at systems, such as my complete piece-of-shit Samsung Galaxy Tab A 9.1" Android tablet, for the multiple ways and means in which the UI and OS simply gets in my fucking way for doing the shit I want and need to do.
It's a capable piece of hardware. But for the entirely self-serving choices of Samsung and Google, it is crippled beyond any and all possible forgiveness. And cannot be adapted to those useful purposes due to further lock-down on the device itself (I've wasted days of my life trying).
Not all of those limitations are specific to the GUI, but the limitations of the GUI leave me looking for CLI alternatives which cannot be applied thanks to yet more limits on the system, making me all the more painfully aware of the original set of insults.
In a GUI, you can right-click on the file icon and select "Open with another application" from the menu. What's your point?
You criticise point-and-click GUIs for poor composability and programmability. Those are valid criticisms; nobody seems to have figured out how to implement these qualities well in a pure point-and-click interface, and this will perhaps never happen. But these criticisms have nothing to do with directly reflecting the abstractions offered by the operating system. The GUI offers access to the same abstractions; it's manipulating them that is different.
And the Unix command line is not particularly well-designed either. I can already spot a potential bug in the sample you gave: what happens if your list-generating command outputs a file name containing a space? (Don't answer "don't use spaces, then"; it's as valid character as any other.) You probably know how to fix this mistake, but have you thought why you made it in the first place? It is a consequence of one of the poorer choices in the design of Unix: basing IPC on text streams that each program has to parse and serialise on its own.
Also, remember that part of the reason the Unix command line matches the underlying operating system primitives so well is because many of them were in fact designed to provide better support for command-line interfaces. In a graphical environment, concepts like "the current directory of a process", or its "controlling terminal", or "session ID", or distinctions between SIGSTOP and SIGTSTP, are superfluous.