> i've worked for a company that wanted a tool that could automate any task, and that anyone could use. They ended up writing a large, clunky program with a custom configuration format
Lots of people design terrible, overly complicated systems. This is an argument for good design, not for ad hoc shell scripts for everything.
Would you write a build system in shell? Why not? Because "make" is far better suited to the job. (Even "make" has a lot of room for improvement, but that's a story for another day).
As another example, even Debian (known for its conservatism) decided that systemd was a net benefit over sysvinit, where the latter could quite accurately be described as a pile of shell scripts. Abstractions that suit the problem domain are far superior to ad hoc imperative bags of commands.
And my argument was that a simple design is good design, and in my opinion, better than something that isn't simple.
You could definitely write a build system in shell. In my opinion, "the job" is subjective and you shouldn't have to pick one tool for it. But I would prefer a simple one where I can see pretty much everything it's doing up front and not have to download source code or search through docs.
Debian picked up systemd because it was a popular decision, not because it was better. As far as i'm aware there was nothing broken in sysvinit that systemd fixed; there was just a vocal minority that wanted some new features and insisted it be shipped as default.
And in defense of shell scripts, nearly every single operating system you can think of other than Windows uses scripts to initialize (hell, even Windows has some startup scripts), and that is an abstraction that's suited to the problem domain. It's not like for the past 50 years people have been too stupid to come up with something better than shell scripts. The whole point of init scripts is to let anyone be able to modify their operating system's start-up procedures quickly and easily. If we didn't need to modify them to suit our individual cases they'd be statically compiled C programs or assembly (which, by the way, I have done in the past; it's not a good idea)
> Debian picked up systemd because it was a popular decision, not because it was better. As far as i'm aware there was nothing broken in sysvinit that systemd fixed; there was just a vocal minority that wanted some new features and insisted it be shipped as default.
I don't feel that's a fair summary of the lengthy debate had about this. There is a long page here [1] listing the reasons for selecting systemd. There is also [2], a very good summary by Russ Allberry of the different init systems being suggested (including staying with sysvinit). Debian has rarely been accused of taking a major technical decision because it is hip.
Shell scripts are not simple. They rely on mutable state and loads of it. For every bit (in the binary sense) of mutable state you double the number of possible states your system can be in. This rapidly expands to ridiculous levels of complexity, the vast majority of which the programmer is ignorant of. This is not good design.
It may not be as "good design" as functional programming, but functional (in the sense of usable by human beings) it is very good design. It's essentially a flat program that can be interpreted by non-developers, edited on the fly, is incredibly flexible and customizable and is backwards-compatible by 40 years. It's actually pretty damn useful. But I can see how someone might not like just getting things done and would rather design an immutable state functional program to start their ssh daemon.
It's not about getting things done, it's about reliability and repeatability. When you deploy large numbers of nodes in a system you don't want little bits of state causing random failures. You want everything to be as homogeneous and clean as possible.
> You could definitely write a build system in shell. In my opinion, "the job" is subjective and you shouldn't have to pick one tool for it. But I would prefer a simple one where I can see pretty much everything it's doing up front and not have to download source code or search through docs.
You likewise have to read docs or the source to know what your shell is really doing. Or search through sysvinit docs to know which shell scripts it will run and when. I have to look up how to use shell conditionals every time, and it's extremely easy to get it subtly wrong such that it's not doing what you think it's doing. There are subtle distinctions between shell built-ins and programs of the same name, and it's very unclear in some cases which is being called. The shell is far from being a beautifully simple and transparent abstraction. Perhaps you know it too well to be able to appreciate this.
The shell and sysvinit are abstractions over writing C programs that make POSIX library calls directly. It's all abstractions unless you are writing machine code directly, (and even then you have to read docs to know what the instructions really do). So given that we're all using abstractions, we ought to use the best ones; the one that gives us the most advantages and the fewest disadvantages given the problem we are trying to solve. The ones that have the greatest combination of simplicity, transparency, and expressiveness.
If you want to make sure that only one copy of a program is running, is it a better fit to write a shell script that creates a pidfile, checks to make sure there's a running process that matches it, and that handles a million cases about how the two can fall out of sync? Or is it better to be able to express directly "I want one copy of this program running" and then let the abstraction handle the hard cases?
You might say "well the shell script is easier to debug, because I wrote it and I understand it." Well of course, you always understand the things you wrote yourself. But if I had to debug either a pidfile-making shell script or a well-designed babysitter tool, I'll take the specific tool every time. If it's mature, it's also a lot less likely to have bugs and race conditions.
> Debian picked up systemd because it was a popular decision, not because it was better. As far as i'm aware there was nothing broken in sysvinit that systemd fixed; there was just a vocal minority that wanted some new features and insisted it be shipped as default.
I don't think you're very well-informed about this. Your claim of a "vocal minority" is particularly suspect, considering the decision was made by a vote of the technical committee. Though there was disagreement, almost no one was in favor of sysvinit. And the position paper in favor of systemd identifies specific things that are lacking in sysvinit: https://wiki.debian.org/Debate/initsystem/systemd
What you're arguing for is essentially the difference between a shell script and a more complicated shell script [some tool designed to "express directly" what you want to do]. You ask, why wouldn't we use something more complicated if it does exactly what we want to do? Because unless you really need to do some exact thing, it adds unnecessary complexity which leads to many additional problems. And if you need the added functionality, you can always add a feature or use a pre-existing tool.
Your debugging argument is bonkers. You claim shell scripting is too hard, then say it must be easy to troubleshoot a "well-designed babysitter tool", which requires WAY more domain-specific knowledge of debugging tools! If you don't know how to write bash scripts, you sure as hell aren't going to have an easy time figuring out why your package manager's borking on an update of openssl.
Did you even read the executive summary of the position paper? "Systemd is becoming the de facto standard" .. "Systemd represents a leap in terms of functionality" .. "People are starting to expect this functionality [..] and missing it could [..] make Debian lose its purpose." They only want it because it has new features, and people are starting to expect it. It's a popularity contest, and systemd won.
> What you're arguing for is essentially the difference between a shell script and a more complicated shell script
You seem to have a really shell-script-centric view of the world, as if the shell is somehow a fundamental and "pure" tool, and everything else is a more complicated version of a shell script.
What you are missing is that the shell is just another tool, and does an awful lot behind the scenes to achieve the functionality that appears "simple" to you. Bash is tens of thousands of lines of C and its manpage is hundreds of thousands of words. Using the shell is not "free", complexity-wise. The shell is a programming language, and not a particularly well-designed one at that. Shell script programming introduces its own peculiarities, and it is known not to scale in complexity very well.
> Your debugging argument is bonkers. You claim shell scripting is too hard
No, I claim that the shell is not a particularly simple tool. There is a difference.
I write JIT compilers for fun. I don't shy away from things that are hard. But my brain has only so much room for uninteresting and arbitrary facts. The very pecular way that conditionals work in the Bourne Shell is not something my brain can be bothered to remember.
> Did you even read the executive summary of the position paper?
You have edited out precisely the parts that contradict your position: "It replaces the venerable SysV init with a clean and efficient design [...] It is better than existing alternatives for all of Debian’s current use cases." Yes, the momentum of systemd was clearly a factor, but your claim that it is only but a popularity contest is not a conclusion that a dispassionate observer would reach.
Is it really so hard to remember these three forms of conditionals?
if ! grep foo /some/file ; then
run_something
fi
if [ $INTEGER -eq 0 ] ; then
run_something_else
fi
if [ "one" = "two" ] ; then
do_something
fi
Those are the only conditionals I ever use. I don't really use conditionals in any other way, and it's really not that complicated to see how they work. Sure, there are more complicated forms, and forms that other shells use (what you see above are fairly compatible, common forms of conditionals, except for the -eq). But if you go back to the original shells and how they did scripting, that should work for most if not all other shells today.
Also, those parts you quote don't contradict anything. It's saying systemd has a "better design", which means it is a shinier, fancier new toy to play with. But the paper never once points out any flaw in sysvinit. But that's obvious; Debian had been chugging along for over over two decades with sysvinit without any problems. If your argument is that suddenly, after 20 years, someone realized sysvinit was some horribly flawed design that needed to be replaced, and it just so happens that systemd came along right when they realized it, I don't buy it. What the paper does spell out, though, is all the advantages of systemd for things other than system init. Basically it says "Hey, we want all these new features, and we need systemd to replace init for it all to work, so please just go along with it because it's a much better design."
Here are some things that seem reasonable but don't work:
FOO=""
if [ $FOO = bar ] ; then
echo "equal!"
else
echo "not equal!"
fi
This errors out with:
test.sh: line 4: [: =: unary operator expected
This is because the shell is based on text substitution. So once it's replaced "$FOO" with nothing, it ceases to be an actual token, and the expansion of:
if [ = bar ] ; then
...is an error. This is terrible.
One solution you sometimes see to this is:
FOO=""
if [ x$FOO = xbar ] ; then
echo "equal!"
else
echo "not equal!"
fi
This handles the empty string and other simple strings, but once $FOO has any spaces or shell metacharacters in it, it will break also. This is also terrible.
> It's saying systemd has a "better design", which means it is a shinier, fancier new toy to play with.
You seem dismissive of new technology. If you want to keep using CVS while browsing the web with lynx and sending your email with pine, more power to you (after all, graphical web browsing is just a "new feature"). But the rest of us are moving on.
It's not new technology that bothers me. Me having to do more work bothers me. Systemd is going to make my job more difficult in terms of troubleshooting and maintenance - way more difficult than remembering that an operator requires two operands to evaluate.
What's really funny about systemd is I think that all its features have tons of value, and I would definitely use them. But I also think its creators are completely fucking batshit insane for making it mandatory to replace huge chunks of the operating system just to get those features. You should be able to just run systemd as a normal user process and still maintain the same level of functionality, but for some fucked up reason somebody thought it would be a great idea to make it a completely non-backwards-compatible non-portable operating system requirement. It's a stupendously bad idea, and the only reasoning anyone can come up with for why they designed it that way is "It's Advanced!" Of course, I should add the caveat that I don't care at all about boot times, and so people who are obsessed with short boot times will find systemd very refreshing, in the way an Apple user finds replacing their old iPhone with a new iPhone very refreshing.
Who says scripts have to be ad-hoc? They can be very well designed and tested. In fact, in some ways I agree with the parent: simple well tested scripts are a very powerful and often underutilized tool.
Lots of people design terrible, overly complicated systems. This is an argument for good design, not for ad hoc shell scripts for everything.
Would you write a build system in shell? Why not? Because "make" is far better suited to the job. (Even "make" has a lot of room for improvement, but that's a story for another day).
As another example, even Debian (known for its conservatism) decided that systemd was a net benefit over sysvinit, where the latter could quite accurately be described as a pile of shell scripts. Abstractions that suit the problem domain are far superior to ad hoc imperative bags of commands.