Is this really a shell scriping language? Hush isn't an interactive shell, nor does it compile to a common shell script. The only way to run these scripts is to install the hush interpreter and run the script through it.
Isn't that just a normal scripting language? What's the real benefit of using this over Node or Python? I suppose the syntax is more aesthetically similar to shell scripts... but I don't exactly see that as a plus.
System level stuff sucks in Python. Dealing with files, I/O, permissions, etc is a real pain. It easily takes 5x as long and as many loc to do the same thing as in bash.
I can see the benefit of dropping to a command block to, say, run a command and filter the output with some | grep | awk | sort of whatever, and then seamlessly come back up to a more fully featured language to deal with that data.
This is very true, particularly if your script is just an imperative list of commands to run.
Python scripts win when you actually need to handle errors, or non trivial output parsing, or when the concept of “list of things” doesn’t fit neatly into the “lines of text / pipe symbol” paradigm.
I try to think of it like Donkey Kong. Every time you do something like this:
var=$(command | awk -vID=$ID ‘$6 ~ /foo/ and $7 == ID {print $1 “ “ $2} | head -3)
…you get hit by a barrel. Three hits and you turn it into a Python module with a __main__.py or an if __name__ == “__main__” at the bottom.
If it’s a truism that most shell scripts are better off written in Python, it’s just as true that all Python scripts are better off as reusable modules (that may also be standalone scripts.)
My experience is different. For me, Python has a good balance of readability, access to system functions and consistent, predictable interfaces. I'm not saying you're wrong, just that "it depends". I'm pretty adequate at writing BASH, but it's not my "daily driver" programming language, so I have to still try things out, google a lot of things that aren't easily obvious, and excessive trial and error. But as a Java programmer I also know that Java, with all of its abstractions and verbosity, is a complete non-starter for shell scripting. For me, Python has often been a helpful compromise.
Part of this may also be that I've never learned to do a great job at organizing a BASH shell "project". So, those often get unwieldy maintenance nightmares. At plenty of time me and my peers have agreed: when your BASH script exceeds [some low number of] lines, find another language.
Again, it matters a lot what your team's culture and comforts are. If that's BASH for you, then r.e.s.p.e.c.t.
You’re comparing Python and Bash as programming languages, not as shells. Python’s great as a programming language, but it’s a terrible shell. Likewise, Bash is a terrible programming language, but as a shell is pretty good and is probably the most used shell. The point parent was making is that it’s far more difficult in Python to run a process, capture the output, and grep something from it, than it is in Bash. In Bash this is a one liner with extremely minimal syntax. In Python, this is a big multi-line affair invoking several modules and multiple functions. Python isn’t a shell purely because you can’t execute an executable file by typing the bare name of the file. Python also doesn’t pass the cut & paste test - you normally can’t paste snippets from somewhere into a Python repl due to indentation issues, this alone makes it very difficult to use as an interactive shell. And Python is not meant to be an interactive shell, so there isn’t much of a problem.
It might take a few more lines of code to do stuff in a real programming language like Python (I would recommend Deno actually) but at least there's a decent chance it will actually work reliably.
> then seamlessly come back up to a more fully featured language to deal with that data
That's the thing - I don't see how you can do that. The command blocks return nil or error, but not the output, so that you'd have to dump that to a temporary file, scrape the file separately, remember to delete it, and so on. I can make it work, but it doesn't live up to the idea of seamlessness.
The point is that you can do things like create external processes, pipe them together, redirect output to files, with the same ultra-lightweight syntax of bash etc. Compare that to all the nonsense you have to do in Node or Python to pipe two processes together!
You’re right in that writing Python using shell paradigms is awful.
Writing Python that shells out to do stuff — but using Python paradigms — isn’t so bad. Particularly if you are used to calving off your shell stuff into shell functions that run something and massage the output. In Python you end up doing something pretty similar: yield the text you want and use that as arguments to the next call to subprocess.run.
The purpose of OP's project kind of reminded me of shell.js (shx) [1] which is a nodejs library that wraps all kinds of common UNIX commands to their own synchronously executed methods.
I guess that most shell projects start off as wanting to be a cross-platform solution to other operating systems, but somewhere in between either escalate to being their own programming language (like all the powershell revamps) or trying to reinvent the backwards-compatibility approach and/or POSIX standards (e.g. oil shell or zsh).
What I miss among all these new shell projects is a common standardization effort like sh/dash/bash/etc did back in the days. Without creating something like POSIX that also works on Windows and MacOS, all these shell efforts remain being only toy projects of developers without the possibility that they could actually replace the native shells of other operating systems.
Most projects in the node.js area I've seen migrate their build scripts at some point to node.js, because maintaining packages and runtimes on Windows is a major shitshow. node.js has the benefit (compared to other environments) that it's a single .exe that you have to copy somewhere and then you're set to go.
When I compare that with python, for example, it is super hard to integrate. All the anaconda- or python-based bundles for ML engineers are pretty messed up environments on Windows; and nobody actually knows where their site-packages/libraries are really coming from and how to even update them correctly with upstream.
I love Python, but if all I need to do is grab one part of one line of something, and put it into some other command, then I'm just going to use some unholy mixture of sed, awk, or whatever. If I did the same thing in Python, I'd end up with thirty lines or more.
Thirty lines? Unholy one liners are not limited to shell scripting:
from subprocess import check_output
sh = lambda script: check_output(script, shell=True, text=True)
ips = set(line.split()[-1]
for line in sh('last -a').splitlines()
if line and 'tmux' not in line)
The first two lines are pure overhead.
The last line is the equivalent of a shell script one liner but now has all the advantages of a language that supports “\N{clown face}”.
In 200 bytes half of which is overhead. That's not a great start.
> The last line is the equivalent of a shell script one liner
It's slower and requires much much more typing, and anything I want to add isn't going to go in the right place. It has a gross hack to support "tmux", and produces other spurious output (bugs). I don't want my X sessions or other ptys; I have reverse DNS disabled, and I really just want IP addresses.
This is what I would write in shell:
last -a|sed -e 's/.* \([0-9a-f]*[:.][0-9a-f.:]*\)$/\1/;t;d'|sort -u
> now has all the advantages of a language that supports “\N{clown face}”.
This is a joke right? I have twenty cores, this is going to use three. Python is going to use one. 30% less typing, less bugs, faster. Those things are important. A fucking emoji is not.
for those of us with not as much grey in our beards (+1 this response if you get the joke) the python example is a ton more readable to me. Now that probably doesn't matter if the utility is only for you. I've a number of helper programs/scripts/etc that I use that I wrote and are only for my consumption.
Re the paralell stuff and python it's easy to import multiprocessing and take advantage of all cores. I think where python wins is how easy it is to handle errors and organize things as it's a full fat programming language vs a shell scripting language like Bash + the gnu userland.
> the python example is a ton more readable to me.
The python example is also wrong, as in it produces the wrong output: Maybe if you are only worried about your own consumption you can ignore all those poor souls running literally any other application besides ssh and tmux, but at some point you're going to have to stop admiring how "readable" it is and fix it, and then what?
I don't buy that conservative coder mentality of keeping the code clean and readable, because one drop of shit is going to be impossible to remove later. In fact, I think that's bananas. Get correct first, then improve.
> Re the paralell stuff and python it's easy to import multiprocessing and take advantage of all cores
You jest.
> I think where python wins is how easy it is to handle errors
Pray tell what errors do you think we need to handle?
Your shell thing is honestly fine. The emoji thing was a joke, sorry that wasn’t clear.
The thing is with 1 IP lookup in N addresses a grep is also fine, but with M IP lookups the set is O(1) instead of O(N) and that makes a difference.
It’s not just theoretical. Bash slowness adds up fast and you will need to upgrade to a real language anyway.
Example: IPv6 addresses need to be truncated to /64 but last prints them with 0 truncation. Good luck doing that in bash. You’ll need a fully fledged IPv6 address parser, if it’s code that has to survive.
You make a fine point, honestly, about what you can get away with in shell scripting. The bigger picture is that literally nothing ever survives the sticky fingers if many engineers over time if it is a shell script. It gets worse and worse until it is ported, at which point it begins a new life as maintainable code.
> Unholy one liners are not limited to shell scripting
You’re mostly proving the parent’s point here. Yours isn’t a one liner, and even in Bash people don’t write multi-line for loops in one line. I don’t know what you mean about the first two lines being overhead; this doesn’t work without them, and you have magical options “shell” and “text” in there that matter. Your script doesn’t pass the result to another process, so you need another line. This also splits output on spaces, it should be more generic. Doing that involves another module (regex), another function call (or more likely several), and several more lines.
Compared to “last -a | grep tmux | process”, your script is much closer on a log scale to 30 lines than 1, even if the parent’s 30 is a little exaggerated, and even if we use the 5 lines you showed and not the ~10-12 lines it really would be.
I don't have exact rules, but if it's (1) quick, (2) simple to read/write in shell, (3) separate from my app's business logic, then I'll use a shell script.
For example, to take a line off of the end of a file, sed is pretty easy:
sed -i '' -e '$ d' file.txt
After a few pipes, I will reach for Python. Regarding line count, after a `if __name__ == '__main__':` block and some helpful docstrings, it ends up being about thirty lines or so. "\N{man shrugging}"
Check out marcel (https://marceltheshell.org). Marcel is a shell that allows the use of Python functions, and pipes Python values between commands. E.g. remove the files/directories listed in victims.txt (one per line):
read victims.txt | args [f: rm -rf (f)]
Parens delimit Python expressions, so (f) just returns the value of f, one of the values read from victims.txt.
Marcel also provides a Python module so that Python can be used for scripting much more conveniently than in straight Python (i.e. without the marcel module). E.g., print the names of recently changed files in the current directory:
import os
from marcel.api import *
for file in (ls(os.getcwd(), file=True, recursive=True) |
select(lambda f: now() - f.mtime < days(1))):
print(file)
Using your editor to drive a shell is a huge win because it: (a) flips the development bit in your brain, and (b) creates a shared history across all your machines.
Combine that with literate programming and you have a `duck-talking` history of every development or outage response tied to what you actually did.
Isn't that just a normal scripting language? What's the real benefit of using this over Node or Python? I suppose the syntax is more aesthetically similar to shell scripts... but I don't exactly see that as a plus.