Hacker News new | past | comments | ask | show | jobs | submit login
Xonsh: Python-powered, cross-platform, Unix-gazing shell (xon.sh)
204 points by ahmedfromtunis on Sept 14, 2018 | hide | past | favorite | 93 comments



XonSH has been my main shell for a solid two years now. The concept is awesome, but unfortunately it has many flaws.

* Nesting bash and python scripts makes performance terrible compared to using python's subprocess module directly. I have a xonsh script at [0], with an equivalent written in straight bash. The python scripts ends in less than a second, the xonsh script takes ages.

  - As a sub-problem of the above: You can't ctrl+c while in a python function in xonsh. That stinks.
* $(program) returns a string that ends with a \n. In bash, when the program returns a single line, the \n gets stripped.

* It sometimes has problems with certain interactive programs. My last experience with this was sudoedit starting neovim in xonsh caused neovim to crash. XonSH -> Neovim works. Bash -> Sudoedit -> neovim works. but Xonsh -> Sudoedit -> Neovim doesn't.

And that's just a small list of my biggest gripes that I keep running into. I really hope to see the project improve though! Despite all the bugs, it's still my favorite shell by a mile.

[0] https://gist.github.com/roblabla/d82c440908d08c8a232ac483e6b...


I use zsh daily but type `xonsh` any time I do something in the shell that would normally require a google.

“Uh ok so I have this JSON file. When I pipe this to `jq` I forget: to pivot an array of objects to an object keyed by array[x].id is it like `to_entries` then chain it to `map`? Uhh let’s look at the docs...”

However in Python I can write that function at the speed of thought. It may be a couple more lines of code but I’m also 100% confident in the output.

Like I understand some people can legitimately program in bash [0]. But I just can’t bring myself to practice. Why would anyone inflict this language on themselves?

[0] https://github.com/docker-library/python/blob/master/update....


> Like I understand some people can legitimately program in bash [0]. But I just can’t bring myself to practice. Why would anyone inflict this language on themselves?

Bash has a lot of pitfalls, but once you learn how to deal with them, it can also be quite superb for some tasks. Piping the output of a complex set of loops into GNU parallel and watching your computer process stuff at blazing speed is a very nice experience. Doing the same in python would be many time more verbose.


> I use zsh daily but type `xonsh` any time I do something in the shell that would normally require a google.

Personally, I stopped using `jq` in favor of python oneliners, they a bit more verbose but also cross-compatible, works out of the box (not everyone has jq, but most of people have python (pre)installed on their machines) and it is easy to implement a oneliner, i.e.:

   curl -X GET "https://httpbin.org/json" -H "Accept: application/json" | python -c 'import sys, json; print(json.load(sys.stdin)["slideshow"]["author"])'
Do xonsh helps you write a tar command without googling [0] or browsing a man?

[0]: https://xkcd.com/1168/


Bash combined with utilities is like a swiss knife of scripting, you won't be doing any major woodwork with it but it sure is a heck of a useful tool.


Filed https://github.com/xonsh/xonsh/issues/2815

$ yes test | tee /tmp/xonsh.log

$ wc -l /tmp/xonsh.log

  208076 /tmp/xonsh.log
bummer ctrl+c does eventually work several seconds later (about 10x slower) which is still a deal breaker as it could be disastrous when you need to abort a mistyped command or for scripts that give you a few seconds to abort before it does something the destructive step...


Sounds neat. However, the fact that `ls -l` either subtracts two numbers or calls a shell command, depending on whether there exist valriables 'ls' and 'l' in the Pythom environment, looks like a security nightmare waiting to happen. What if I create a python object 'ls' deep inside a script that does Evil when subtracted from?


Python variables are scoped as normal, so the security story for xonsh is the same as for Python. We are always open to suggestions and PRs to harden the code base as needed.


It could be worse. It could be a Wolfram shell, in which case 'ls -l' would return 'l(s - 1)'.


What's stopping a Bash script from doing something similar? You could alias ls to a malicious actor for example.


Well, aliases are ignored in scripts usually, but you could prepend something to the PATH so your script/exe gets found before built-ins, no? I agree, I don't see that this is a problem unique to Xonsh.


I mean, that's the underlying concept behind the fork bomb isn't it? With ":() { :|:&}" you're essentially redefining the bash noop to be a function that pipes/forks itself into itself recursively.

It's the reason './' is not in the default path as well (so you can't place an 'ls' in your home directory and have the admin run your command instead of the real ls).


Why not just do the evil directly in the script if you can do that?


Obfuscation minimizes time to discovery? Indirection is a possible technique for obfuscation?


This is a super silly concern.


You


Sometimes I wonder which languages people use for system tools. I mean, it is hard enough that C programs (with dynamic linking) break sometimes, but building a shell on top of Python seems like a particualar bad choice to me (similar to building a package manager with python).

In my opinion, such few-dependency, high-importance tools should linked statically.


I think that anyone who is trying to imagine "the next shell" owes it to themselves to try PowerShell.

It's .NET based and very, very different from the sh lineage, but I absolutely love its ability to use objects instead of strings as the underlying transmission medium between "processes". E.g. ls gives you file descriptors, ps gives you process descriptors, etc.


I was just about to ask if it is available for Linux, but then I saw that the official page advertises its cross-platform properties [1] :D

I know about the different concept of Powershell for quite a while now, but since it is MS tech I didn't expect it to be available for Linux. Maybe I will try someday how it feels, but I am sceptical as the thing I like most about POSIX shells is that everything is just text and I wonder how Powershell will integrate with other *nix tools.

[1]: https://github.com/powershell/powershell


The boundary layer between text and object is definitely a weak spot in the abstraction for PowerShell. That's the main reason why I don't consider PowerShell strictly superior.

That said, the way the language is designed makes calling into compiled .NET code seamless, which in turn makes doing really sophisticated shell scripting a breeze. When I saw what xonsh was trying to do, I recognized a lot of "PowerShell solves this problem a bit more gracefully".


I've heard of it but never heard a compelling reason to try it. Until your statement: I'm now very interested.


> building a shell on top of Python seems like a particualar bad choice to me

can you say why? Python's stdlib handles a lot of cross-platforminess, Python is mostly easy to write, and is available as or more easily than a C compiler on many platforms.


It's about stability. It is much easier to break a dynamically linked or interpreted script than a statically linked binary. The worst case of fixing a system with a software failure is booting a live-cd and replace the files which cause trouble, but a broken shell and sometimes also a broken package manager are precisely those cases when you have to boot a live-cd.


> It is much easier to break a dynamically linked or interpreted script than a statically linked binary

If we’re talking about new code, why is this the case?


Gentoo's portage package manager is written in Python.

https://github.com/gentoo/portage/


And that is probably one of the reasons why projects like Paludis were started years ago.

The funny thing about Portage is though that Python doesn't cause any performance issues as Portage/Emerge is slow by design. So while Paludis is written in C++, it isn't much faster, as it still has to interpret all those thousands of bash-style ebuilds/exheres.

This might sound as if I wouldn't like Gentoo/Exherbo et al. but in fact, my desktop PC is running Gentoo or Exherbo for the last 10 years at least. But it was exactly that experience which made me conclude that system tools like package managers, shells, etc. should be written in statically linked languages.


Love xonsh!! And gets more and more stable with every release.

it's the shell I always wanted. And cross platform to boot.

Alias ls -l?

aliases['ls'] = "ls -l"

Oops no I don't want that any more:

del aliases['ls']

Add dir to path:

$PATH.append('/foo/bar')

Just the awesome python syntax sugar and readbility right in the

May I never right a bash script again.


> May I never right a bash script again.

Bash is not going away any time soon, for better or worse..


del aliases[‘ls’] seems like a step back from unalias ls.


Only if you're coming from a different shell.

For me, it's easier to remember that aliases acts like a dictionary, so I can treat it like any other dictionary. With "unalias" I have to remember the command. As I rarely use unalias (a few times over a period of years), I typically have to search the web on how to remove an alias.

And if it really bothers you, I'm pretty sure it's trivial to define an unalias function in Xonsh that will act as you expect (without needing parentheses).

It's part of the beauty of Xonsh - many such commands you're used to can be brought into it by writing fairly simple Python code.


Sure, but aliases is a real mapping object that you can access and manipulate


no the point is that if you know Python, you know how to do it without looking it up


So I just tried it and it's surprisingly usable (I am always somehow skeptical about non-Bash shells). It even takes into account my ~/.inputrc, so kudos.

However, I struggled a little bit with

$ time ls

where the output of `time` is mangled in-between the output of `ls` - this may be related to STDOUT and STDERR buffering?


I've always been thinking there can be a better shell like having a split pane to show the list of command options with descriptions as you start typing in the command or show snippets of a preset of command options that does task X as in search by use case instead of going to stackoverflow every time.

I think the way people handle shell, as in typing commands on a dumb shell, has not improved for decades and I don't know why. The only thing they help without explicitly asking is with autocompletion.


I really like fish. It doesn't open a split pane, but it can show lists of command switches in tab complete based on man pages.

I'm really tempted to try xnosh again as my daily driver for a few weeks though. I like the idea of typing out arbitrary Python code without spawning up IPython.

The problems people seem to be stating about xonsh were similar to a lot of the early bugs with Fish, but those things tend to go away once more people start using the shell and reporting issues.

I hope we got a whole new generation of usable shells with xonsh, oil and fish.


> a split pane to show the list of command options with descriptions as you start typing

https://github.com/mhayashi1120/Emacs-shelldoc


I love the concept, and I am generally happy with how it works. I think it should catch on.

Except the damned name. Come on guys, there are already enough crazily named things out there, you didn't have to be another one :(


ehh.. I keep wanting to call it "nosh" or spell it "xnosh" but I'm sure if it catches on people will remember. Plus it's easy to use with a search engine, unlike "Go" or "Rust."

I will say I prefer names that can be searched for on their own (e.g. with Go people typically search for "Go lang").

And as much as I love Void Linux, it's better named than all the xbps-* package manager commands.



oh yes! I forgot about nosh, the process manager/supervisor.

:)


nosh is the non-shell script processor.

pen2l will be reassured to learn that the process supervisor is straightforwardly named. (-:

* http://jdebp.eu./Softwares/nosh/guide/service-manager.html


Nosh means something different in the UK... Check Urban Dictionary and you might rethink it as a shell name :P


You know it's a pun on conch shell, right? ;-)


Wouldn't it make more sense if it was called "Ponsh" then? Heck, now that I think of it, "posh" would be a perfect name for a Python-powered shell.


PowerShell went for "pwsh" instead of "posh" because posh is already a name of a shell - https://www.commandlinux.com/man-page/man1/posh.1.html


Damn. Maybe go for "Posh Again" instead, in the bash tradition :)


Perhaps “pysh”, although that has some negative connotations.


That makes it worse. Too many names are based on insider-jokes, too many of it just scares away noobies.

posh, I agree, with the other poster, is a fabulous name for something like this


It doesn't even make sense, since "conch" is pronounced "konk".


A better URL is https://xon.sh/



Note a couple of those use the .org tld. That is not a legitimate site, do not use that.

Ref: https://github.com/xonsh/xonsh/issues/1077


Thanks, we've updated the link from https://github.com/xonsh/xonsh.


Note, while you should feel free to use any interactive shell you like, you should only write scripts with POSIX sh. A friend of mine is working on another shell, mrsh, which aims to be POSIX-sh-as-a-library and will serve as the basis for building more sophisticated interactive shells on a POSIX base, if anyone is interested in that:

https://git.sr.ht/~emersion/mrsh


I have a better idea: don't write scripts in shell language at all. They're terrible at basically everything. Instead, use a scripting language, like Python, Ruby, or Perl. They have control flow that make sense, are not riddled with subtle bugs due to legacy and compatibility reasons, and are simply easier to wield.

Shell scripts are a maintenance nightmare - even when used for personal scripts. They should be avoided at all cost.


Ruby and Python are pretty unwieldy for the kind of situations that are well-suited to shell scripts. It's a pain to spawn a command, a pain to make sure you get the process's output in the right place, and just far more overhead than necessary.

I agree, however, that classic shells are worse, because they lack many of the niceties expected in modern languages and have opaque, sensitive syntax.

fish shell has made me pretty happy. It's a great middle ground. It makes no pretense at POSIX compatibility. Truly a shell for the 90s.


Agreed.

There's definitely a good demand for a modern shell scripting language but fish is an existing solution that is saner than bash/zsh.

I don't use fish as a shell but for shell scripts only as it acted a bit weird on me, though it's possible I didn't search enough but zsh can cover pretty much all I want from fish with plugins.


rc from plan 9 is simple but quite powerful and has very clean syntax. It also does away with the cruft in bourn. http://www.scs.stanford.edu/nyu/04fa/sched/readings/rc.pdf

There is a port of a few very useful plan 9 tools, including rc and the Acme editor, in https://9fans.github.io/plan9port/

https://en.wikipedia.org/wiki/Plan_9_from_User_Space


shell is incredibly powerful when you interact with tools that use piping. Doing the same in python is much more verbose.


Nonsense, shell scripts are an incredibly useful tool for a huge variety of use-cases. Shell can be learned like anything else, and it mostly makes sense when it does. More so than I can say for Ruby and Perl, that's for sure!

You should know when to stop and reach for a different tool, but that point isn't before you use shell scripts at all.


While that's true for portability (there's obviously reasons why nearly all distros ship with a POSIX sh compatible shell at /bin/sh), for simple tasks on my own system, oftentimes I simply want to get the job done without banging my head against a wall dealing with all the idiosyncrasies of sh. For internal use, I think this project excels for automation and administration tasks. Of course, I probably wouldn't ship anything written in xonsh-python, but I'd totally use it to, say, back up my dotfiles to a git repo automatically.


Because many distros ship with /bin/sh linked as bash, I think using arbitrary shells for your personal scripts encourages bad habits that can easily lead to mistakenly writing non-portable scripts elsewhere without realizing it. This has happened to me many times!

POSIX shell is not really worse than bash or any of the other choices imo. For tasks to which it's not suited, bash et al probably isn't either, and you should just use Python.


Sounds amazing. If it ever gets a relatively active plugin ecosystem like zsh or fish, I'd definitely switch. Being able to write shell functions in Python would be very refreshing.

If they want bigger adoption, dedicating a bunch of people who do nothing but port zsh and fish plugins would probably go a long way.


We have a couple of dozen very handy plugins available already (powerline, conda support, etc) and are always looking for more!


Curious: What zsh/fish plugins do you think are missing from xonsh?


Is there a plug-in for fzf searching your command history?


Since I've not used fzf, I don't know how good this is, but there is a "xontrib" (i.e. plugin) for some of fzf capabilities in Xonsh:

https://github.com/shahinism/xontrib-fzf-widgets

You can get a list of (known) xontribs here:

https://xon.sh/xontribs.html


what exactly does "Unix-gazing" mean?


Since the start of the project there has been a desire to point out that most things that work in Bash also work in Xonsh.

The motto started out as "BASHwards-compatible", but that was unsatisfactory because on one hand it sounds like nonsense, and on the other hand, it's not actually compatible with Bash. So that devolved over time into what we have now.

https://github.com/xonsh/xonsh/blame/ebd4e597a3cf063b124e5a5... https://github.com/xonsh/xonsh/blame/82cac3fff08b5cc26e421f1... https://github.com/xonsh/xonsh/blame/bd964ce552282334c881181...


This looks very promising:

  >>> ls `a(a+|b+)a`
  aaa  aba  abba
Regex filename matching triggered by backticks. I like that design choice!


Good that they didn't choose slash delimiters.


Awesome that it uses repl.it embed. But leaves so much to be desired. I'm the repl.it ceo and have a few things in the works to make this experience so much better:

1. caching pypi installs between instances 2. using a real terminal emulator which means you'll get to use xon.sh (or any shell) in it's full glory

Still a nice surprise to see repl.it embedded in the wild.


Your best option for designing scripts that must talk to other programs and also do what you want (as bash won't).


Xonsh is awesome! Execute Python directly on the command line?! Sign me up!

Sadly it doesn't play well with all of my three thousand various vim plugins, so I still haven't been able to use it as my daily shell. Hopefully someday!


Can you please open an issue detailing the problem? https://github.com/xonsh/xonsh/issues I'd love to help make this work. I believe that the emacs folks have something that works for them


If vim plugins are the only thing holding you back you might want to consider just adding

   set shell=/path/to/bash
to your vimrc and see how you get along.


I current have that set in vim when using fish shell.


lol do you really have 3000 vim plugins?


The demo on the front page raises an error when running "ls"


Shameless plug: if you like modern shells that you can use to do real programming, try Elvish too: https://elv.sh


Anybody switched from fish shell to this? What is your experience? I like fish shell because the preconfiguration is pretty good. Same on xon.sh or config hell like zsh?


Ugh. I have so many conflicting opinions on this question.

So, Xonsh is superior in the manipulation of shell objects. I felt super at home changing the shell prompt and configuring/extending the default Xonsh functionality. This is all because it's literally a superset of Python.

Fish however, blows Xonsh out of the water when it comes to autocomplete and navigation. Xonsh's autocomplete / suggestions are very very weak in comparison to fish's. You feel bogged down when trying to navigate directories quickly with Xonsh. I wrote a small blog post about it here [0]

There was a piece of software I was trying to write that was meant to be Xonsh but with Fish's autocomplete [1], but really didn't succeed and have slightly abandoned it for the time being. But maybe I'll pick it back up soon.

[0] https://ezb.io/thoughts/programming/mollusk/mollusk_1.html

[1] https://github.com/enricozb/mollusk


>I wrote a small blog post about it here [0]

I work on fish, and one of the things that the upcoming fish 3.0 contains is a new `math` builtin (previously we shipped a function of that name that just wrapped `bc`).

So your math expressions here are interesting to me, because they work in math (with the caveat that `` is still the symbol for globbing, so it needs to be quoted or escaped):

    $ math 0x113e8b3
    18081971
    $ math '719 * 114679'
    82454201
Adding them to the core syntax carries backward-compatibility questions and also isn't really what fish is about.


The benefits of xonsh are more than just the quick math, but I'm glad fish is getting something like this. I've had a Python script that does math for me, something like this [0]

I do think something like a Python shell is the best solution (for me), however, simply because of the ease of use with modifying the current state of the shell (environment variables, etc).

[0] https://gist.github.com/wackywendell/919042f19ab6932447df


This seems really cool. Trying it out with repl.it was really neat. One thing I'd do is add PowerShell to the comparison table.


what does the "-gazing" suffix mean?


It means that it is not fully sh-lang compatible (i.e. a true unix shell), but is as compatible as reasonably achievable while maintaining full compatibility with Python 3.


What sort of capabilities does sh-lang provide that Python cannot? Does sh-lang have some sort of privileged access to the OS not available to Python?


posix sh mandates certain constructs work. For example:

   x=foo; x=${x%o}; echo $x # prints 'fo'
Supporting that is non-trivial in python. There's no quotes around the strings, '%' is kinda weird there, etc.

Of course, in python you can just say:

   x="foo"; x=x[0:2]
... but those are not compatible with each other. colon does different things in posix sh than python, and it would be quite complicated to reconcile it.

I'll also point out the parent poster, when they wrote "sh-lang", meant "the superset of posix-sh that zsh/bash users expect", because at this point quite a few bash features are expected to work in most sh-like shells.

Naturally, both xonsh and sh (and all other shells pretty much) are turing complete and can execute programs, which makes them all equally capable.

Having equivalent capabilities does not mean things are compatible though. I recommend reading up on what a POSIX shell is, and then reading fish and xonsh's arguments why they're not POSIX compliant (spread through various locations)


He means it's not POSIX-compliant. You can't replace /bin/sh with it and expect stuff to work.


I have wanted something like this for a while. I can't wait to try it out


I tested it out on mobile, I like the ability to script with it in a HLL like Python, will give it a try for sure on desktop.


A shell scripting language written in a shell scripting language. What could go wrong?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: