Hacker News new | past | comments | ask | show | jobs | submit login
Show HN: Exa, a replacement for ls written in Rust (bsago.me)
132 points by cytzol on Feb 21, 2015 | hide | past | favorite | 87 comments



I wonder if it's a good thing or a bad thing that I've never considered, even for a moment, replacing a tool like ls. My first reaction was that it's hubris to think "ls isn't good enough for me, I'll make something better." A bit more reflection has almost convinced me that it's a very good thing that people are always thinking and working on ways to improve things, even something that's taken for granted.

On a tangential note, I learned something new and thought that a few other folks might find this of interest: In the screenshots, we see a file with the name "Licence", which to my American eye looked like a typo. On further investigation I found out that "In British English, Canadian English, Irish English, Australian English, and New Zealand English the noun is spelt licence and the verb is license."[0]

[0] https://en.wiktionary.org/wiki/licence


Indeed.

To address your first point. If we can't even question the most basic of our tooling then we'll just inhabit a stasis that may be just a local optimum.

To address your second point. The way I think these licence/license words should go is - think of advice/advise. The verb's got the 's', the noun's got the 'c'. Am I wasting my time in actually caring about shit like that? Probably. But that's how I'm wired.


Also practice/practise - a doctor's practice / practise the piano.

I don't think it's a waste of time, but language is fluid. Sometimes it changes very rapidly and sometimes slowly, but it's never static. Since the US uses a single spelling for both (and most of the world defaults to using US English) my guess is that the distinction will vanish eventually.

I'm slowly learning not to care about these distinctions when it comes to English, as long as the meaning is clear. The closer you look, the fuzzier language becomes - even in the UK or US alone, there are vast differences in the way it's used.


… and advice/advise and prophecy/prophesy. For some reason, only the latter has kept the difference in spelling between the noun and the verb in American English.


I'm not sure about ls. As for this tool all it seems to add is git status, but it's not working for me for some reason (not sure why) so I can't comment on the usefulness of this.

I do think some other coreutils could be replaced. cp for example, doesn't give you a progress bar so if you are copying a very large file it could take 2 minutes or 2 hours. I suppose you could create a wrapper for cp that repeatedly ls-es the destination directory to give you a progress of the copy.


This has dropped off the frontpage, so I feel better about going off-topic: but you can use rsync instead of cp if you want a progress bar!


Ah yes, I always forget that rsync does basically everything!


Would also be nice if the cp replacement (optionally) checks for available storage space before copying files.


A similar, but more broadly scoped project is uutils/coreutils [0], which is a cross-platform re-write of the GNU coreutils in Rust.

IMO, both the coreutils project and Exa are interesting for the same reason: they provide an example of Rust is one of it's niche environments - low level tooling.

[0]: https://github.com/uutils/coreutils


I find it interesting that so many are disparaging this for not being backwards compatible, etc., when there is simply no need. The title of the post clearly states it's a replacement for ls, so why should it be backwards compatible?

It's not like everything everyone ever makes has to be a perfect fit in every capacity. A guy made a replacement for 'ls' as a fun thing to do in a language he likes. We make toys all the time and if you don't you probably just don't even like programming. The fact that he'd share this with us because he thought other people might be interested is precisely why places like these are good.

We might as well just start disparaging every posted project for not being a solution to starvation around the world if we're going to condemn everything that isn't an immediate step 'forward'. Let people make things they think are neat and try to see the positive side of things instead.

If someone's selling you a product I can see this behaviour being warranted, but when someone is trying to show you something they made because they think it's neat, you don't jump on it as if he's trying to change the world with it.


I think people were in general quite nice to him, understanding the scope of the project. Who uses ls in scripts anyway...? :-)

On the other hand I think it would be great if together we would try to go back to the early attempts of writing utilities that do only one thing and can be chained together with pipes. If ls isn't a good utility for pipes we have to come up with an alternative. You can then pipe it into a frontend like exa and everybody is happy. :-)

I think at hackernews there are a lot of brilliant people who would definitely be able to formulate interesting goals/projects to work on (and which will be of benefit to us all). And no, that does not always need to be an idea with which you can start a new startup.

That ls is one of the utilities that needs more love from us all, I think we probably all agree on. Check your history on how often you used it in the past.


> ...so many are disparaging this for not being backwards compatible...there is simply no need...it's a replacement for ls, so why should it be backwards compatible?

Because unless someone feels like editing every script or tool that makes use of ls, then it cannot be used as a replacement, i.e. You'll have to have both ls and exa on your hard drive instead of just exa. Also, people are used to the options of ls, and backwards-compatibility (in the form of a shell alias) is one less barrier to entry.


> Because unless someone feels like editing every script or tool that makes use of ls, then it cannot be used as a replacement

It can be used as a replacement just fine, just not if you symlink/alias 'ls' to 'exa', which I don't see as relevant. 'fish' is a replacement for bash, but I don't remove bash because of having it and I still have plenty of scripts using bash. It's just that my personal usage is exclusively with fish.

> You'll have to have both ls and exa on your hard drive instead of just exa.

Yes. I don't think this is a problem.

> Also, people are used to the options of ls, and backwards-compatibility (in the form of a shell alias) is one less barrier to entry.

I agree that it is a barrier to entry, yes, but this is clearly a toy created just to have fun with a language and make something that also happens to be fairly useful. It's not meant to take over the world, as far as I can see.


> not if you symlink/alias 'ls' to 'exa', which I don't see as relevant. 'fish' is a replacement for bash, but I don't remove bash because of having it and I still have plenty of scripts using bash.

You're not using fish as a replacement for bash.

You're using them both alongside each other. There is, of course, nothing wrong with this, but it is important to know that that's what you're doing.

Because many of the scripts on your computer depend on bash (or another shell with sh-like syntax) it would be difficult or impossible to use fish as a replacement for bash.

Similarly, it would be difficult or impossible to use exa as a replacement for ls, because ls has options that exa doesn't.


I have replaced bash with fish, actually. To which extent is the question. If I replace it for all of my personal use, that's still replacement. You can pretend that the domain/extent of which you replace something is not important, but this will be a factor no matter what you believe.

I AM USING fish as a replacement for bash. Whether or not scripts that run on my machine use bash or not is entirely irrelevant to the point.

Your reasoning is that since there are things I am interacting with that use ls I won't be able to replace it by actively exclusively using exa. This is the same reasoning that would conclude that if you compile a language to C you have not replaced your use of C by using only this compiled language, since things that you use in turn still use C. You will still need things connected to C on your machine, but to say you have not replaced C for yourself is ridiculous in this example, as it is in yours.

Take the following situation:

Person A is working on a project and says to person B that he has recently replaced the Haskell parts of it with Common Lisp. Person B takes a look at A's repository and concludes that he has not. He still has a tool made in Haskell that he uses for one thing in his project. He didn't make this himself, it's a third party tool.

Now, in this scenario, wouldn't it be fairly sane to assume that while A has not completely eradicated all traces of Haskell in his project, he has still replaced Haskell with Common Lisp? His own usage has been completely replaced with Common Lisp. I think most people would think that is the important part of the statement he made in the beginning.


> I have replaced bash with fish

No, you're using both of them. If you are using scripts which themselves use bash, then you're using bash, albeit indirectly. You may not be using it in interactive mode, but you're still using it nonetheless. If you had replaced bash with fish, then you would not need bash on your computer at all.

Replace (in the context of Unix-like software) means completely replace, as in: no longer needing the original for any purpose.


> Replace (in the context of Unix-like software) means completely replace, as in: no longer needing the original for any purpose.

There is nothing about the context of Unix that makes this use case different than any other. If you yourself always blindly take "replace" to mean "completely replace", then that's fine. I would contend, however, that most regular people see the above scenario precisely like I outlined.

> then you're using bash, albeit indirectly.

This line, however, makes me think that you purposefully ignore the example because with your reasoning we are all using C no matter if we've never seen the language at all, if the case is that a language we use is compiled to C. By this reasoning users of Chicken Scheme and Gambit Scheme are using C, because both of these Scheme variants are compiled down to C.

I find it strange that you would be so interested in setting the limits for what other people constitute "use". I've told you several times that for personal use I've replaced bash with fish. This, apparently, is not enough for you. I made a scenario to illustrate how silly it sounds to say Person A hasn't replaced Haskell.

Someone calls you and you are in your study. Your friend Janet is in the living room downstairs. The person on the phone asks: "Is Janet there?". How would you respond to this question?


> There is nothing about the context of Unix that makes this use case different than any other.

I disagree; the programs which make up a Unix-like system are designed to be modular (even extremely so, compared to {say} Windows), which greatly facilitated the development of GNU, a replacement for Unix.

"As the GNU Project's reputation grew, people began offering to donate machines running Unix to the project. These were very useful, because the easiest way to develop components of GNU was to do it on a Unix system, and replace the components of that system one by one...it was legitimate to use a proprietary package when that was crucial for developing a free replacement that would help others stop using the proprietary package...Today we no longer have any copies of Unix, because we have replaced them with free operating systems." --from https://www.gnu.org/gnu/thegnuproject.html

> With your reasoning we are all using C no matter if we've never seen the language at all

Well, we kind of are. For example: if we haven't replaced all our copies of libc by 2038 with one that supports dates after 2038, then we're all screwed. Whether or not we 'personally' use C or not.

> Janet

Well, skipping over the fact that I don't know anyone named Janet (;-D) I'd say "Janet's downstairs, let me get her for you."


On most Linux systems these days bash is symlinked to sh.


Since Debian/Ubuntu-based systems are probably in the majority, /bin/sh is usually Dash by default.


Sometimes i feel as if these things should be treated similarly to theories in physics. Einstein extends Newton but do not replace his theory of gravity for instance.


Einstein's general relativity does replace/generalize Newton's theory of gravity.


This would make some sense if it were backward-compatible with normal "ls" for basic things like "ls -l". That way, people could alias "ls" to "exa" and not break too many things. But it drops the group and adds Git markers (which are only occasionally relevant), so I don't see myself trying this.


I would not joy until exa is tried over a fs with more than a million files. Huge file numbers are a thing now and it is becoming to be an engineering challenge.


exa can certainly handle a million files. Try running "exa -lRT ~" and watch it produce a tree view of your entire home directory!

However, by 'handle', I actually mean 'run without crashing'... if you run that command, it'll be several minutes before it produces any output. exa aligns its output into a tabular format with every column having the same length, which means it actually has to stat and examine every file in the list before it's able to output the first line. This is annoying, but (as far as I know) unavoidable.


I have not yet looked into the code but if it's using glibc and corresponding getdents code path with suboptimal buffers, it most probably will be slower then ls.

more information about the getdents buffer issue: https://www.olark.com/developers-corner/you-can-list-a-direc...

Again, beware before downvoting me to oblivion I have not yet looked into the code.


I don't even have downvote privileges yet!

I was unaware of the getdents issue - thanks. However, it is most definitely slower than ls, because I've only tested it on large directories, rather than using it with large directories daily, so I haven't found places to optimise it quite yet. I'm not sure if using Rust will let me avoid that buffer issue, as right now I'm just using the function provided in its standard library.


You can call the raw system call using, for example, kmc's syscall! macro: https://github.com/kmcallister/syscall.rs

I do wonder if it's worth it eventually for libstd to speak the syscall interface directly, instead of going through another language's standard library (and whatever bugs or design decisions that standard library might have).


Nope, If you are using libc readdir, gettdents issue is unavoidable.


With my limited knowledge of rust I've looked into it.

Seemingly on unix types of OSes exa uses std::old_fs::readdir, it uses sys::unix::readdir and it uses libc::readdir_r Which would also have the suboptimal getdents buffer issue.


I also wish for perf measurements, cpu, memory usage etc. How shorter is it from GNU ls (even though exa is still new)? Also I wish you didn't attempt to add complex layout to a ls-like program. I don't know why all low-level utilities have formatting logic in them, in GNU ls, IIRC, it was 30% of the code !


Works pretty good! I really like the symbolic-link-preview. But I think it would be better when you just add -l when you enter things like -h, -T, -b, ... or allow an option for that.


Thanks! Years ago I wrote a wrapper for ls that just added colours and did highlighting - but not only was it much slower, I couldn't do things like preview a symlink's path because that information wasn't available in ls's output. So exa was born.

And as for automatically adding --long when you use --tree or similar: this is something I've hummed and haahed about, and I'm still not confident that I've done it the best way, so I'm open to suggestions here. It seems that most tools just seem to ignore it if you pass in two conflicting options, or redundant options, and personally I'd much rather have the user always give correct behaviour than potentially get it wrong without knowing. But I'll just have to see how people end up using it.


Not tried yet. Wonder if it's got the option to display the size of directories. On the screenshots, the directories are shown with '-' just like the case with the ls.


Do you mean recursing into a directory and summing the file sizes of its contents? exa doesn't do this, sorry, but it's something I've been thinking of adding.

ls actually displays the 'file size' of the directory, which I've left out, as that number has never benefitted me, ever.


On ZFS, the number of entries in a directory is reported as the directory's file size in bytes. Pretty nice, but unfortunate if a tool would hide it.


I am using it right now and it is great! My only complaint is that it is a lot easier to type `ls` than `exa` :) (all the letters are on different key rows with the same hand)


If `e` isn't already used by another command or function in your environment, you could do something like this in .bashrc, .zshrc, or wherever you put your aliases:

    alias e='exa -1'
    alias ea='exa -a1'
    alias ee='exa -hla'
Note, that's a number "1" in the first two, and -h here includes the headers, which is different from ls -h, which exa appears to do by default. Other than the one entry per line printing, it's similar to the ubuntu default aliases for ls.


Just symlink it as 'ls' and put it in your path :-)


I think this is actually most interesting as a demonstration of Cargo's support for "features" (http://doc.crates.io/manifest.html#the-%5Bfeatures%5D-sectio...), which is a concept I haven't encountered in other package managers before. Is it something that Cargo inherited from Bundler?


Though it's not a package manager, configure scripts have had this for a while -- e.g. ./configure --no-git. Still, it's certainly cool to have it built into Cargo!


When it comes to Mac system package managers, MacPorts has port "variants" and Homebrew has "options", which are basically the same.


I'd argue that they all derive from, and are quite similar to Portage's USE flags.


It would be useful to re-implement the basic UNIX utilities (at least the ones in BusyBox) in Rust. But without all the extra bells and whistles this program has. Lots of embedded systems need a reliable set of tools.

An exploit was discovered in "files" recently. Somebody put a decoder for Linux executable files in it, creating a security hole.


There is uutils [1], which I believe is doing exactly what you are saying.

[1] https://github.com/uutils/coreutils


What is 'files'?


GNU "strings" may have been meant, as it had this issue recently.


Thanks for clarifying.


Right, got that wrong.


I just hope that it's backward compatible with ls, as of the moment, it's not a replacement.


It's not a drop-in replacement, it's an alternative.


Another alternative is lsp: https://github.com/dborzov/lsp

Unlike exa it fully supports Windows and does not have any external dependancies (as it is written in Go, everything is included already).


Am I the only one who, while I think this is awesome utility and idea, find that unless you are Rust enthusiast, likelihood of using it is very low.

I think there is a need for such utility, if it could be made in more portable fashion. I am not installing rust just for this.


This comment inspired me to write this utility: https://github.com/brson/rustle

It installs Cargo applications without requiring an existing Rust install. Instructions for installing exa are on the README.

Since I wrote it in a few hours, using it may set your hard drive on fire.


You don't need to be a Rust enthusiast to install it if it enters to a repository for the package manager/distro you're using. However, we need an enthusiast (or maybe author) to do that first. It is as if I don't need to be a perl programmer to use ack over grep.


If the binary could be distributed, then there's no reason you'd need to install Rust to use it.


I'd really like for that to happen (compiling to a standalone binary is one of the reasons I picked Rust), but I have no idea how to go about doing it...


You could just distribute the executable you get when you run `cargo build`. rustc / cargo statically link the Rust standard library into the app, so it should work fine on a machine without Rust installed. It'll still dynamically link C libraries.

One catch is that (especially if you're using C libraries other than libc, but sometimes with libc) you'll need to worry a little bit about forwards- and backwards-compatibility of those libraries. It's somewhat safer to find a semi-old distro like Ubuntu 12.04, build there, and see if it works everywhere newer.

This is kind of a pain, and the long-term answer is to get Rust into the distros so that they can build software written in Rust. (This doesn't require rustc being installed on any machine other than the builders.) In the short term, a cross-distro package generator for binary crates sounds like a maybe-fun project....


Isn't it possible to statically link C libraries too?


Not glibc, which is the issue here.


Then not sure how its supposed to distribute Rust programms... Maybe some installer at least..


Basically every system has libc on it already, so it's not a big deal. Unless you build on a new system, and then distribute to an old one. Most people pick which systems they'd like to target, and build against the lowest available version.


OK, yes, that is what I would like to see. Maybe if it would show up in homebrew, then it would be good to install.


Ironically, doesn't Homebrew build everything from source? So in that case it would end up installing the Rust compiler before building the application... What you want is a binary download, it seems.


Although many packages are still source only, Homebrew has an initiative for packaging (or 'bottling') larger tools into binaries for convenience. It can always be overridden with --build-from-source


No, homebrew supports pre-compiled binaries (known as bottles).


Mozilla plans to use rust for parts of its code by the end of this year. This isn't set in stone yet, but I'd expect rust code to be pretty standard soon.


How do you pick the colours? The website uses a dark background. If I ran this tool on my terminal (with a white background), would the colours still be readable?


It uses the colours you pick in your terminal. I made the website dark because I like that colour scheme, and it's what I look at all day, so copying it for the website was a natural fit :)

A few of the filetypes use colours in the 256-colour range, but these should work well too - the grey colour you see for non-existent '-' table cells in the columns view is the exact middle grey value (colour #244) because it's the only one that's guaranteed to work with light-backgrounded terminals.


Forgive me for veering off topic, but would you mind sharing the terminal theme?


Well, it's originally the Tomorrow Night theme, but I've tweaked all the colours to my own liking (such as making the bold colours slightly lighter) that it's not really officially that anymore. But it's a great theme, and the nearest.


This is the sort of thing that Rust is meant for. Correct, well-designed low-level tools. It's not an application language; it's a systems language.


Well, yeah, maybe, but I've never felt like missing something from ls…


xml or json output option would be nice.


As more people use it they eventually push higher up the stack.

People said Go was just a systems language and we've seen it being used for lots of different use cases.


Go is fundamentally flawed as a systems language and should never have been used as one.

In Go, the GC is opaque and non-optional. In rust it's the opposite. Go fails at one of the fundamental requirements of a systems language.

I have not heard many people claim that it really is one.


Go is fundamentally flawed as a systems language and should never have been used as one.

The problem may be large binary size (mostly due to the runtime size and static linking). But I don't see why using a garbage collector is a problem for plain UNIX utilities like ls-replacements.

In fact, if you are running a Debian or Ubuntu system, a lot of utilities are written in Perl or Python.


Just use Go with C or Assembly when its needed, no big deal


You could do the same with Python, but it would be a stretch to call Python a systems language...


It seems awesome, specially the --tree option. I would love to try it, but

    Download and install Rust Nightly for your platform.
    Install libgit2 and cmake using your favourite package
      management system (or pass --no-default-features to the
      next step)
    Run git clone https://github.com/ogham/exa.git to download
      the latest version of exa.
    Run cargo build in the new directory to compile exa.
    Move the resulting target/exa executable into a directory
      in your $PATH, or add an alias for it in your shell.


But what? This is some code that some dude wrote and published courteously open-source, it's written in a rather-newborn language and it is for computer-savvy users. Were you expecting a binary installer?


I wasn't. I am not complaining, just saying that there is too much friction in all this. Happens with every programming language.


That all just takes a couple minutes.


A better ls! I love the git integration.


It is very promising but sadly the alignment of the column mode looks very buggy at the moment.


Actually, I just updated the repo and rebuilt exa and this issue is gone. As soon as the integration with gitlib2 becomes fast enough, this will be a perfect ls replacement.


slightly overkill on the coloring ;)




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: