I agree, but I’m happy to see that Nix, Guix, and the one I do the most work on Spack are starting to gain some traction since they all do basically this. Making it work is not always trivial though, and making it work efficiently is harder. Only in the last few years have I felt like we’re finally getting to the point where this is actually a maintainable model for the majority of software distribution. Here’s hoping it gets the rest of the way!
They are, yes, but there is a very important core difference here that I think is not mentioned.
The big cost about Nix/Guix that puts people off is that it eliminates a human-readable filesystem. The traditional layout is gone or empty, replaced by a tonne of folders under `/nix/` called stuff like `240-572-9837wfgjh234098672-_bash` and you have to just... *trust* that your path and so on will work and stuff will just get found somehow.
You have to let go of navigating your own filesystem.
That's just too much for a lot of people. The filesystem layout is one of Unix's defining characteristics.
Whereas Gobo's goal is the opposite: yes, let us discard the traditional filesystem layout, but let's do it by making it more human readable instead.
You can work out where things are without knowing. There's better isolation. It's like semantic versioning, applied to the filesystem: a semantic filesystem layout, where folder names encapsulate versioning info and are more meaningful than the old 1970s reduce-typing-effort-at-all-costs approach.
Nix says ignore paths, ignore directories, you don't need them, we'll manage that for you.
Gobo says forget traditional paths, here are some better ones that you will find easier and more useful.
For me, that's an attractive proposition.
In real life, it seems that both were too much for most people.
My suggestion would be: why not merge them? Try to bring the advantages of Gobo -- readable, meaningful directory paths -- to a version of Nix.
Instead of a flat directory tree with hashes, a consistent algorithm that categorises apps and puts them in a tree:
/gonix/apps/gui/productivity/images/krita/5.1/
/gonix/apps/console/shell/bash/5.2/
/gonix/apps/programming/compilers/fpc/3.1/
/gonix/libraries/c/glibc/2.39
/gonix/apps/console/editors/vim/9.1
I am totally making these up, you understand, they're merely illustrative examples.
> Instead of a flat directory tree with hashes, a consistent algorithm that categorises apps and puts them in a tree:
> /gonix/apps/gui/productivity/images/krita/5.1/
> /gonix/apps/console/shell/bash/5.2/
> /gonix/apps/programming/compilers/fpc/3.1/
> /gonix/libraries/c/glibc/2.39
> /gonix/apps/console/editors/vim/9.1
This is a nice illustrative example that works well when the only difference between software packages is semantic versions. The reality is that there can be many variants of packages and using something like a hash of package inputs is very practical. Take for example a library called GDAL, which has an insane amount of configure flags. It is quite common for scientific software to be tuned and package maintainers cannot supply a one-size-fits-all solution.
1. Put the human-readable part first because humans see that first.
2. Retain hierarchical layouts. For humans this makes things more manageable and accessible, while for programs, as long as it's computationally-parseable the impact is negligible.
3. Lean in to the hierarchy. If it's necessary to duplicate slightly different versions of dependencies for different programs, then put them inside that program's hierarchy. That's one of its reasons for existing.
4. Embed the hash anywhere a simple regex can find it, no problem; tab-completion can sort that out.
All I am trying to suggest is that a Nix/Guix/successor/replacement type tool could focus more on ordinary joes who find it hard to remember the difference between `/bin` and `/usr/bin` as it is, and make stuff human-readable first and machine-readable second.
@tgamblin beat me to it in a sibling here, but this is exactly why I like spack’s way, it mostly gives you the best of both. Now I need to go finish that PR to make it usable as an actual distro…
The only one that doesn’t quite work out is 3. Dependencies are vendored like that when it’s a patched version that can only ever be used with one package, but in general it helps more to have the regular structure to let them be shared. Otherwise we would have to move them when a new dependent appears and some other kinda funky things. I have played with having each package get a “view” of links to all its deps in its prefix, but it’s a high cost in inodes for an only moderate increase in observability. Pretty easy to generate your own (there’s literally a command for it) so not sure if it’s worth it.
FWIW this is what Spack does, and it uses a store layout like nix/guix. here are some chunks of the spack install tree path. All you really need is the hash, but we provide a bit more.
One reason `nix` uses `/nix/HASH` is because it results in shorter paths and avoids issues with long paths. We use sbang[2] to avoid the most common one -- that long shebangs don't work on some OS's (notably, most linux distros).
>The big cost about Nix/Guix that puts people off is that it eliminates a human-readable filesystem. The traditional layout is gone or empty, replaced by a tonne of folders under `/nix/` called stuff like `240-572-9837wfgjh234098672-_bash` and you have to just... trust that your path and so on will work and stuff will just get found somehow.
Not really trust. Nix and Guix are not magic.
You can still use ldd to figure out what (now exact!) library anything is gonna use. Guix and Nix use the rpath feature.
And your profile directory in Guix is a normal directory called $HOME/.guix-profile . Currently, it has "bin" and "share" and "sbin" and "libexec" and "lib" and so on in there. That is what the end user uses. Entries in those do link to /gnu/store/xyz/bin/xyz (using regular symlinks), yes.
>Instead of a flat directory tree with hashes, a consistent algorithm that categorises apps and puts them in a tree:
One hash represents all the dependencies of that package, too.
One directory name in Guix's /gnu/store encodes both the direct sources and all the dependencies, all the way up to a bootstrap root (the latter of about 250 Bytes of binary). If any of the dependencies of bash changes, the hash will change, too. That also means that you will have multiple /gnu/store/*-bash-5.1.16 in there in a normal system.
But in your proposed case, you'd have to introduce some magical things which figure out and STORE the dependencies somehow differently in the background. Guix doesn't do that. It stores that in the file system, the end (as in directly in the file system, not as in "the system database blob is in file /foo"). No magical extra store.
That said, it would be possible to only change the guix profile layout to be what you suggest, and then still have symlinks to /gnu/store/<hash>-xyz in there. Probably like 5 h of work, and end user programs most likely will work on the first try. Patches welcome.
See guix/profiles.scm for what creates the profile. %user-profile-directory is $HOME/.guix-profile . profile-derivation is the function you want to change.
You can also use guix on top of whatever other Linux distribution and try it out like that. The package manager (which makes the profiles) also supports containers--so you can safely try out whether your change works.
The reader comments I get is that a non-human-readable filesytem (or only marginally so, by ignoring the hashes and looking in the new single-level hierarchy for recognisable fragments of names) is just too high.
They don't have enough of a problem with how things are to choose a whole new method with this very high price.
(Which is the same reason Plan 9 flopped, as I have both written about and spoken publicly at FOSDEM about this month.)
People on the whole have no idea how this stuff works, and they just copy magic incantations from StackOverflow to get stuff to happen. If that doesn't work, then this OS is broken. The end.
He "fixed" it. Systemd now works in WSL2. All those guides for noobs now work. Everyone is happy.
In a world where tools like Flatpak and Snap are proliferating and it's driving deep divisions between Linux distros, if you think the average person struggling with Linux is going to use `ldd` to work out where the dependencies for something live, I'm afraid you are a deep guru who lives on a different plane of existence.
We now have widely-used packaging systems which simply embed an apps entire dependency tree into a package to avoid people having to work out the difference between `apt` and `rpm`. Thousands of terabytes of disk are being burned to make this stuff go away.
> I'm afraid you are a deep guru who lives on a different plane of existence.
I'll take that as an amusing tip of the cap, I hope that is how it was intended. I don't know if I consider myself a deep guru, but if something doesn't run when I expect it to, I do tend to reach for ldd pretty quickly.
Incidentally, lack of systemd in WSL2 was also a showstopper for NixOS, so before the Poettering hire and WSL2's systemd support, some NixOS hackers worked out a chroot solution to get systemd running under WSL2. :)
(I used to use it at work when I still ran Windows, even after the official systemd support, because Microsoft's implementation had some problems)
> People on the whole have no idea how this stuff works, and they just copy magic incantations from StackOverflow to get stuff to happen. If that doesn't work, then this OS is broken. The end.
I don't think small, weird Linux distros like NixOS or GuixSD or GoboLinux really have the resources to support careless users of that kind, or stand to gain much from them. Projects like that are seeking contributors more than they are seeking users at all!
Is this a tragic flaw? Or is it just a mundane fact about a project that can still thrive in its niche?
PS: Any idea why Microsoft 'needed' a special init system for WSL2 anyway? Full fat systemd systems boot in like 8 seconds anyway! WSL's imitators on macOS (e.g., Lima, Orbstack) work with regular Linux distros and just leave the init system intact.
> PS: Any idea why Microsoft 'needed' a special init system for WSL2 anyway? Full fat systemd systems boot in like 8 seconds anyway! WSL's imitators on macOS (e.g., Lima, Orbstack) work with regular Linux distros and just leave the init system intact.
My slightly educated guess would be on the point that in WSL2 distros, those do share the same running kernel, the same network stack and probably couple of other things under the hood (something about binfmt?).
From the product perspective, booting another instance (say I have Ubuntu 22.04 and 18.04 both available) in ~ 1 second, can be seen, and even proved with some A/B testing, as a huge advantage.
PS C:\Users\coolcold> wsl -t 'Ubuntu-18.04'
The operation completed successfully.
PS C:\Users\coolcold> wsl -l -v
NAME STATE VERSION
* Ubuntu-22.04 Running 2
Ubuntu-18.04 Stopped 2
PS C:\Users\coolcold> netsh wlan show interfaces|wsl -d 'Ubuntu-18.04' -- fgrep Mbps
Receive rate (Mbps) : 390
Transmit rate (Mbps) : 390
the call for `wsl` is almost instant even for stopped instance
> From the product perspective, booting another instance (say I have Ubuntu 22.04 and 18.04 both available) in ~ 1 second, can be seen, and even proved with some A/B testing, as a huge advantage.
For me at least, they went to more trouble than it's worth for that ~6-7 second gain, given that a normal systemd distro will boot in 7 or 8 seconds anyway. Maybe the difference is bigger on spinning rust. But when I used WSL regularly I was in one VM all day every day. It was always running, so I really wasn't worried about the 10 seconds it took to start up or whatever.
At the same time, their custom init setup caused compatibility issues with my distro of choice and some applications (somehow). So it didn't come for free.
> I don't think small, weird Linux distros like NixOS or GuixSD or GoboLinux really have the resources to support careless users of that kind, or stand to gain much from them.
You're missing the point. It's not what the distros need from the people. It's what the people need from the distros.
Linux software packaging is junk. It's vastly over-complex, it's horribly fragile, there are multiple rival systems (apt, rpm/dnf, pacman, apk, etc. etc.) and none work with each other and none is a complete answer, and as a result, there are now 2nd level schemes on top of that (appimage, snap, flatpak, docker, etc.) and those reproduce the problems -- they are incomplete, incompatible, etc. -- and they reproduce it with a level of bloat that makes their packages 100x bigger and 100x slower to download and update.
Android just makes this work and has several _billion_ active daily users, who rarely have big issues. I've never heard of anyone "bricking" their phone installing an app. It's more reliable than Windows with 100x as many users.
That is what Linux needs, and it needs it yesterday, lest it become a weird way of packaging apps for Windows boxes. ("Oh, yeah, that? You need to install a wrapper thingie first, then it will work. Install this wsl thing and it'll run.")
We need a better answer. Nix is better in many important ways: it solves these problems, but it does so at a cost, and the sort of people who like Nix don't realise, or don't care, that the cost is terrfying for ordinary mortals.
Gobo survived 20y of neglect by being better in different ways. It makes the existing filesystem less cryptic.
Nix throws it away and tells you that you don't need it.
Better to say "hey, look, we give you a better filesystem" than to be that bearded mystic on a mountain saying "for true enlightenment, you do not need a filesystem".
> Any idea why Microsoft 'needed' a special init system for WSL2 anyway?
Um. Have you used it? This is _not_ just a VM.
It seamlessly extends Windows so that Windows can run Linux binaries.
It is not another OS in a box.
I'm not saying it's perfect -- it is not -- but it's about 20 years ahead of simple VM solutions.
And it does that by integrating with the guest OS via a special init system.
Yes, I used it every day at work for most of a year! Its showstopping bugs (a killer memory leak, freezes after suspend, data loss issues (!!), native systemd support making Emacs freeze somehow (???)) are a big part of why I finally took the leap to macOS at work.
How much do you use WSL? Because its bugs go well beyond rough edges and those reliably show up if you actually use it for most of your work every day.
> This is _not_ just a VM.
Sure it is! It's a VM with the following features/integrations that start working without any manual configuration:
- nice management interface that can download distros for people
- filesystem sharing via 9pfs, which is cool but unfortunately too slow for serious use
- a pair of command-line interfaces for invoking commands on each side from the other
- this doesn't involve the init system btw. it just uses good old binfmts on the Linux side with the Linux-side wsl command, just like some users do with wine for Windows binaries or qemu for RISCV Linux binaries
- automatically forwards network traffic between the Windows host and Linux guests so that users don't have to manually configure port forwarding
- a shared display server (Wayland implementation) on the host side and some plumbing to automatically forward its sockets to guest VMs
I'm not sure I'd wanna count this, but some third-party applications also do some socket forwarding tricks (and did so before Microsoft's Wayland implementation was a thing) so that guests can share one Docker implementation, like Docker Desktop, Rancher Desktop, Podman Desktop, etc.
That's it. There's no other magic.
> It seamlessly extends Windows so that Windows can run Linux binaries.
I wouldn't characterize that feature as seamless— in fact it's so bad that you can't reliably pipe the output of WSL commands evoked from the Windows side into a Windows-side pager or clip.exe (sometimes wsl.exe would just hang or eat the output). And then there's the PATH translation issues, which often have to be handled quite manually with subshells calling cygpath or wslpath or whatever.
That particular feature is so bad that I hardly ever used it! I'm astonished to hear you call it 'seamless'. Have you ever tried to share a Yubikey for SSH auth and GPG encryption between the Windows host and Linux guests on WSL2? How 'seamless' was that for ya?
> I'm not saying it's perfect -- it is not -- but it's about 20 years ahead of simple VM solutions.
The things I compared it to (Lima, Orbstack) have the exact same features without ever having replaced systemd. Lima uses cloud-init as the interface for injecting its startup hooks. Idk what Orbstack does. Lima is definitely jankier than WSL2 (which is saying a lot, unfortunately), but Orbstack seems solid.
Anyway, both of them reproduce the port forwarding, filesystem sharing, and command forwarding that WSL2 has without replacing systemd, and Lima is older than the systemd integration for WSL2. Have you used either tool? There's no way WSL2 is even one decade ahead of either, though it might be a couple years ahead of Lima.
Me, no, I have only played around with it. From a technical level, I personally preferred WSL1, which I thought was more elegant. I am happy to concede your points as it certainly sounds like you have more experience with it than me.
WSL2 is just a VM, yes, I totally agree -- but a much better-integrated one than most non-techies could ever hope to achieve, and still better than most techies could achieve unless they really knew their stuff and they were competent in both of 2 different OSes.
From what I've seen over my working life, I'd say about 0.1% of Windows users would have the level of knowledge needed, and possibly more Linux ones -- but they mostly wouldn't want it or care.
My personal impression is that it's an attempt at the classic MICROS~1 "embrace and extend" manoeuvre on Linux.
No, I've never tried the Mac tools you mention, and I don't own a Yubikey or anything like it. I try new distros almost daily on my Mac but I have no need of any integration -- I am not a developer. I review distros sometimes, though, and occasionally things like hypervisors:
> From a technical level, I personally preferred WSL1
Same. WSL1 was ambitious, wild, and incredibly impressive despite the flaws that eventually convinced Microsoft to start over with WSL2.
> WSL2 is just a VM, yes, I totally agree -- but a much better-integrated one than most non-techies could ever hope to achieve, and still better than most techies could achieve unless they really knew their stuff and they were competent in both of 2 different OSes.
Agreed! WSL2 has a very impressive OOTB experience in terms of getting you from nothing to a VM that you can use. It would be a lot of work to set up comparable integrations yourself.
> My personal impression is that it's an attempt at the classic MICROS~1 "embrace and extend" manoeuvre on Linux.
Unfortunately I can't disagree. People seem to think that MICROS~1 is dead, but as far as I can tell they're in a very similar monopoly position with more or less the same interests now as they've ever had.
> No, I've never tried the Mac tools you mention, and I don't own a Yubikey or anything like it. I try new distros almost daily on my Mac but I have no need of any integration -- I am not a developer. I review distros sometimes, though, and occasionally things like hypervisors:
In that case, Orbstack and Lima might be tools worth your writing, the same way hypervisors and virtualization apps are! They're attempting to be WSL2-alikes but they don't use their own hypervisors (Lima supports qemu and Apple's virtualization framework, idk about Orbstack.)
> The big cost about Nix/Guix that puts people off is that it eliminates a human-readable filesystem. The traditional layout is gone or empty, replaced by a tonne of folders under `/nix/` called stuff like `240-572-9837wfgjh234098672-_bash` and you have to just... trust that your path and so on will work and stuff will just get found somehow.
> You have to let go of navigating your own filesystem.
Your broader point makes sense in that I get how a user-navigable filesystem is attractive, especially in the context of store paths being exposed to users as they are embedded in binaries and scripts. But I think the above is an overstatement.
Stuff on NixOS gets on your path the exact same way as it does on any distro: a bit of shell! Users can read through the `set-environment` script and the config NixOS generates for their shell in /etc and all that. They can check what binaries are on their PATH with `which -a` as usual. They can ask their shell where a given variable was defined using the usual tools it offers.
And the human-readable part of the filesystem does exist, just on top of the Nix store. The symlinks under /nix/var/nix/profiles/... or /run/current-system/sw are about as human-friendly as Unix ever is, imo.
Perhaps NixOS could do more to make those 'official', foreground them and document them, and even provide more loosely versioned paths to them like you suggest (making it easier to impurely manually build against them).
Secondly, from my position as a tech writer (for about 25 years) overlapping for well over a decade with some 20-odd years as a software tech support person and trainer, I think you are dramatically over-estimating the level of understanding of the average Linux user.
A Linux user can be expected to be a bit more clueful than a Windows user, but not much. They probably understand the concept of a "directory tree" and know that there are "files", some "executable", in "directories", one of which is "current", and stuff like that, but you can not rely on them grasping advanced concepts such as "search paths" and "links" (of different kinds, no less!)
The idea that there's a second, "more standard" tree, and it's optionally overlayed on top of the flat software-managed Nix tree, made up of symbolic links, and that you don't need to know where things are because the software magically makes sure that the OS can find them... I'm sorry but after coming up on 40 years in tech, my reaction to that might be captured by the crying-laughing emoticon. You could no more explain that to the average distro-hopper than you could explain it to a Labrador dog.
How it gets set? Don't care. A script is what I'd assume, sure, but that's no more important than the colour of the script's author's hair.
I'm saying that, professionally, as a writer, I have now presented the concept to smart interested users that they let go of the filesystem hierarchy and just let the OS manage that, and I'd say about 9 out 10 of my readers responded "oh to h3ll with that! No WAY!"
It is a step _much_ too far. It's comparable but worse than self-driving cars to these folks.
To refer to memes again, you need the god-level intellect step of the intelligence meme:
https://imgflip.com/i/45rmf2
... to grasp the concept that you don't need to know where files are so long as the OS knows.
This is not one step too far. It is a two-year backpacking trip across all of Asia too far.
> Secondly, from my position as a tech writer (for about 25 years) overlapping for well over a decade with some 20-odd years as a software tech support person and trainer, I think you are dramatically over-estimating the level of understanding of the average Linux user.
This is easy for me to believe :)
> you can not rely on [Linux users] grasping advanced concepts such as "search paths" and "links" (of different kinds, no less!)
Those are required for understanding how conventional Linux distros work, too, though. So the users who won't understand $PATH on NixOS are also users who don't understand how $PATH works on Ubuntu.
I will admit that there's a difference here in that you can have a very half-assed understanding of Ubuntu, and even a half-assed understanding of NixOS probably has a higher skill/knowledge floor.
And maybe I've unthinkingly assumed that half-assed understandings are worth less than they really are, at least psychologically. I have a very half-assed understanding of how my car works, but having it not be a complete wonder and mystery to me does give me more confidence in driving the thing— even if most of my knowledge of how it works is too incomplete to be practically useful and I'll always take it to a specialist if it has problems.
> You could no more explain [the Nix symlink forest] to the average distro-hopper than you could explain it to a Labrador dog.
Maybe the average distro-hopper matters because the average Linux user is a distro-hopper. But I think distro-hoppers are kind of a pathological case here, because distro-hopping is precisely the habit of throwing your hands up, blasting away your whole setup, and clicking Next Next Next instead of learning how your system works. Of course distro-hoppers are going to struggle to understand how their distros work! To distro-hop is to enact a commitment to not understanding your distro!!
I also wonder if this is really true, and this is probably the part where I am just naive. But a forest of symlinks would be super easy to diagram perfectly faithfully, and I think a visual aid like that could go a really long way for the average person.
> the concept that you don't need to know where files are so long as the OS knows.
Have you seen this article, posted to HN to some infamy, about how zoomers don't expect to know/care where their files are, and are in fact somewhat baffled at the very question of where they've saved something?
I what that generational gap says for the viability (in terms of user acceptance) of the Nix store in all its ugly glory in the future.
I want to be clear that I do think some of your suggestions for laying out the files differently (like Spack apparently does) are perfectly reasonable. There are technical and historical reasons that the store is hash-first, but maybe they're worth re-evaluating. Sometimes I think the real winner, some day, in terms of executing this idea beautifully and in a user-friendly way, will be on a 'brand new' OS where a different executable format and/or a different kind of filesystem can mean the Nix store or its equivalent doesn't need to be exposed as bare files at all. One Nix hacker has actually proposed doing something like that on top of the Linux kernel, but I think RedoxOS already uses some ideas like this.
> Of course distro-hoppers are going to struggle to understand how their distros work! To distro-hop is to enact a commitment to not understanding your distro!!
Fair point. :-)
> I what that generational gap says for the viability (in terms of user acceptance) of the Nix store in all its ugly glory in the future.
I think there's a word, or several, or maybe just a letter or two, missing here.
But yes, I thought of that, too.
What irks me a little is that the Nix folks I've spoken to seem to feel that they have simply fixed this issue now, and nothing more need be done. While in fact I think there is a lot more work to be done here, and maybe GoboLinux and Spack and things show the way.
The (to me) madness of things like Btrfs volumes, layering another filesystem namespace on top of the Unix one, allowing insanity like multiple distros in one partition:
Hints at other approaches. As in: why not both? Maybe it's possible to have multiple _views_ of a filesystem, so via one API you "see" a flat namespace with hashed directories, and with another a conventional hierarchical one -- just as a Gmail account can be viewed as flat and hierarchical at the same time using an IMAP client.
> Maybe it's possible to have multiple _views_ of a filesystem, so via one API you "see" a flat namespace with hashed directories, and with another a conventional hierarchical one
Spack supports the notion of arbitrary "views" of the store, which can be defined declaratively [1]. Apparently we need to write more highlight blog posts and submit them here b/c people don't seem to know about this stuff :)
For example, if you want to make a view that included only things that depend on MPI (openmpi, mpich, etc.), but not things built with the PGI compiler, in directories by name, version, and what compiler was used to build, you could write:
That puts all your zlib installs in a short name-version directory, most things in directories named by the package and version and a subdirectory for what compiler it was built with, and for packages that depend on MPI it also adds the MPI implementation (openmpi, mpich, etc.) and its version to that.
You can choose to map everything into a single prefix too:
view:
default:
root: ~/myprefix
That is useful for what we call a "unified" environment where there is only one version of any particular package -- it basically converts a subset of a store into an FHS layout.
There are options for choosing what types of links to use and what types of dependencies to link in, e.g., you could exclude build dependencies if you didn't want the cmake you built with to be linked in.
> Apparently we need to write more highlight blog posts and submit them here b/c people don't seem to know about this stuff
100% you do. I find the term a bit odd, but I've been a "Linux professional" for coming up on 30 years now and I've never even heard of this stuff before.
There is _so much_ in the Linux world that is just badly explained, or not explained, and it's very hard to find cogent explanations. Trying to find them, summarise them, and in some cases, just write them myself is a large part of my job...
And it's a huge field.
One of the reasons some things dominate the headlines is that the creators spend a significant amount of time and effort on outreach. On just talking about what they are doing, why.
That's why there are tonnes of stories about GNOME, KDE, systemd, Flatpak, Snap, Btrfs, ZFS, etc. They talk. They explain.
It's also why there are next to none about Nix, Guix, Spack, and legions of others. They don't.
To pick a trivially small example: Nix talks about being declarative, but it doesn't explain what "declarative" means. And it mentions "purely functional" but it doesn't explain that either. Those things right there are worth about 1000-2000 words of explanation per word. Omit that, and the original message becomes worthless, because it becomes insiders-only.
As it happens through decades of reading about Lisp and things, I more or less get it, but not well, and I struggle to explain it.
> I think there's a word, or several, or maybe just a letter or two, missing here.
Oops, I certainly did accidentally a whole word there ;)
> What irks me a little is that the Nix folks I've spoken to seem to feel that they have simply fixed this issue now, and nothing more need be done. While in fact I think there is a lot more work to be done here, and maybe GoboLinux and Spack and things show the way.
I think you're probably right. I'm sure there are others in the Nix community who agree, but presenting a facsimile of a normal filesystem, or reversing the decision about the order in which to put hashes and the human-readable part of filenames (which has compatibility and performance implications that others will fight against it for), these things are pretty far outside what most people who love and contribute to Nix care about. And I don't mean that to say that Nix contributors are aloof, but to emphasize that the people on the Nix core team and other contributors to the ecosystem all have long laundry lists of features that solve problems that are deeply interesting to them or professional vital for them/their company. And those laundry lists don't include backwards-incompatible, far-reaching changes for the sake of what it feels like for newbies and casual users to look at the filesystem on NixOS for the first time.
I kinda suspect that if this changes in the Nix world it will be because Spack's approach becomes popular and widely praised for a long time, so that there is a long period in which the change is seen as an obvious choice despite it seeming like a 'minor' issue to technical stakeholders. But I can picture it happening.
> Hints at other approaches. As in: why not both? Maybe it's possible to have multiple _views_ of a filesystem, so via one API you "see" a flat namespace with hashed directories, and with another a conventional hierarchical one -- just as a Gmail account can be viewed as flat and hierarchical at the same time using an IMAP client.
GoboLinux does have a mechanism like this, IIRC, called 'GoboHide'. So if you run ./configure && make && make install on GoboLinux, autoconf and automake or whatever will find a /usr/lib, and if you run ls against that dir, it will succeed. But you won't see it with `ls /`. NixOS could add this kind of feature, but I'm store paths will still leak out in the settings screens of applications and the outputs of various commands, so it's not quite what we're looking for. That latter problem seems impossible to fix without patching applications, because hard coding store paths in them is an important part of how Nix works. That suggests to me that maybe the Spack way is better— at least then leakage of store paths isn't as shocking to users.
> madness of things like Btrfs volumes, layering another filesystem namespace on top of the Unix one, allowing insanity like multiple distros in one partition
Have you played with Bedrock Linux at all? That might be a fun one. That distro layers other distros together into a single functional whole. So in addition to living on a single partition, you also might have X11 from one distro, systemd from another, and command line utilities from a smattering of 3 others, all running as one system!
> Or installing Windows on Btrfs
I have also tried once to install macOS on ZFS, which apparently can be done. I might try again soon, since I have real Apple hardware to do it on. Last time I tried that on a Hackintosh but quickly ran up against my lack of macOS knowledge.
> these things are pretty far outside what most people who love and contribute to Nix care about.
I suspect you're right. :-(
I have sympathy for the "servers as cattle" approach, but then again, the Linux world is in danger of forgetting where it comes from: from enthusiasts having fun. From people hacking around, learning, exploring, for a laugh, partly because it's free.
And as a vegetarian since my teens, but a sysadmin since my 20s, I have more sympathy for cattle than sysadmins. Sysadmins have choices, and can learn from their forerunners' mistakes. If they don't: tough. Sympathy gone.
I know about GoboHide, yes.
> Have you played with Bedrock Linux at all?
I know of it, but I haven't. I should.
> I have also tried once to install macOS on ZFS, which apparently can be done.
Well, could once. Probably not any more: Apple backed away from ZFS about 15 years ago or so.
It installs and creates filesystems just fine. The only part I haven't successfully tested myself within the last year or two is actually booting from it.
> I have sympathy for the "servers as cattle" approach, but then again, the Linux world is in danger of forgetting where it comes from: from enthusiasts having fun. From people hacking around, learning, exploring, for a laugh, partly because it's free.
One thing I believe deeply is that no one ever really masters the Unix environment without living in it. Without a solid foundation of interactive use, programming with the shell as your REPL, puttering around your filesystem with `cd`, you'll never properly learn your way around. 'Learning Linux' in a 'cloud-native' context tends to mean being alienated from all of that, and I think it profoundly hampers the development of fluency and comfort with the system.
To get that from a box (server or desktop), it has to be a home to you, and homes aren't disposable (the Impermanence framework for NixOS aside).
I do think NixOS has a place there, in keeping computing fun and exciting. For me, it was the first truly interesting distro (at least among those I'd consider usable/practical for me) in a long time. (This was back in 2015 or so.) I was a desktop Linux hobbyist before I took up any devops-related work, and for several years, NixOS singlehandedly breathed new life into that hobby for me. As far as I can tell, this is not an uncommon reaction to NixOS among enthusiasts, if not among casual Linux users.
I knew it was around -- I have written about ZFS several times -- but if it's not in the kernel I didn't think you could boot from it any more, and in recent versions of macOS you can't readily add new kernel extensions any more.
(It is possible, by disabling some of the security settings, but it's very much not trivial.)
> no one ever really masters the Unix environment without living in it.
An interesting take. I can't really argue with that.
> NixOS has a place there, in keeping computing fun and exciting
With Nix /run/current-system/sw/ is basically the /usr of mutable FHS distros, with the difference that it's all read-only symlinks. So you can still look at the layout of your active system.
Homebrew has been doing it for a long time. It's not known for being fast, but without profiling the application I'm hesitant to blame the file layout.
