This ended up breaking the entire install. After a few hours of troubleshooting I found that the reason it broke was that it updated itself from version "nix-2.3.10" to "nix-2.3.10-x86_64-unknown-linux-musl" because it saw that package's version string as a version bump. The suggestion in the github issue was to instead use an unofficial third party package for basic package management because this was a known, long-standing issue that is not likely to be fixed.

https://github.com/NixOS/nixpkgs/issues/118481

The experience came across as a massive red flag and I decided not to pursue it further.

nix-shell lets you dip your toes in the water, without learning the Nix language. Amazing value added... but sometimes it's annoying that it forgets that package you used yesterday.

nix-env lets you "install" software, without learning the Nix language. Great, now I can use that Nix package repeatedly without having to nix-shell it. We're still better off than "apt install" at this point, and it's no harder than that.

Everything else -- nixos config, home-manager, flakes, overlays, etc. -- requires Nix language knowledge, and non-superficial comfort level with the Nix data model (nixpkgs, attribute sets, and a very long tail of other things).

Nix-env is a beneficial stepping stone for converting dabblers into diehards, and is one of the Nash equilibria for some Nix users. (i.e., not everyone wants to learn the language and drink the koolaid, they just want to enjoy some additional benefits beyond nix-shell.)

Honestly, if people just used nix for `shell.nix` in projects, it's still major value added

- mkdir myproject ; cd myproject

- nix-shell-env init # creates an "empty-ish" shell.nix

- nix-shell-env install nixpkgs.whatever # adds dep. to shell.nix

- nix-shell # activate it

So you don't have to learn any Nix-language / nixpkgs ropes, but can still have a viable shell.nix built for your project. I could see that being a nice stepping stone between "nix-shell -p" and "learning the language."

http://chriswarbo.net/git/nix-config/git/branches/master/ove...

You could theoretically use Nix for managing user apps too, but that's a niche use case and (as you found out) not in a working state.

    {
      pkgs ? import (fetchTarball "https://github.com/NixOS/nixpkgs/archive/cd63096.tar.gz") { }
    } :
    with pkgs;
    let
      mypackage = callPackage ./mypackage.nix { stdenv = stdenv; python = python3; };
    in
    dockerTools.buildImage {
      contents = [ mypackage ];
    }

(Dev environments work in a similar fashion except with mkShell instead of dockerTools.buildImage.)

I haven't used Nix flakes but I think it develops this simple idea further.

I'm a big believer in the Nix approach to builds, and that this approach holds a lot of potential for DevOps (especially CI/CD); however, there are so many practical problems with Nix and the community doesn't seem especially interested in fixing them. Of course, what volunteers spend their time on is there prerogative, I could volunteer my time rather than criticize, shame on me, etc but it's a loss for the broader world. Personally I would really like to see a community with the ethos of the Rust community pick up the torch and try to build something inspired by Nix, but perhaps with more pragmatic choices (e.g., instead of inventing an obscure scripting language, they might use something more familiar and thus easier on new users; rather than the chaos of nixpkgs, they might give users a type system and/or good editor tooling; rather than the difficult-to-Google "Nix" label, they might pick something a bit less overloaded; etc).

Unfortunately, I'm not aware of any efforts to add (optional) typing to the Nix language.

[1]: https://github.com/tweag/nickel

It kind of looks like they're trying out different interfaces to the same basic core functionality and seeing what catches.

My personal gripe is that wrappers for languages that have their own package managers are abysmal. But griping about Nix on HN is just extra publicity. Now we're just waiting for someone to explain it in a sufficiently succinct way for the crowd.

Come to think of it, that's kind of like Monads in Haskell a few years back. Anyone remember the series of watered down explanations that failed to address the actual problem people were having, which was that that people were confused by the notation, if I recall correctly.

Nix's roots are close to Haskell turf, so it comes as no surprise they're struggling with explaining their abstractions, resulting in the series of mutually overlapping "official" ways to do things (none of which, incidentally, is the "obvious" way if there is one). After all, Nix is a build tool and not a whole language, so they get to try different CLIs and config APIs within with less overhead than experimenting with a language's syntax.

You can also stub the hash (e.g. with lib.fakeSha256). nix-build will fail with a hash mismatch and tell you the correct hash.

I mentioned elsewhere that I spent a weekend working with the Nix discord to try to install VS Code with some extensions for Rust development,

That's tricky with e.g. IntelliJ as well. For development, I now just use rustup and use nix-shell/nix devshell to define native dependencies. Outside development I do full Nix builds with crate2nix or Naersk.

In the past I've tried to package a simple Python app thinking it would be a simple reference to a pypi package, and I ended up having to package a never-ending matrix of the most obscure C dependencies and ultimately gave up.

Python in Nix is painful for various reasons. One is that since the Python interpreter cannot handle multiple versions of a package, nixpkgs can only have one version of each package. However, since many Python packages do not follow semver, it often requires a lot of patching. And then there are issues like Python packages writing directly to their package directory (which does not work with a read-only store). I think packaging Python packages with Nix is worthwhile, but for Python development I think it is easier to just build an FHS environment and use regular Python packaging tools (pip, Poetry, etc.).

This is an issue with some other language ecosystems as well, their packaging models are incompatible with Nix.

This results in a combinatorial explosion of inefficiently grepping around the codebase just to get the type of a symbol*

nix repl* has helped me in many cases. But I agree that Nix would be so much nicer with static typing.

however, there are so many practical problems with Nix and the community doesn't seem especially interested in fixing them

I think most of us are interested in fixing problems. It's just that nixpkgs is large and people have deployed various Nix versions. So, it is quite hard to make fundamental changes to Nix (e.g. typing) and roll them out. Things are happening, e.g. flakes provide a standard interface to package sets, there is work on supporting content-adressed outputs (outside fixed-output derivations), but with the size of the Nix ecosystem, these things take a long time to crystallize and roll out.

I get the perception that people think Nix is a panacea for common CI/CD problems (and to be clear, I think it could be such a panacea one day) but rather it solves some problems but introduces many more, harder problems--which no doubt individually have workarounds and so on, but finding them probably requires intimate knowledge of Nix and the ecosystem that you're trying to package. Unfortunately, I can live with suboptimal builds and a little nondeterminism/impurity, but I can't be dead-in-the-water because I can't figure out how to package a dependency that my app depends on.

So I'm rooting hard for Nix, but I want to share my experiences so potential users know what they might run into and so any Nix contributors know a bit more what users are running into (if it's not already known).

> I think most of us are interested in fixing problems.

I didn't mean to imply you weren't interested in fixing any problems, but rather the problems that I care about, which have persisted since I tried Nix for the first time circa 2014.

> So, it is quite hard to make fundamental changes to Nix (e.g. typing) and roll them out.

Surely gradual typing would be relatively easy to roll out, as the name implies? Similarly, documentation improvements could be rolled out pretty easily (e.g., put beginners on the happy path, don't recommend nix-env, tell people about homemanager if that's the thing they should be using, etc). I mean these things in the spirit of constructive feedback--if you want to court new users, these seem like low hanging fruit.

> Things are happening, e.g. flakes provide a standard interface to package sets, there is work on supporting content-adressed outputs (outside fixed-output derivations), but with the size of the Nix ecosystem, these things take a long time to crystallize and roll out.

I'm glad to hear things are moving--I'll need to read up on content-addressed output and flakes!

Are you kidding me?

The product had an external license file you'd need to drop alongside, and on boot if your licence key didn't match for your name + connection settings, the retail build would spit out both the key in the file, and the expected correct key for your settings.

It was not a priority fix for them, either. At least this one is merely informational, and not a potential gate on revenue.

The hash requirement is kind of a chicken & egg problem, because to fetch a file you need the hash, and to get the hash you need to fetch the file. So this gives a way to escape it.

There's also the "proper way" to use the prefetch command (I personally like the nix-universal-prefetch) which basically fetches file to the cache and outputs the hash, but in that case it is one extra command which you need to figure arguments for, so the former method is just faster.

This can be address by adding the version to the dependencies store path. This has been done for rust builds, not sure about go.

One of the big wins for Nix is its declarative nature (especially for reproducibility) and installing packages in an imperative manner like this just seems to put folks back in the same position as using pacman or apt-get. Please correct me if I'm wrong Nix-ers.

Having used NixOS as the main OS on my personal laptop for the past year or two (and loving it), I've only used `nix-env -i` to install a package once, and immediately regretted it after trying to add a different version of the same package to my main configuration files. This resulted in a conflict, which from memory required me to force reinstall the package again via `nix-env` but specify a lower priority so that it could be replaced automatically when enabling the package via my configuration files... A really strange experience that re-affirmed that nix-env is at least not for me.

For testing packages, I would always recommend `nix-shell -p <package>` as a preferred option. Just type `exit` and then you've left the shell, it's no longer in your PATH and there's no risk of conflicts.

It sucks that you ran into this, but also totally understandable. I think generally the community would consider the nix-env approach of installing and updating packakges an anti-pattern these days, but this certainly isn't obvious when reading through the docs.

It being similar to the other commands means we can use nix almost no learning curve, only having to know the equivalent commands. It's the beginner's interface, allowing you to get things done right away, and letting you postpone the complexity until you have need for the features/stability it provides.

For example, I originally started using nix because I needed a Java version not in Ubuntu's repo, and being able to say "just install it" instead of having to learn a whole new specification language is a massive benefit.

  $ nix-env --rollback

Specifically the suggestion is referring to home-manager, which is community supported (being in nix-community) and is the recommended way to install packages rather than imperatively doing it with nix-env.

The problem is: recommended by whom? The Nix Manual presents nix-env as the main way to install Nix packages. So, as long as the manual centers around nix-env, newcomers will try to use nix-env and get bitten by it.

The same is true for the NixOS Manual, it suggests that people install packages with nix-env before talking about declarative package management.

Also, I love home-manager, but it is perhaps too much for people who just want to dabble in Nix a bit. It would be nice if a simpler declarative way of managing packages was presented first. Perhaps surprisingly, another way is documented but it is deep in the nixpkgs manual:

https://nixos.org/manual/nixpkgs/stable/#sec-declarative-pac...

I have a different opinion myself though. While I don't use nix-env anymore I think it is important because serves as a bridge between nix and what people are typically expecting.

I think nix-env syntax should be improved to be similar what you expect with other package managers though.

Once you understand how nix operates you can then stop using nix-env.

I personally use nix for development. For every project I define the whole environment including all tools I need for development. This ensures that if someone needs to work on my project can get the same dev environment. If you add a tool like direnv to the mix it makes the experience much, much better, because then environment is automatically set up as soon as you enter the project directory.

The correct way is to search for packages as the following: https://nix.dev/tutorials/ad-hoc-developer-environments.html...

And then install via: nix-env -iA attribute-name

New Nix CLI as part of next release fixes that, but it's not out yet.

But as a complete outsider, this doesn't fill me with confidence. If Wikipedia is to be believed, NixOS was launched in 2003, so 18 (!) years ago, I assume that's when Nix was created.

The issue mentioned seems quite basic and you're replying that it will be fixed in a new Nix CLI.

This seems like a basic QA fail, I wouldn't want anything like this within 100 km of any production environment I'm working on.

I imagine Nix is underfunded, but few projects come back from stuff like this. If they can't get their stuff together in almost 2 decades, who will ever trust them that they ever will?

Basically the nix repo unlike other package repos it isn't flat it is a tree of many different derivation. A lot of them even repeats. For example parent poster had problem because he installed version of nix that used musl instead of glibc. You can similarly reference packages that cross compile for different platforms. For example you can install Raspberry Pi version of a package. But if you're not running RPI that binary won't execute on your computer.

Now, nix-env by default tries to guess where in that package tree application with given name exists and then installs it.

This probably worked well early on when nixpkgs repository was small, but as it grew, they added cross compilation or other variants like installing packages with musl etc. It doesn't make a good guess. That's why now -i and -u should be followed with -A option, which disables the guessing and provides a path in the tree, so the command should be (besides being accurate it is also much faster since nix doesn't need to scan the whole tree):

    nix-env -uA nixpkgs.nix

    nix upgrade-nix

It’s not like you have found a bug in the way the fundamental package handling is done which hasn’t been fixed for a long time, so I don’t think it’s fair, especially over an in context negligible nitpick.

I guess it depends on what Nix wants, if it wants to remain a niche system, then things are good as-is.

Though, on the other hand, the comment says a new CLI is close to release, so I guess people are trying to improve things.

I don't want to be too critical, I love Nix. But the experimental new UI was already in development (and available) when I started using Nix in 2018. It has also changed quite a lot recently and is now very strongly tied to flakes. However, the flakes RFC was withdrawn [1]. Given that flakes are a large change, I'd expect (but maybe I am wrong) that there will be a new RFC.

At any rate, I would be surprised if the new CLI with flakes are close to release.

[1] https://github.com/NixOS/rfcs/pull/49

If it was about 50 users, we could just do it.

Since there are many users, it takes a while to move from one UI to another.

It is most definitely not a Windows Store one click user-friendly tool, but it is infinitely powerful, allowing you to run only one specific app with even something like a patched libc, upgrade the system and rollback wherever you want, it is the only package manager that doesn’t leave lingering files all around your system.

Before this bug I was running nixpkgs on top of all the Debian machines I'm using (including crostini vms on chromeos devices), mainly as a package manager to mange and keep up-to-date on things outside of debian's apt ecosystem (firefox, thunderbird, go, pipx, etc.). After this bug I removed nixpkgs from all of them.

While I agree that nixpkg is great for declarative use cases others mentioned, and also it's very useful to try something temporarily (e.g. `nix-env -i something`, do things, then `nix-env --rollback`), those are not my main use cases (while at least not for my debian machines), and my main use case of it is broken because of that bug.

I even recommended nixpkgs as a better alternative to homebrew to my friends using mac, I guess that's not really true any more.

Now I keep up-to-date to things not in apt-ecosystem by subscribing to releases on github :(

I simply do not grok the syntax or what the functions do. I tried searching for the functions shown in the examples on the website to no avail. I searched packages, options, and even resorted to ctrl-f while clicking through the site "documentation"...

It sounds awesome, but its in dire need of some better documentation if it wants to be accessible, IMHO. I simply didnt have the patience to delve any deeper.

I won't focus on the good parts, but focus on your comment.

Nix has three things that I think are confusing and took me an embarrassing long time to grok, and I find other people confused by as well.

1. Where does the semicolon go? Reading code makes it feel arbitrary, but really the only time a semicolon is used is to terminate an assignment (=). If there are no equal signs, then you don't need any semicolons.

2. The heredoc strings ('' '') are in fact strings. Nixpkgs passes these strings to a shell quite frequently and their docs also make it feel like these statements are shell scripts and not strings. The problem here is that to pull a variable from nix you use ${VAR} and to use an environment variable you use $VAR. When you look at these heredocs it looks like a pure bash script, but it doesn't quickly jump out at you that ${VAR} are being templated in by nix. It also isn't obvious that $VAR is not being templated in by nix.

3. The nix language is very very tiny, and almost everything comes from nixpkgs which is basically stdlib as well as the package repository. Almost everything, including callPackage is in nixpkgs. The documentation is fairly decent, but when trying to figure out how do something, you are basically hunting in that code base.

Bonus 4. In nix it is really easy to build single language applications. However if you need to mix two, finding good docs or examples is really hard. IE, if you want to build the javascript front end of your web app as well as the python backend. These combinations need at least two derivations, and how to make it happen is bespoke every time.

> 1. Where does the semicolon go? Reading code makes it feel arbitrary, but really the only time a semicolon is used is to terminate an assignment (=). If there are no equal signs, then you don't need any semicolons.

The semicolon is also required when using with expressions (i.e. augments the environment of an expression like let, but confusingly doesn't shadow let bindings)

  nix-repl> with { a = 3; }; a
  3
  nix-repl> let a = 5; in with { a = 3; }; a    
  5 # ??????

One situation I've run into is how to quote ${ in a multiline string, and the solution is to write ''${ :

  nix-repl> '' ''${ ''
  "${ "

This is very much true, essentially Nix and its ecosystem is a big instance of Hyrum's Law[0] whereby people go with what they actually observe rather than what should be according to the spec (if it exists).

[0] https://www.hyrumslaw.com/

Well, you recognized the gotcha, but there are people who don't, they inserted a bug into nixpkgs, then someone else fixed it not realizing its genesis. There is a lot of spaghetti code in nixpkgs caused by misunderstanding of `with`

For anything that doesn't fit a common nixpkgs function (like `buildPythonPackage`, etc.) then I always start with this:

    nixpkgs.runCommand "my-application"
      {
        dependencies go here
      }
      ''
        shell commands go here
      ''

How do you add files to this build? For example, if I have this program.rb that depends on ruby:

    puts “Hello World”

    let pkgs = import <nixpkgs> {}; in
    pkgs.runCommand "my-app"
      {
        buildInputs = [pkgs.ruby];
      }
      ''
      pwd && ls;
      ruby ./program.rb
      ''

    let pkgs = import <nixpkgs> {}; in
    pkgs.runCommand "my-app"
      {
        buildInputs = [pkgs.ruby];
      }
      ''
        pwd && ls
        ruby ${./program.rb}
      ''

    let pkgs = import <nixpkgs> {}; in
    pkgs.runCommand "my-app"
      {
        buildInputs = [pkgs.ruby];
        program = ./program.rb;
      }
      ''
        pwd && ls;
        ruby "$program"
      ''

If you want to write to nix store with stdenv use $out.

An javascript front end can have thousands of dependencies, and the nix build environment does not have network access unless it's computing a fixed output.

To do that manually would be a nightmare, so you must automated either with tools you write or tools someone else has written.

Then you still have to figure out how to combine it all together.

Also on allowing network access in the sandbox requires elevated privileges which means you cannot do that method on environments you don't have full control. And it would create builds that are likely not reproducible.

It's really not. I've used this successfully for Javascript projects (running npm and yarn), Python projects (running pip install), Scala projects (running sbt), Haskell projects (running cabal), Racket projects (running raco), Go projects (running go get), etc.

It's pretty much like writing a Makefile, except we have to specify the buildInputs, and there are occasional annoyances like tools trying to write to $HOME (requiring an `export HOME="$PWD"` line at the top of the script).

> the nix build environment does not have network access unless it's computing a fixed output

Yes it does, if we avoid the sandbox. This used to be done by specifying a `__noChroot = true;` attribute; I don't know if that's changed in more recent Nix versions.

> Also on allowing network access in the sandbox requires elevated privileges

Sandboxing is a sensible default, but for practicality I usually set it to "relaxed" on NixOS and disable it entirely on macOS (since that often uses stuff from the host system).

> which means you cannot do that method on environments you don't have full control.

Could you give an example of where that would ever be an issue? The only thing I can think of is hydra.nixos.org, but there's no way in hell such a derivation would be acceptable for nixpkgs, so that's pretty irrelevant (similar to how it's really easy to make a .deb package from a directory and a control file; but there's no way in hell it complies with the Debian project's packaging guidelines)

> And it would create builds that are likely not reproducible.

Yes, that's why I mentioned we might want to factor out the input-fetching at some point.

> Then you still have to figure out how to combine it all together.

That has nothing to do with Nix. As long as a project has a build script, or a Makefile, or a list of commands in a README, or whatever, then we can plug it into that template and see if it works. Are you complaining that the Nix package manager doesn't have specific documentation for how to build every random application ever written?

> Could you give an example of where that would ever be an issue? The only thing I can think of is hydra.nixos.org, but there's no way in hell such a derivation would be acceptable for nixpkgs, so that's pretty irrelevant (similar to how it's really easy to make a .deb package from a directory and a control file; but there's no way in hell it complies with the Debian project's packaging guidelines)

In any production environment, using any external build service like nixbuild.net, self-host hydra. Basically anything beyond personal use.

The post about all this is from Channable, Channable runs nix built code in production. Their developers (likely) cannot disable the sandbox and ship that code.

You even say that there is "no way in hell such a derivation would be acceptable for nixpkgs"

If you cannot share it, it's only good for you and you alone.

> in production

Lol. If someone doesn't want to be executing random unverified binaries fetched from arbitrary online locations, then they shouldn't write such things in their build scripts.

That has nothing to do with Nix. The same goes for Makefile, or whatever.

Three things I find immensely helpful:

- Syntax highlighting (I use Emacs, which has a Nix mode)

- Syntax checking (again I use Emacs for this)

- Writing complicated scripts in a separate file. This makes it harder to splice in Nix values (there are ways to substitute in text, but I find them more trouble than they're worth). Setting some extra env vars in the build environment is usually the easiest approach.

I find it easier to think of "attribute sets" like `{ foo = bar; }` more like a JSON objects (e.g. `{ "foo": bar }`) rather than "assignment".

Nix does have a `let` syntax, which probably looks more like assignment, but I've never used it in the 7 years I've been using Nix. I find `with myAttrs; myExpr` to be far better (except for the WontFix issue that `with` doesn't shadow function arguments).

Off the top of my head, the following expressions also require `;`:

    with myAttrs; myBody
    assert myCondition; myBody

To make it clearer, he is saying that he does

with { foo = "bar"; }; <some other block>

instead of

let foo = "bar"; in <some other block>

The differences here so other people can follow along.

1. with will not shadow variables defined higher up. This can be either desirable or very confusing.

2. let is recursive by default, meaning you can reference other variables in that block. This behavior can be mimicked with `with rec {}`

The thing about shadowing is that `with` will not shadow anything that's in the implicit context, which includes things defined by `let` (hence why I avoid it) and function arguments (unavoidable). We can shadow other `with` bindings as much as we like, e.g.

    with { foo = 1; };
    with { foo = 2; };
    with { foo = 3; };
    with { bar = with { foo = 4; }; foo; };
    [ foo bar ]

This doesn't work for bindings that are in the implicit environment/context, for example:

    with {
      f = foo: with { foo = 42; }; foo;
    };
    f 123

The rationale for this behaviour is to avoid changes made to an attrset from overriding explicit arguments. For example:

    foo:
      with bar;
      x + foo

I understand this rationale, especially when using huge, constantly-changing attrsets like `with nixpkgs; ...`, but it's still annoying ;)

If you use Nix in a "drive-by" kind of way, it's very hard to make any inroads into the language. The documentation and tutorials are not really intended for a casual user - it is expected you will sit down with the Nix site like a good book, and go at it from start to finish. If (like me) you instead touch Nix little and often, doing small invocations or on-the-fly editing, the docs are much less useful.

There's also no way to understand it in terms of a few simple concepts for beginners. Do you want an override, an overlay, a flake, or something else? Does specifying an option in multiple places combine them, or does one spec override the other? It feels as though Nix is a technology of special cases, and the docs don't make it easy to understand what to do in each case when all you know is the kind of change you want to make.

The worst culprit of this is NixOS services, which each declare an ad-hoc API that I normally find myself digging into the service declaration file in the Nix repo to try and understand.

This is super serious, major, I hope the devs are paying attention.

Most users of anything are drive-by, rather casual users. Think of any big tech out there: they became popular precisely because they were easy to get started (at least back when they were launched/became popular): C, Unix, Bash, Perl, PHP, MySQL, Javascript, Java, Ruby, Go, etc. Almost everything that's big was usable by people don't read an entire book before they can make "Hello world".

On top of this, Nix could be extremely well suited to casual users. It inherently avoids most of the problems that plague casual users of traditional package management tools.

Unfortunately, most of the more serious Nix contributores I have interacted with are in too deep to see the usability problems. Even worse, usability issues get trivialized, and users who ask for basic improvements get talked down and told they just aren't getting the magic of Nix.

For example, the topic of versions has been argued about for over five years: https://github.com/NixOS/nixpkgs/issues/9682

I've used nix for a few projects over the past couple of years, always because I wouldn't have done the project without it (or I would have had to learn a series of separate solutions). I have never learnt nix, except to get it to do what I want.

When I started, it was full on "what is the magical incantation to make it do what I want?" mode. Now I am usually capable of saying "to do X I need to do Y; what is the documentation for Y?", and when I find the documentation for Y it usually conforms to my presuppositions and merely adds detail and color.

This is a really weird feeling. The idea that Nix/nixpkgs makes sense is so shocking to the core of my being. But it is still difficult even to find the documentation I want. But nix does give me another feeling that I like: confidence that I am building a system.

The benefits of Nix is that it dramatically organizes the absolute disgusting mess that is how most people manage their computer. I don't want to drive people trying things out away, but Nix will never be worth it unless you embrace it, and half-ways usage will deliver all the downsides without enough of the upsides.

Not every value function is monotonic like that, I am not going to pretend it's otherwise.

The goal is to help people fully embrace it, not to allow them to comfortably come to a stand-still half way.

At least my impression from a while ago is that the actual documentation Nix has seems to be aimed at "drive-by" or at least casusal/entry-level users, and to get the deeper insights that make the whole thing enjoyable, you need to give up on the documentation and just absorb all the details from github discussions, through extensive trial&error, etc.

Reference first, then tutorials.

Good tutorials first, reference second.

[1]: https://search.nixos.org/options

Recently my laptop died, and I just needed another one going in a hurry and I tried Xubuntu because it's the only distribution which had Wifi drivers that worked for a 12 year old Mac I had lying around without any trickery.

It's funny because Nix worked fine, but my Ubuntu installation just stays out of my way and lets me work too. I started to make me question whether all the added Nix complexity really added much value. Would investing in learning Nix really pay off over spending my time learning a new programming language with real world application?

I also think there is something to be said for the fact that, unless you team mates are using Nix,good luck using any of the standard tooling or scripts others will use. One of my favorites are shell scripts that start with `#!/bin/bash` never working and always having to patch them to use `#!/usr/bin/env bash` I understand the latter is the correct way to do things in 2021, but we all know it's not going to happen :)

I agree that I'm not sure the extra complexity is worth it for day to day use, but just the ability to avoid catastrophic failure and near instant setup on new machines is worth it I think.

- boot linux livecd

- mount your broken filesystem

- bind mount the important bits from the live kernel (/proc)

- chroot

Like this:

    cd /

    mount -t ext2 /dev/sda1 /mnt #Here's your broken install

    mount -t proc proc /mnt/proc

    mount -t sysfs sys /mnt/sys

    mount -o bind /dev /mnt/dev



   chroot /mnt /bin/bash #boom you are in.

usually apt stuff for me(pulled power during a package upgrade)?

What needs to be done to fix the system may not be obvious and the fix may not be simple. Furthermore, the trial and error for fixing may leave a trail of new issues.

And that is why I use NixOS. I have to wrestle with it to get something working, but once I'm done I can guarantee it will keep working until the heat death of universe. Compared to that, Ubuntu and others let me get off the land fast but I never know when I'll crash or if I'll able to fly back from it.

https://xkcd.com/1205/

Also, check out flakes, which somewhat steers away from the central repo design.

Nix is a great idea, and it’s goals are ambitious, but I haven’t found it to be usable yet. Almost any time I try to do anything I get mired in a tar pit and I basically end up having to give up. I’ve learned lots of tools and technologies in my career, but Nix remains elusive.

I tend to use lots of custom tools and commands, that are really painful to install and setup for a new machine. With NixOS all of it is just one command away.

But, I agree, it is REALLY HARD in the beginning to grasp things.

Here's my configs if you want to see how I approached my own setup: https://github.com/pimeys/nixos

In other cases, things are just broken--my nix installed rustup is failing to build packages this morning with a clang linker error (can't find iconv) and Googling the error message + 'rust' indicates that this problem is only encountered in Nix environments. Similarly, many of the other problems are that it requires you to repackage other libraries from the ground up. For example, most Python packages are available in Pypi such that users can just `pip install` them. Nixpkgs has a `pypiPackage` function that allows us to import a package from Pypi, but if that package has C dependencies (and many do, because Python's performance sucks and Python's performance sucks because everything depends on C packages precluding optimizations to the interpreter, but that's a gripe for another day) then Nix requires you to package those C dependencies in Nix first, and because every C package has its own bespoke build system with implicit dependencies, you must be expert in each individual C package in the dependency tree. Again, there's no amount of Nix Nirvana that will save you from this.

In many other cases (e.g., my 'packaging vs code with my rust plugins' example from my earlier post) we have to know the specific conventions for specific nixpkgs functions which will vary from others, and even that might not be enough. These conventions can't be derived from any general understanding or intuition of Nix (i.e., "Nix Nirvana") although it certainly doesn't hurt.

Further, NixOS is easy mode. It's the first class citizen in the Nix ecosystem. Other platforms don't work so well. In my vs code + rust plugins example, there were several posts documenting the process on NixOS, but for whatever reason Nix on other platforms (e.g., macOS) requires different configuration which is undocumented and even proficient Nix users couldn't figure out the issue.

But yeah, to your point, it is really hard to grasp as well, which exacerbates all of the above problems.

I'm mostly writing Rust, so as long as I have `libssl` available, I don't really need to tweak my system that much to be able to work. I definitely would reconsider, if my work would involve using Python...

Although, I'm kind of interested about the challenge of understanding the packaging of these dependencies you're talking about. Making the whole compilation setup nice and clean with nix. But, I don't know, I get it it's not for everybody. And it also might be these lockdowns that put me to this mode of doing stuff such as tweaking NixOS.

Then, again, computers are my hobby. And new challenge is always accepted!

The examples you find for creating a shell.nix usually work individually, but if you need a combination of two nix files there seem to be too many ways to do things, because I find it almost always impossible to combine two examples into one.

Also for searching packages I always end up looking on the website. Using the built-in search you have to do some weird incantations that aren't easy to remember and you still don't get the full list of packages that are related.

It also doesn't help that the package names are random. There's no standard format it seems: gcc 9 is "gcc9" while clang 9 is "clang_9". Why not have packages be <namespace>.<package>.<major[.minor[.build]]> or any other standard convention.

Then you could also say you want a certain version of a package instead of having a name for each. Or a minimum version.

Maybe we need a wrapper around nix that exposes a user friendly interface. I really like the basic idea.

At mindbuffer[1] we've started using it for our recent art installations. The big benefits for us are reproducibility, ease of deployment, and the ability to collaborate on the composition of the whole system. I.e. rather than sharing a README of how to install things one by one and hoping each of us has followed it correctly, we just work on the same set of config files via a git repo (like we would any other code) and can be sure we're all on the same page as a result.

Very much looking forward to Nix 3.0 landing with all its UI improvements and flake support. It seems like these changes will go a long way to making Nix more accessible, and provide a smoother on-ramp to learning the language itself.

https://mindbuffer.net/

For example I spent ages this morning trying to get a gRPC Rust environment working and battled the nix file for a good two hours but now it works I know it will always work and never have to do it again.

Edit: I should clarify Nix is the build tool at work for Haskell and some of my colleagues are contributors so I know I can get help if stuck. I’m a noob though.

This kind of breaks. For people like myself who already spend a lot of time debugging dependency and compatibility issues in less sound package ecosystems (scripting languages I'm looking at you), Nix alternates between enlightening and maddening. Which I like.

Absolutely dread having to figure out something new.

People can commend it as much as they want, but the steep learning curve is largely self inflicted because of their resistance to writing clean, comprehensive, up to date docs

It has also led to the community being filled with a lot of arrogance and pretentiousness

I wouldn't run nix in production because of the lack of accessibility and toxic community. There are other ways to get reproducibility, etc without using an arcane and poorly documented toolkit

I would conjecture that you have cause and effect reversed.

When people think they're doing you a favor, the ego protects itself from hearing how their help is not actually that helpful. See aid to Africa in the 80's and 90's for example.

Anecdotally my experience with nix community has been unpleasant. They're very resistant to suggestions and offer little in terms of guidance or documentation.

Also, regarding DevOps, the tooling around Nix makes it a little brittle for anything event based--rapidly changing configurations on the fly due to network conditions (Consul, Ansible, etc). This is where configuration management is heading, and due to the static nature of Nix, delegating dynamic changes is hard/anti pattern.

Channable uses Consul, Vault, etc. for dynamic configuration and it works with Nix just fine.

You don't have to use static configuration files with Nix. Either fetch dynamic stuff using the Consul, Vault, etc. APIs at runtime or use a tool like vaultenv [1] or similar if you don't want this logic in your application code.

Put those tools in your systemd service before launching your app, and you're good to go.

(NB: I was DevOps teamlead at Channable while a part of this work was being done. Sad that I haven't seen the final picture. I imagine it's lovely compared to what we had before)

[1]: https://github.com/channable/vaultenv

It does cause all sorts of problems with non-nix-aware software, but most software can be tamed, and what absolutely refuses to run, can be run with steam-run or by a user-created environment mimicking a “normal” distro.

For me, the big benefit is that even though I have to read the upstream documentation, I can use the same syntax for everything and do not have to learn and use N different configuration file syntaxes.

https://www.hashicorp.com/blog/announcing-consul-terraform-s...

I wonder if something like that for nix would help you.

E.g. have the nix config in consul; and have the local daemon `watch` the consul config (And services) and build and acitvate the nix thing every time the consul config changes.

Could maybe even do automatic rollbacks based on consul health checks.

I think it could work pretty well!

I guess it depends on how full on nix you want to go

As for the article, I hadn't actually considered using it as a replacement for Ansible etc - but I could see why it might work better. I could pin packages etc to the versions, and get the exact same thing each time - no guarentee with ansible etc that the apt repository hasn't been updated since the last install.

I think the whole Nix thing is an interesting idea, and I like the ideas behind it, I haven't played enough with it yet to dare use it in production (or on my main desktop for that matter) but it is certainly a tempting idea.

But on the server is where I think Nix really shines today. The biggest thing for me is that every deploy is "clean" and can be rolled back. Systems like Ansible (and basically every other config management system) only modify the part of the system that they know about and mutate global state. This means that your service might just happen to work because of a library that just happens to be installed by another service, or even on that is still installed from a service that used to run on the system. Unless you excercise Perfect Constant Vigilance and update configs to delete things instead of just removing the config to install them your configs will soon be be broken and depend on the current state of the system. I remember thinking that setting up a new VPS would be trivial since I was using Puppet (or whatever at that time) but just about every service depended on some feature that was (no longer) specified in my configs so I needed to fix my configs as I deployed the new VPS.

I wrote a blog post on each of server NixOS and desktop NixOS if you want more of my thoughts.

https://kevincox.ca/2015/12/13/nixos-managed-system/

https://kevincox.ca/2020/09/06/switching-to-desktop-nixos/

Having worked with puppet for a long time, I can relate. There is this false sense of confidence that everything is reproducible "because we had the discipline to never do manual changes and always use puppet" but in real life:

- dependencies between resources are forgotten, it worked the first time because luck in the order in which resources were applied, but would fail in from scratch applies.

- hidden dependencies that were not specified but it worked because an other resource was installed at the time.

- hell, even a failed apply could have partially installed dependencies that would make subsequent applied work.

- there's always _that time_ when you had to fix something in a hurry and manually installed a package which is now a potential hidden dependency for everything else.

One of things that makes it far superior to these solutions, is that while many of them try to be declarative it still an iterative hack to emulate it.

For example if you use your CM to install a package on the system, or create a file in /etc. If you remove that statement the file or package will still be there.

In NixOS it will be as if it was never there. The reason for it is that the existing CM work by modifying current OS, while NixOS rebuilds the entire OS every time you make a change (thanks to cacheing that operation doesn't take as long as it would normally do).

As for using things in production, I generally like things that are well established. So it took many years before I got the courage to propose it, but frankly I don't think it ever let me down. As I used it always act predictable, both the NixOS as well as build (as long as you pin down nixpkgs).

The only thing to worry about is if the outside environment demands a change you better know how to do it. For example application works fine, suddenly you have higher traffic and things are crapping out and maybe you need to change some settings or there was a bug in one of your dependency and a new version isn't in nixpkgs. You better know how to do those things.

The readme was/is more of a short note to myself than for users, nevertheless this is how i manage my entire home env on my mac and it works beautifully.

Another reason I'm quite excited about Nix is that in my team we're currently using a whole bunch of tools like pip, Conda, npm all at the same time to manage our dependencies and it's starting to become really cumbersome to set up the dev and production environments correctly across different operating systems and architectures. For instance, Conda doesn't exist on arm64, lots of pip packages have to be compiled by hand on arm64 etc. etc. It's a mess that I'd love to get rid of.

[0]: https://github.com/danielfullmer/robotnix/

Some friends and I have been experimenting with this for the last year over at https://tvl.fyi (code etc. is linked at the page bottom).

We've run into some issues with the current implementation of Nix, but stay tuned on that :)

Like right now at work we have a small team making a website with docker-compose to specify the dev environment, pip-tools to pin python dependencies, and saltstack to deploy new state to bare metal in production.

How exactly, command by command, would nix solve the same problems? What would change from my current solution? How would I think differently? How would my life get better? What parts wouldn't be solved and I'd have to keep some of the old stuff around?

Those are the tutorials that convince me to switch, because they get me over the hump of not getting the new paradigm, and being afraid of the downsides I don't know about yet. It's tricky because it's pretty specific to how someone's currently solving problems, but maybe knowing what I'd optimally want can help with figuring out something doable.

(This blog post didn't work for me because it was too vague. I already have old boring solutions for their python pain points, like pip-tools, so I can't take their assessment of how much better things can be on faith, and they don't show me the details of the new thing.)

This might be problematic, because to understand these commands, you would have to understand Nix. It's not just a tool that replaces your existing tool, it's a total workflow change, and without understanding what's happening under the hood, you can't really appreciate how much it makes your life easier. I could show you a `nix-build -A deploy.somemachine`, but without understanding why it's called like that, and what exactly it does, and how I build most of that myself using nixpkgs tooling, you wouldn't really 'get' it, because you probably wouldn't understand how is that better than doing an `ansible-playbook -i foo.yml`, and I wouldn't be able to explain without explaining Nix/nixpkgs from scratch.

Nix, compared to other DevOps tools, is alien technology, and requires unlearning a lot of things, learning a whole new language and ecosystem, beginning to think in terms of that ecosystem and general programmability and functional programming, and then building on top of this to solve your problems. Nix doesn't immediately bring solutions to your problems, it brings you a new way of thinking about these problems and solving them in ways you didn't even consider before.

I know this is all handwavy, and that is unfortunate, but in my opinion unavoidable when dealing with radically different tooling. From my experience, the current 'classic' devops tooling (Docker, Terraform, ...) has reached a local maximum of usability. Getting out of that maximum requires a vastly different approach, and thus requires a much longer learning experience than what most people are used to. If you keep switching to different tools that behave how you expected them to, you're really just using the same thing as before, but with a different syntax.

That's the entire point of a tutorial like GP describes. Yes, Nix works on a higher level of abstraction than existing tools, but it solves the same problem, and comparing it to "alien technology" doesn't help.

Nix is simply:

* a large corpus of work (nixpkgs; which contains build instructions for source code that can combine all the way to an operating system or a service image)

* organized in a particular way (using the Nix language; used to declare the expected results of all the different build strategies out there)

* and coupled to the tools that let you turn a description of a process into the result of that process (nix-build, etc).

This is abstract but hopefully less handwavy? The concrete details of the Nix paradigm really lie in the syntax itself so what you're saying about "same tools with different syntax" doesn't make much sense to me, as every programming tool is defined by its syntax.

The on-ramps to becoming fluent with the Nix language are, admittedly, subpar. I got nearly nothing out of "nix-pills" and cringe every time someone recommends that tutorial which takes 10 chapters to build your first package in the most roundabout way possible.

What Nix needs is a single page specifying what goes in a Nix derivation. Like the Compose YML reference but simpler. Also the nixpkgs manual is quite central but impenetrable for casual reading. Also how about someone writing a tutorial for generating a static HTML site with Nix, then deploying it, then incrementally adding dynamic backend services to it?

I'm arguing only the one point that I cited - that Nix's advantages can be presented as a step-by-step 'write these commands' getting started guide for migrating, I don't know, some ansible based production into NixOS. Following some steps blindly and expecting immediate results will only result in frustration and disappointment.

The 'alien technology' is in my experience quite helpful in getting expectations straight when teaching people Nix. Driving the point that this is not like switching from npm to yarn, but that you will have to spend a serious chunk of time learning a whole new thing.

> The concrete details of the Nix paradigm really lie in the syntax itself so what you're saying about "same tools with different syntax" doesn't make much sense to me, as every programming tool is defined by its syntax.

There is more to programming languages than their syntax. But maybe I couldn't have been more clear. What I meant, is when you're comparing Prolog, Ruby and Python, Ruby and Python basically look like the same language. In the same vein, when you're comparing nix{,pkgs,os}, Ansible and Puppet, Ansible and Puppet basically look like the same tool. Switching from Ansible to Puppet is relatively easy, while switching from Ansible to Nix requires switching your mindset and approach considerably.

> What Nix needs is a single page specifying what goes in a Nix derivation. [...] [A] tutorial for generating a static HTML site with Nix, then deploying it, then incrementally adding dynamic backend services to it?

All of these are good ideas in my opinion. Nix needs better documentation, but the answer isn't more "build production on Nix in 5 minutes" guides, but more "go on an adventure to understand Nix thoroughly and make using it obvious" guides, IMO.

To illustrate what I mean here is my history with nix (simplified, but it's meant to show the evolution):

So I started using nix as a reproductive dev environment. Initially using shell.nix to have all my tooling. This is great if you also have direnv installed, which basically automatically makes tools related to the project to appear.

Anyway I started then using various tools that would translate python project to nix initially was also a big fan of setup.py/cfg and pip-tools. I wasn't happy with either of them until I found poetry2nix. Poetry is also quite decent so now that's my favorite way to build a project.

So what poetry2nix does is translates on the fly pyproject.toml and poetry.lock file to nix expression. With it you can build project in nix you can also create a shell that has python with all packages available as described by poetry. So that's now quite decent reproducible environment.

I realized that to make the dev environment nice I'm adding a lot of nix boilerplate to every project, also as I'm learning new things and improving it it is getting hard to synchronize that boilerplate across other projects.

So I created a new repo where I put everything there, I also learned how to use modules system (which they created for NixOS) ultimately I made repo that you reference in default.nix and shell.nix in my project it loads project.nix file which describes information about my project. Allows me to set various aspect of the application, like what additional packages should be installed in dev shell (maybe I need a local postgresql installation). How the project will be deployed (if it is meant to run as AWS lambda, a serverless (npm application) is added and package.json/package-lock.json files are sourced. If it is a docker asks what binary should be executed in the container. Whether to compile using minified python (with missing some of core libraries) etc.

That ended to be some Nix code, but simplified things greatly, and makes things usable by people not familiar with Nix.

We use gitlab for CI/CD so I created a special gitlab-runner for nix builds. It actually is already built in in NixOS and only needs to be enabled, and they also show configuration in example. Basically how it works is that it stills spins a docker container for the build, but exposes /nix/store (read only) and the unix socket to trigger build. I also created an S3 bucket and configured it for storing caches of builds.

Having this runner lets me utilize one of major benefits of Nix which is caching (if all inputs (source files, architecture, interpreters etc) don't change then output will be the same). This is great, because if I only change source file, and don't change dependencies, the pipeline will only rebuild my app. If I work on a branch and branch passes all tests, I merge it and merge is just fast forward, then build job in master branch just takes couple seconds, because nothing change. If the branch wasn't rebased and actual merge was done and some files are now different the nix will rebuild things that were modified.

Deployment of my python app as a lambda ends up being just:

    nix-shell --run "sls deploy"

This is quite decent and I'm still testing it on a single app, but it basically cut deployment time from 10 minutes to 5 minutes. And I know I can cut it further (serverless doesn't know about nix, but it understands poetry, so once again again it downloads all dependencies it needs, extracts them and bundles them into the lambda. I plan to populate its cache from the packages that nix stores when is running the build).

There is also possibility that I'm aware of, but haven't explored yet, but seems interesting. It might require some work. For example you could configure your local computer to use the machine that is used for CI/CD build to help with local builds. Similarly you could expose the cache as well to prevent unnecessary rebuilds. You probably shouldn't do that if the user isn't trustworthy, otherwise you might need to create some additional safety layer.

Now in your scenario you have bare bone servers to deploy. If those would run NixOS I believe the deployment step would be simply to deliver new configuration.nix referencing your application to them and also have those machines configured to use the same cache as you had with CI/CD so they won't have to rebuild everything again. There are several tools that make the process easier, for people who use public cloud there's terraform support, NixOps (last time I used it was the weakest part of Nix, but maybe it improved, it was great tool when I wanted to test everything a la hashicorp's vagrant)

The docs for using nix as a package manager are decent. But the rest, less so. They’re a recursive rabbit hole of confusion.

I hope it helps.

Definitely Nix deserves a try.

If that sounds interesting, you can sign up for the public beta: https://www.floxdev.com/

Here's an announcement blog post about our initial release: https://discourse.nixos.org/t/introducing-flox-nix-for-simpl...

- What is the story in nix for packaging untagged branches of software, or reasoning about "snapshots" where pools of unreleased repos/packages are able to be treated as a single versionable unit? (think not just a nightly build, but builds corresponding to unmerged feature branches, etc)

- Does the nix hash account for dependencies only changing the hash for ABI-impacting changes such as when a header file changes? Or does it change dependent hashes always? Or never?

- I have an existing system for managing packaging metadata which I don't want to migrate from. How much trouble will I get into if I want to generate the metadata on the fly each time (as I currently do for my debs)?

- How much pain is it to roll a nix package "by hand" (basically with the dpkg-deb equivalent tool rather than the dpkg-buildpackage equivalent tool)?

- Nix isn't supported in Artifactory (RTFACT-19998 has been open since 2019). Nominally, I can use the dumb WebDAV option, but is that going to affect my user experience and/or will it be a maintenance headache?

- What is the apt/nix interop story? I would likely need it to be bidirectional, so that my nix workspace could depend on system debs that I don't want to port over, but also potentially have "gateway" debs which able to do the opposite, of depending on the nix workspace from a deb shim, and installing/updating it in the postinst.

I've researched all of these in the past at various points and had a lot of trouble finding clear, straight answers on them. I'd love to hear from other users or nix experts, though, who could help calm our fears on this.

- What is the story in nix for packaging untagged branches of software

If you want to do unreleased software, you would create overlays that added them. You're overlays most likely would not need to define the build, just create an override of the one in nixpkgs with a new source: https://discourse.nixos.org/t/how-to-override-package-versio...

- Does the nix hash account for dependencies only changing the hash for ABI-impacting changes such as when a header file changes?

The nix hash changes whenever anything in the derivation or it dependencies changes. It has no concept of what an ABI-impacting change would be.

- I have an existing system for managing packaging metadata which I don't want to migrate from. How much trouble will I get into if I want to generate the metadata on the fly each time (as I currently do for my debs)?

The nix code _is_ the metadata. There isn't an equivalent Packages like there is for Debian.

- How much pain is it to roll a nix package "by hand" (basically with the dpkg-deb equivalent tool rather than the dpkg-buildpackage equivalent tool)?

Don't fully understand what you are asking here, but the equivalent tool of dpkg-deb would be nix-build.

- Nix isn't supported in Artifactory (RTFACT-19998 has been open since 2019). Nominally, I can use the dumb WebDAV option, but is that going to affect my user experience and/or will it be a maintenance headache?

Nix only writes new files to a remote store. As such nix caches can be served by anything that can serve files. I personally upload to S3 and have artifactory use that as a remote.

- What is the apt/nix interop story?

This is a complex question. At work I use nix on debian hosts, and it works very well. Nix packages have _no dependencies_ at all on the host OS. IE binary installed from nix will use required libraries from nix as well. However nix creates environments were only the packages explicitly installed show up. As such if you install firefox only bin/firefox will be added to that environment, despite using all of its dependencies from nix. Nix hard references all it dependencies to the /nix directory.

What about the maintenance side, though, like being able to clean/reap old builds from the cache, reason about which ones are part of important/supported chains vs throwaway builds from merge pipelines, etc?

With S3 you can create file lifecycles that will move them to cheaper storage and eventually delete them.

You could potentially create two buckets. One for throw away pipeline builds, and another for when things graduate to something you want to keep.

It wouldn't be very hard to make some tooling, the files in the cache have almost all the metadata you need: http://cache.nixos.org/0ljamf3irbyahd00849b2v1cdddypn8a.nari...

But because it all hashed based, you would need something to read all that into a database. I am unaware of any tooling that does that today.

In any case, if I rolled my own hash package scheme with debs, I'd have to build this piece of the tooling regardless.

Most Nix packages define their sources using `src`. You can pass in whatever you want. E.g. fetch a tarball from GitHub with `fetchUrl`, fetch from a git repository, or point to a local directory.

> or reasoning about "snapshots" where pools of unreleased repos/packages are able to be treated as a single versionable unit?

This is basically what the Nixpkgs collection is minus the "unreleased" part. You can extend Nixpkgs with the packages that you care about using overlays. You have control over the stuff that you put in the overlay, so you can put in unreleased software as well like I described above.

> - I have an existing system for managing packaging metadata which I don't want to migrate from. How much trouble will I get into if I want to generate the metadata on the fly each time (as I currently do for my debs)?

This is difficult to answer without knowing more details.

- You can generate the Nix source files based on the metadata like the sibling comment pointed out.

- Or you could read the metadata with `builtins.fromJSON` and generate Nix derivations programmatically from within Nix.

> - What is the apt/nix interop story?

If you don't want to bother packaging certain applications using Nix (or work towards that gradually), you can. Nothing prevents you from referencing things outside of `/nix/store` in a Nix package. You can launch stuff from `/usr/bin` from a Nix binary if you are so inclined. Libraries are going to be tricky though.

As for the other way around, I'm aware of two options:

- At Channable, we would package the Nix closure of a package in a `.deb`. This works, but you run into trouble when multiple debs need the same store path. You can work around this by packaging the Nix closure under e.g. `/var/lib/<your_package>/nix/store` and bind mounting to `/nix/store` before launching.

- Eventually we realized we didn't want the `/nix/store` stuff in our `.deb` packages. IIRC we created a `postinst` script which ran `nix-env` to realize a store path from our cache + create a GC root so it doesn't get garbage collected.

Some of that stuff may have changed since I left.

Feel free to contact me if you'd like to discuss further. Info is in my profile.

> This is difficult to answer without knowing more details.

The situation specifically is the ROS ecosystem, where metadata is managed in these package.xml files:

https://github.com/ros2/rclcpp/blob/master/rclcpp/package.xm...

The federated nature of the ecosystem has led to a culture where it's very normal to be building dozens of these at once, in the same workspace together, often from multiple repos (the repo above has four in it). So there are several build tools which automate the work of examining a source workspace and building all the packages within it in the correct topological order, in parallel, while respecting build_depend tags. The newest of these tools (colcon) has actually made the package.xml optional in many cases, as it can examine CMakelists, setup.py, BUILD, etc, and discover for itself what the dependencies are.

Your "distribution" of ROS is formed by listing all the packages and repos in this big file, for which there is other tooling to manage pulling dependency sources, whatever: https://github.com/ros/rosdistro/blob/master/foxy/distributi...

Anyway, so the existing ROS/nix efforts (1) seem to basically consume all of this package/distribution metadata at once and generate a giant parallel structure of nix definitions (eg https://github.com/lopsided98/nix-ros-overlay/blob/master/di...), which I fear would be completely opaque to my users. And any system which required a flag day where everyone leaves behind these existing workflows would be an immediate non-starter.

I think the ideal scenario (and what it would look like if I built this myself based on debs) would be that you could source the "base" workspace as usual (enter the nix-shell?), and check out source, build groups of packages as usual with colcon, but there'd be an extra plugin/verb/flag for it, which would make it build each package as a nix package instead of into the usual installspace. The verb would generate the nix definitions on the fly, and probably handle the invocation and build-parallelism side of it as well.

[1]: https://github.com/acowley/ros2nix, https://github.com/lopsided98/nix-ros-overlay

A Nix derivation (a.k.a. 'packaging unit') can be built from sources either fetched remotely, or from a local directory, or a combination of both; plus any other derivation that it depends on. Any version semantics come from how you implement them, abstracted and formalized in any way you want. Nix is a programming language, you are free to do what you want.

> Does the nix hash account for dependencies only changing the hash for ABI-impacting changes such as when a header file changes? Or does it change dependent hashes always? Or never?

By default, any change to any dependency will cause all dependent derivations to be rebuilt, and their hashes to also change. You can 'break' this by introducing stable ABI barriers preventing rebuilds and performing runtime loads of dependencies from .so (or whatever, executing some binaries from $PATH). This is for instance how my firefox can handle any OpenGL library that it can use at runtime, without having a firefox rebuilt for every possible GPU driver out there.

> I have an existing system for managing packaging metadata which I don't want to migrate from. How much trouble will I get into if I want to generate the metadata on the fly each time (as I currently do for my debs)?

You will have to generate nix source code, or importable nix source code, from that system, and pull that into your local nix source tree. All nix evaluation is done from purely defined locally available sources, or sources fetched from remote resources, but marked with some consistent hash.

> How much pain is it to roll a nix package "by hand" (basically with the dpkg-deb equivalent tool rather than the dpkg-buildpackage equivalent tool)?

Nix does not work this way. Nix derivations are a recipe on how to build a package, you have to specify that in one way or another so that Nix can perform the build for you, and reason about its status within a larger build graph. You can 'cheat' building derivations by eg. importing binary builds and only patching them up to work under Nix(OS), but generally you shouldn't.

> Nix isn't supported in Artifactory (RTFACT-19998 has been open since 2019). Nominally, I can use the dumb WebDAV option, but is that going to affect my user experience and/or will it be a maintenance headache?

You will need to run a Nix cache, that CI will push into and other consumers (all having a checkout of your nix definitions) will pull from instead of building everything from scratch themselves.

> What is the apt/nix interop story? I would likely need it to be bidirectional, so that my nix workspace could depend on system debs that I don't want to port over, but also potentially have "gateway" debs which able to do the opposite, of depending on the nix workspace from a deb shim, and installing/updating it in the postinst.

Not aware of anything out of the box. Nix builds only run on Nix (as they all need dependencies from /nix/store/..., and that is populated by Nix itself), so don't expect to easily run them on Debian as normal dpkgs without some disgusting home-rolled hacks. You could technically consume binary .debs from Debian within nix derivations (see previous point), but that's ugly. Generally, just install Nix on all your hosts.

This is not true and no hacks are needed. Since nix only depends on /nix it can be used in combination with ANY linux OS.

In fact there is an official debian binary for nix: https://packages.debian.org/bullseye/nix-bin

Just so that noone runs into issues. This is not completely true. E.g. you cannot use Nix on Fedora Silverblue, because it also uses an immutable filesystem and you cannot make top-level directories like /nix without ugly hacks.

Also, multi-user Nix does not work on most distributions that enforce an SELinux policy (e.g. Fedora/CentOS/RHEL). Incorrect SELinux contexts get set on store files/directories. Single-user Nix typically works fine though.

(These issues can be worked around with rootless Nix, but that is much more high-friction than a globally installed Nix.)

Does this imply full control over hash holding/breaking for derivation authors, or is it like a single field that's basically just meant to be used for the soname?

I would love to see more docs detailing how this works and what the user extension points are for it. Most of what I've found is pretty high level.

I'm only aware for how this is done for OpenGL on NixOS, which is done by some bespoke LD_LIBRARY_PATH modifications. See: nixos/modules/hardware/opengl.nix.

There might be some nixpkgs (non-NixOS) abstractions for this that I'm not aware of, or one would have to be written (which doesn't seem too hard to do).

[1] https://github.com/NixOS/nixpkgs/blob/842f900e73c7ce985218cc...

It's not perfect, but it works, its fast, and its better than the alternatives.

I'm interested in rying Guix at somepoint instead as I think using Guile might be easier than learning the Nix language, cli tools, and more. I think guix could do much what I do now as well.

I can't see myself using guix as an OS though, since they seem to think systemd is evil or something, and really systemd is quite nice in practice in my experience.

I just wanted to say that Guix doesn't hate systemd, or think of itself as an anti-systemd project or anything like that.

Systemd was not chosen because GNU already had a (quite simplistic) init system written in Guile Scheme, and we want to use Guile for the entire system.

Eventually, our goal is to offer a fully integrated interface for managing the system, just like systemd. We are not there yet :)

Nonetheless, I should definitely get to know the project more, because it was some time ago I tried it last. So, Good luck with the project! :)

I'm a Guix developer but I still use Guix on Debian for important deployments, mainly because of systemd / journald.

But, we could also think of it as an opportunity. Well, I hope you'll find Guix useful and enjoyable, and maybe even help us improve it :)

I am yet to find any posts about this or any tools, but have had great success with a small hand rolled tool that essentially lets me decoratively describe cloud resources, parts of my application, deployment "end state", and run the tools required to materialise that (Terraform, nixos-rebuild, etc) similar to GitHub Actions. Hydra is distributed CI for nix-build but does not handle CD or is designed to handle jobs with side effects AFAIK.

NixOS is great, but for me this is even more valuable, as I now have an introspect-able view of my entire deployed real estate, and can essentially `nix run ...` to deploy any part of my application in its entirety along with any dependencies.

There's also https://terranix.org/ to configure Terraform using Nix.

It's a generic deployment tool for Nix that also supports NixOS machines but can target any nix profile

[1] https://github.com/NixOS/nixops

That being said the article jogged my memory on some related thoughts...

If you’re building deb packages for your own apt repo aptly is a great tool for powering it https://www.aptly.info/

If you haven’t tried asdf for managing versions of tools it generally does what you’d expect and does it well https://asdf-vm.com/#/ It uses pyenv under the hood for Python management AFAIK.

(You can do the same with nix and direnv asdf - althrough I havent got that far yet)

But:

* if Nix is already very niche, Guix is the niche of a niche. The ecosystem is tiny in comparison.

* The Nix interpreter isn't exactly fast, but Guix (with it's scheme) was significantly worse for me, even in relatively small tests

* The hard stance on open source requirements is understandable, given the projects origin, but it will severely hinder adoption. No, I don't want to package every commercial app I need myself. nixpkgs has almost everything.

Documentation is a bit lacking, even though it has an excellent manual. How I am supposed to specify extra command line parameters to the kernel I boot in GRUB, for instance? Or specify multiboot targets? Get a "desktop" install without GDM? In the end, I went back to alpine on that machine.

Secondly, to use it as a standalone package manager, I wish it didn't require root access, or allowed me to put it in an arbitrary directory. Guix could work as a nice substitute for conda or docker, especially when it comes to reproducibility. I also couldn't find a guix docker image.

This is being actively worked on!

https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more...

> I wish it didn't require root access, or allowed me to put it in an arbitrary directory

I believe `guix pack -RR guix` will do package a relocatable guix for you.

Looking forward to the first, and I'll look into the second. One of my goals would be so easily offload to a more powerful computer, even if it doesn't run guix.

Edit: content-addressed guix stores delivery over a content-addressed network sounds interesting, it would be even better with rolling hashes (I think ostree/casync uses those).

I dream of a yunohost-like operating system based on Guix :)

In "normal" circumstances you don't generally have to do any builds at all - (almost?) everything should come from the binary cache. To be doing all these rebuilds it sounds like you're customizing packages, which is already something other distributions don't tend to let you do at all.

Memory is eaten by lazy evaluation of NixOS modules.

It can be optimized, but it is not a low-hanging fruit. It is still waiting for its hero. I'd compare it with creating the V8 for JavaScript.

Lately I encountered quite a lot of software - fringe and weird of course, but numerous enough - that went for having Nix as the only way to be deployed, typically with a "reproducible" script that doesn't work already after a year or two. As nix files make negative intuitive sense, not even in the smallest-scale bit-fiddling sense, so little intuitive sense that I cried once trying to just make it work - I look around for advice. One of "just take a couple of weeks to learn it, it's not that hard" is seen quickly enough, of course. It's a bit of pure loss from there: everyone who heard something like this knows that it's the moment to either invest fully or ignore and quietly move on and away. Confessing that you just don't think you'd find it in yourself to invest in it is almost a pure loss for one.

However, since we're not in one of those communities, let me say why not. I'm as sure that the good idea of reproducible builds will be implemented soon enough with something intuitive, strongly-typed, easily debuggable/diagnostable and dev-friendly, as I'm sure that diesel cars will be obsolete soon enough not to even think about investing my time and money in them. Regardless of what the current usage says - e.g. in Germany one would make a mistake to follow its wisdom just a few years ago, in case of cars, I mean.

* No multihost support, I want to share / between many hosts, nixos ideas support this but the implementation doesn't (and no I don't want to just share the nixstore)

* No package security, ideally package deps chains should have a per user/group created which won't allow broken programs from rm -rf / or worse.

* Nixos treats a filesystem as an object store, this goes against how most Unix tooling works (e.g how many Unix tools take a s3 obj?)

Things that might be incorrect on my part:

* No config validation, minor and hard though it would be nice if nixos had config validation.

* Single hash for store objects(?), it would be nice to use something like TLSH, etc in addition to a crypto hash

Although until the middle of the article I could make little sense of it because I had no clue what Nix was (have never heard of it), until I reach the part where Nix was introduced. Seems to be a very interesting tool.

Before that I used giant Docker image with all sorts of build tools installed/built, plus persistent VMs which had to be maintained manually, and Nix made it so much easier. Nixpkgs provides a good foundation - it has almost all software you may need and all the utilities to modify it or add new software. Need to patch or switch to custom version some obscure dependency of a compiler building another compiler building a library you use? Override a derivation, and all the dependent stuff will be rebuilt automatically. For building software Nix can be seen as a kind of super-Docker - Nix store allows for more granular caching than just layers, so incremental improvements can be done much faster. Essentially Nix turns files and packages into values in a programming language, so instead of hacky bash scripts trying to imperatively maintain a file dump, you just compose immutable packages by writing expressions.

That said, Nix is really hard to understand at first, comparable to Haskell/monad tutorials. I remember I made a few unsuccessful attempts at it over a ~6 month period, every time becoming more desperate, and then it finally clicked after careful reading of Nix pills [2] for a few consecutive days. To me, the most interesting thing to discover was that while Nix/nixpkgs do necessarily use some "hard" concepts like fixed point, it is in fact quite "old-school" and mostly about Unix, executables, linking, string templating, contains a lot of bash scripts, etc, so it's not really another Haskell. In fact, traditional building of C/C++ software with autotools is supported in Nixpkgs better than building modern stuff like Rust or Go, due to reliance of the latter on own package managers.

[1] https://insatia.kozinaka.com/

[2] https://nixos.org/guides/nix-pills/index.html

```

#!/usr/bin/env nix-shell

#!nix-shell -p "(pkgs.haskellPackages.ghcWithPackages (ps: [ ps.text ]))"

#!nix-shell -i runghc

{-# LANGUAGE OverloadedStrings #-}

import Data.Text.IO

main :: IO () main = Data.Text.IO.putStrLn "Hello, World!"

```

While nix is sometimes hard to learn, it is often a much simpler solution and it works for all languages so learning it is well worth it.

https://stackoverflow.com/questions/1316716/reproducible-cus...

It's kind of sad because it's a far better way to manage deployments and environments...but it takes 1-2 years of practice to really get proficient at

(Is that Conway's law in reverse, or just Occam's beard? I can't help but wonder.)

And I gotta get back to trying to build Deno from Git with Mozilla's overlays. Thankfully, Nix pairs well with git submodules...

Some tools might need to be used slightly different, others might no longer be needed.

Because it requires change the way you think about things is why so many people have problem with getting into it.

https://botsin.space/@complainingaboutmastercommits

Note that those statistics are incorrect, but pushes to master still do happen.

It’s also been awful practice to do this for as long.

While I have no skin in this game, my recommendation would be to immediately block all pushes to master from everyone.