> but nothing you said works in practice for python packages
How do transitive dependencies in the Python ecosystem work, then? I assume Django works with multiple versions of python and bcrypt. I assume pandas works with multiple versions of scipy. Is there no semantic versioning? If everything requires an exact version, how do you prevent everything from grinding to a halt?
> Is it fair to summize that python applications with python dependencies do not really work well as nix packages and shouldn't be used?
Let's not conflate Nix and Nixpkgs. Nixpkgs has its reasons for minimizing redundant packages, however it is certainly possible to package your app with Nix and use the exact specified dependencies.
> How do transitive dependencies in the Python ecosystem work, then?
Not very well.
> how do you prevent everything from grinding to a halt?
I don't have a good answer for you.
> Is there no semantic versioning?
You can read django release process here [1], not sure how it's relevant. I'm not the maintainer of django, but of a project using django. Would it be better if all software was perfect, had no bugs and used perfect semantic versioning? Yes, I would say so. Is that a requirement for using nixpkgs?
> Nixpkgs has its reasons for minimizing redundant packages, however it is certainly possible to package your app with Nix and use the exact specified dependencies.
I'm not packaging it, someone else is, it breaks and they come to the project to raise invalid bug reports.
Well you said earlier that nothing I said works in practice for python packages. My only point is that it must work at some level in the python ecosystem, else the ecosystem would collapse.
Anyways, it sounds like you're unhappy that someone did a bad job packaging your application. That sucks. Elsewhere in this thread someone mentioned that there isn't a strict single version policy in nixpkgs, so this can probably be easily fixed. I'd suggest filing a bug in Nixpkgs.
> Elsewhere in this thread someone mentioned that there isn't a strict single version policy in nixpkgs, so this can probably be easily fixed. I'd suggest filing a bug in Nixpkgs.
There isn't one but we are not collecting multiple package versions for no reason and since python itself cannot well handle multiple versions of packages they are only allowed outside of pythonPackages where all end user applications should live.
In this case I think it is important to distinguish nix (the package manager)
and nixpkgs (the popular package repository / distribution used with nix).
Packaging python applications with nix is doable, but you have to specify the exact versions of your dependencies and for that you can't easily use nixpkgs.
Nixpkgs tries to keep a minimum number of packages (like Arch or Debian as well), so each of the dependencies will typically only occur with one minor version for each release of nixpkgs.
We could still use the nixpkgs to build our application but we have to override each of our dependencies to the right version, but that approach can get quiet tedious for a large number of dependencies.
Fortunately there are tools to automatically generate your dependencies from a requirements.txt such as mach-nix or pip2nix.
What's the point of a minor version change if it's breaking? Does Django not have a versioning policy that enforces non-breaking changes between minor versions?
This is such a condescending attitude. What you mean is applications that are maintained the way that you and the Nix developers think an application should be maintained.
It's incredibly naive for a package manager as ambitious as Nix to assume semver. I'm a big fan of semver myself, but the vast majority of software projects follow it imperfectly or not at all, and for good reason—it's nearly impossible to follow it perfectly, because even bugs are part of your API. Every project I've worked on has eventually had something break on a version upgrade because we were depending on something that was later decided to be a bug (but at the time was just how it worked).
Elm can mostly get away with enforcing semver because they designed it that way at the language level, but Nix wants to manage dependencies written in all languages and ecosystems, which have dramatically different versioning practices.
Ah, fair enough, I misunderstood. I thought the tests they were recommending were tests to ensure backwards-compatibility between version bumps, I didn't realize they were talking about the downstream pacakge's tests.
I still disagree with the insinuation that it's everyone else who's screwing up and if we all did things the way Nix wants us to then Nix would actually work just fine. That's just another way of saying Nix doesn't work in the real world.
It is unfortunate that some in the nix community come off that way, because I would say that in general Nix goes to great lengths to adapt to the world as it is. Especially compared to, say, Bazel.
I myself have been using nix in an org that is blissfully unaware of nix for about 2 years, if that's any indication of how adaptable it can be.
You're totally right. If you're a package maintainer and you find out some package is misbehaving even though all if its included tests pass, it might kinda make you feel like kicking the thing and calling it junk.
But we should recognize that some of what drives that is just defensiveness, and some is personal frustration. At the end of the day, Nix and Nixpkgs are for letting people run useful software more or less as it exists. It's not just for users or developers of perfectly tested, bug-free software. (Nix itself is certainly neither of those things, and neither is Nixpkgs!)
Nixpkgs does not assume Denver that's why we run if possible the package's tests, our own tests and build dependent packages to make sure the most obvious breakages are noticed before things are even merged.
Ah, I thought you were saying that if we all just used e2e tests to ensure we didn't make breaking changes in minor versions, we'd be fine. I didn't realize you were talking about the downstream package's tests.
I do still take issue with your insinuation that it's the package maintainers' poor practices that are at fault here. The real world is a messy, complex place and "best practices" don't translate well from situation to situation.
OP didn't ask for their package to be included in Nix. Presumably OP's system works for them and for their use case, but whoever created the Nix package made assumptions that turned out to be flawed. It's not fair of you to say that those bad assumptions are OP's fault because their package isn't "properly maintained" and doesn't "work as it should".
Someone (you?) made a bad assumption. Don't cast blame for that on someone who only knows Nix exists because it sends phony bug reports their way.
Sounds like the problem is with Python maintainers who don’t understand that breaking changes should only be made between major versions.
If that’s not possible though then as sibling comment said - you can override the dependencies and the nix maintainer should make sure the package works as expected
Sounds like the problem could also be with Nix maintainers who don't understand that "semver" is not a universal law of nature and that not all projects and ecosystems follow it. This kind of blanket dismissal can cut both ways.
Semver (the website and "spec") was created in 2009 by some guy. It's not an RFC, a standard, or anything like that. Yes, it gained widespread adoption. Yes, the guy in question is a cofounder of GitHub. So what? You cannot force it upon everyone. Python is about 20 years older than semver. Django is several years older. Should the whole ecosystem change their conventions because it's more convenient for a few people?
Except Django site says that a.b are feature releases which should be backwards compatible except for specific exceptions. If their software truly breaks “with every update to django_4” then it’s either a problem on Django’s side or a problem in how said person uses Django
> * Versions are numbered in the form A.B or A.B.C.
> * A.B is the feature release version number. Each version will be mostly backwards compatible with the previous release. Exceptions to this rule will be listed in the release notes.
> * C is the patch release version number, which is incremented for bugfix and security releases. These releases will be 100% backwards-compatible with the previous patch release. The only exception is when a security or data loss issue can’t be fixed without breaking backwards-compatibility. If this happens, the release notes will provide detailed upgrade instructions.
Going from "mostly backwards compatible with the previous release. Exceptions to this rule will be listed" to "should be backwards compatible except for specific exceptions" is quite the stretch. There are no "specific exceptions": incompatibilities can be anywhere and you need to read the release notes to know where. In semver, a minor version increment is backwards-compatible, no exception, no ifs or buts.
If you want to shoehorn Django's release process into "semver", then act as if the product is called "Django 4". If the version is "Django v4.X.Y", then X is the major version number, Y is the minor version number, and there is no patch version. It should be version in Nix as "django4 vX.Y.0".
They clearly say “exceptions to this rule will be listed in the release notes” meaning that backwards compatibility is the rule. There’d be no exceptions if there was no rule hence I said they “should” be backwards compatible except for specific exceptions, which shall be noted in the release notes.
Not sure how this conversation is productive, but there's never been a X.Y release of django without noted backwards incompatible changes to my knowledge. Just imagine that djangos X.Y releases are semvers major releases, not much more to it than that.
It also very clearly states that there may be exceptions to the rule. So a package repository that assumes that Django follows semver is unequivocally doing the wrong thing, because Django is very clear that they don't (otherwise there would be no exceptions).
Is it fair to summize that python applications with python dependencies do not really work well as nix packages and shouldn't be used?