This is a utility I build to be able to make sure my JS CLI's actually work across the version I want to support.
Right now, the tool also works across any language where there is docker images with versions, not just for JavaScript/NodeJS. It's enough with having a Dockerfile with $VERSION (that will be replaced by the tool) and changing the test command. I will make this easier in the future.
Thanks for taking a look and please let me know what you think about it.
There have only been 3 major releases with known backwards-incompatible changes. The reason you would want to test your code in so many versions of Node would be to check that you aren't depending on newer features that aren't in older versions that you want to still support.
Testing across 32 versions is probably very overkill for most people though.
(Oh, I meant to say 3 major releases since 0.10, which I arbitrarily picked as what seemed to be the earliest version still in any significant use today.)
They're not strictly backward incompatible: one should ideally only have to test across the major versions (nodejs is currently on major version 5, though that's not strictly reflective given the fork to iojs, and the change in versioning system on merge back)
Testing across all 32 (or 303) versions is still going to be useful for catching bugs and quirks between minor versions (as opposed to deliberate, documented breaking API changes)
actually for the sake of correctness, you should also test version 4 which is lts (if you are a library author) (https://github.com/nodejs/LTS)
I mean v0.10 and v0.12 are also LTS but I doubt that if you have a bigger library that its nearly impossible to support all four
>Are all these 32 actually backwards-incompatible?
Node uses semantic versioning [0] and the current release is 5.10.z
The breaking changes can be found here [1] By searching for `SEMVER-MAJOR`. Some seem rather unavoidable for progress (updating dependencies...) and some are for things that have been deprecated. I'm not an expert but it looks like there are several security related changes as well?
In my eyes a backwards incompatible change that breaks a feature that has been deprecated for years or is insecure is a change worth making. Stability is bad if it means remaining insecure.
If you fix a bug that I'm relying on for the correct behavior of my code then it's not strictly backwards compatible.
These versions include a lot of minor and patch releases. Chances are most code will run just fine on most of these versions, but if you can automate testing them all then why not do it?
That is absolutely true, 32 versions is not all versions of node, but it's the number of tags from the mhart/alpine-node[1] docker images that autochecker is using for the testing.
Semantic versioning -- basically each version here is x.y.z for some x, y, z. Node patch releases (.z) happen reasonably fast and constitute a "version of node" as used here.
Even though Node isn't a framework (it's a cross-platform, runtime environment), it is used a base for application development. However, over time, things are changed and improved, but that breaks some of the API functionality upon which some existing applications are already built around. Since these applications still run fine with a previous version, they are not likely to be constantly updated to work with later versions. Therefore, previous versions are still offered to maintain compatibility with applications that have already been developed. However, if you're starting work on a new Node application, then you are better off using the latest version.
I'm more intereted in why would you need to test against 32 versions of node.js? Doesn't it maintain backwards compatibility? Shouldn't testing against one version cover it?
I would think the more fundamental question is -- are these 32 (or 303) different versions so non-backward-compatible that the need to test on all of them is just an accepted thing out in Node.js-land? That everyone just kind of shrugs and puts up with (the way MS Windows used to crash on a near daily-basis for nearly everyone, not too many years ago)?
I know that other languages also have issues in this regard -- but for more mature languages it seems to be more an exception than than the rule (restricted to certain platforms of subsystems, say).
This is definitely something that's needed, by myself, and many other developers. Determining module compatibility across versions is always a pain. Great tool!
Thanks everyone for the feedback and discussions, been very helpful and also motivating to see that people like/dislike the project in different ways!
As a small update, I've now made autochecker completely language agnostic, so you can test basically anything you can put in a docker container. There is some examples on how to use this here: https://github.com/VictorBjelkholm/autochecker/tree/master/e...
Again, thank you all for taking the time to give feedback, I'm forever grateful to the HN community.
Have you thought of running this as a service? Would be nice to just through an NPM module at it and make sure it will run across multiple versions. Without having to install docker, etc. Not sure I'd pay for it, but maybe somebody would...
Other languages have this sort of test service available for open-source modules. One example is cpantesters.org, which is integrated into Perl's package repository. Not only is it easy to check what passes in your environment, it also makes it easy for module authors to fix bugs in environments that the author has no (other) access to.
An aside - over the last few months I began running my containers on two popular CI services and have had some serious pain. I have a slight feeling you will regret the words: "works well with CI as well!"
Haven't thought about that but it makes a lot of sense to have that. If you don't mind, could you open up a issue on Github, explaining how you can imagine it to work to help you the most? https://github.com/VictorBjelkholm/autochecker/issues/new
Would appreciate that a lot and thanks for the feedback.
Frankly, I feel like this is an indictment of all software development, everywhere. When you need to do this kind of a thing, you are doing it wrong. I do not mean I am doing it any better, I have the same issues e.g. working on Android, but there are probably some fundamental things we could be doing to avoid all these horrible symptoms. Things like NixOS come to mind, where people are actively researching and experimenting with possibly better alternatives. I pray somehow somebody smart and motivated can find some fundamental solutions instead of this rube goldberg overkill type answer.
This tool is the solution. I don't think you're going to live in a static would anytime soon, where programming languages and packages do not change over time.
also 3) have enough time to implement all the features after having this magic insight. A lot of people take for granted that successful software makes tough choices to ship
I see where you are coming from, for sure. I wish I was smart enough and motivated enough to try to come up with a solution to this, but my motivation goes elsewhere.
Sometimes, for you to focus on other (what for you is important), you need to throw together some rube goldberg thing to be able to move forward.
That is exactly what I thought when I saw what had been posted. I have a feeling that we have a generation of developers whose work makes sense on a small scale or in the short term, but becomes ridiculous with growth and/or age.
JavaScript was never 'write once, run anywhere'. There were quite a few differences between the Netscape and MSIE implementations during the first few years of its existence and things didn't stabilise until the IE5 era.
You could write vanilla JS and your code still could fail in some versions of node because of bugs or breaking changes.
And in browsers, the JS language is the same, but the implementation of the various APIs has varied greatly.
Right now, the tool also works across any language where there is docker images with versions, not just for JavaScript/NodeJS. It's enough with having a Dockerfile with $VERSION (that will be replaced by the tool) and changing the test command. I will make this easier in the future.
Thanks for taking a look and please let me know what you think about it.