That's a great theory, but Linus picks and chooses when to decide it's outrageous to make changes that break userland tools based on his personal preferences. They intentionally broke ZFS - then Linus follows up with a wildly inaccurate statement about why he thinks nobody should use ZFS.
It's difficult to take him seriously when he picks and chooses when to enforce his supposed "never break userland" ethos.
Did you read his mail though? He literally answers this criticism. ZFS is not userland so kernel developers are free to break it, just like they can break ext4 and do, and then they fix them. The problem with ZFS is that they can't fix it because it's not part of linux. It's a 3rd party kernel module. So your entire comment reads like a misunderstanding of the situation.
The breakage of ZoL wasn't a side-effect of some big technical change. They literally just made the SIMD enable/disable barriers no longer usable from non-GPL modules (the changed an EXPORT_SYMBOL to EXPORT_SYMBOL_GPL). This means you can't use SIMD-accelerated hashing functions from ZoL -- and there is no way to fix it without hurting performance other than reverting the "license change". And the entire justification for this change was "there are no in-kernel users".
Now, they are obviously free to do that and ZoL needs to work around it. But it's silly to argue that this was not an intentional breakage of a system which many people use and cannot be worked around out-of-kernel. Yes, it's not userspace code but it did cause regressions for users. Linus and GregKH have always disliked ZFS because they feel it was designed to be incompatible with Linux (and they're entitled to that view, however wrong it may be) so it's unsurprising that they are against doing anything that may help ZoL.
> ...disliked ZFS because they feel it was designed to be incompatible with Linux...
Ya... That may have just a bit to do with the fact that ZFS is released under a licence that was explicitly designed to be incompatible with Linux.
"Mozilla was selected partially because it is GPL incompatible. That was part of the design when they released OpenSolaris. ... the engineers who wrote Solaris ... had some biases about how it should be released, and you have to respect that."
I can see how that might suggest to Linux maintainers that they are not welcome to use this code. Maybe sorta.
Please stop spreading this. Simon Phipps, Sun's Chief Open Source Officer at the time, and the boss (?) of the person that you quote, has explicitly said this was not the case:
Sun needed a file-based license that had patent provisions. There were none available at the time so they created their own. Given that CDDL-license technologies (Dtrace, ZFS) have been incorporated into other open source (BSD) as well as closed source projects (macOS) shows that it is quite accommodating.
The incompatibility comes from GPL side terms. As for it being intentional, there are only anecdotes, and conflicting ones. When first CDDL sources dropped (DTrace) management expected to see it incorporated into Linux within month
Thing is, the copyright is owned by Oracle of all companies and and the legality is more than questionable enough to allow Oracle to sue for damages the second the code shows up in use by a big enough fish.
Oracle won't relicense the code (and lose out on a chance at another software copyright lawsuit), and it won't ever be safe to touch without an ironclad licensing story.
You can hope that Oracle folds and gets bought by someone decent. But hope won't take you that far.
Throw it away. Start over from scratch. The code may be great, but it's gone. It's a shame it ended up where it did. Take a moment if you need, but then move on. ZFS isn't going to happen.
Except Oracle isn't the copyright holder for majority of the code, as in fact, it cannot relicense huge chunks of OpenZFS code (which indeed is used in big commercial products), and it cannot "take back" code licensed under CDDL - it can relicense their own copy, not the one in OpenZFS, because there's no "version 1 or newer" clause that allows backdoor change to license like typical GPL case.
To summarize:
- Oracle isn't the only copyright owner
- Oracle can't take back CDDL license
- Oracle can't introduce new version of CDDL and magically change the rules
> because there's no "version 1 or newer" clause that allows backdoor change to license like typical GPL case
There is actually, and in CDDL (unlike in GPL) version updates are an opt-out feature rather than being opt-in. This was inherited from the MPL (which the CDDL is based on). See section 4 of the CDDL:
> 4.1. Oracle is the initial license steward and may publish revised and/or new versions of this License from time to time. Each version will be given a distinguishing version number. Except as provided in Section 4.3, no one other than the license steward has the right to modify this License.
> 4.2. You may always continue to use, distribute or otherwise make the Covered Software available under the terms of the version of the License under which You originally received the Covered Software. If the Initial Developer includes a notice in the Original Software prohibiting it from being distributed or otherwise made available under any subsequent version of the License, You must distribute and make the Covered Software available under the terms of the version of the License under which You originally received the Covered Software. Otherwise, You may also choose to use, distribute or otherwise make the Covered Software available under the terms of any subsequent version of the License published by the license steward.
In fact this is the primary argument that people give when arguing that Oracle could very easily make OpenZFS GPL-compatible -- they just need to release a CDDL v2 which says "code under this license is dual-licensed under the GPLv2 and CDDLv1.1". This situation has already happened -- CDDLv1.1 used this mechanism to change the "license steward" from "Sun Microsystems" to "Oracle".
Updates in GPL, thanks to standard boilerplate provided by GPL and used for years, are opt-out.
With CDDL, Oracle declares they have the sole right to provide newer versions of the license. However at no time can they treat it as an "upgrade path" for third party code, and OpenZFS code is explicitly labeled with CDDL 1.0
All of that has no impact on OpenZFS code which remains unencumbered, including by patents (the patent license is, afaik, the part that makes it incompatible with GPL the most).
ZFS is not user land. It's kernel code but without being in the kernel. So it has to be built separately and may not even be compliant with GPL (still has to be tested in court AFAIK). Linux's promise to not break userspace is just that, It can not be held responsible for code that should be in the kernel but does not play fair with the license.
It's not that it doesn't play fair. Sun published under a license that made inclusion in the kernel impossible thus the current developers are bound by the same decisions regardless of their opinion on the matter.
Furthermore it is profoundly curious to claim that a cross platform filesystem that began life on Solaris and existed for years as a stable complete work before being ported to several OS including Linux becomes by virtue of a bridge a derivative work of Linux. It's magical thinking.
It is entirely acceptable that they aren't responsible for out of tree code working but it would be great if they didn't deliberately sabotage other projects which is what they did here.
>My tolerance for ZFS is pretty non-existant. Sun explicitly did not want their code to work on Linux, so why would we do extra work to get their code to work properly?
The extra work is 10 seconds of work. How childish and unprofessional.
> Furthermore it is profoundly curious to claim that a cross platform filesystem that began life on Solaris and existed for years as a stable complete work before being ported to several OS including Linux becomes by virtue of a bridge a derivative work of Linux. It's magical thinking.
please don't confuse the source code and the resulting binaries. to the extent that the zsh source code does not derive from Linux, it is not a derivative work.
but the binaries built for Linux will necessarily be derivative of both.
and we regularly redistribute binaries; this indeed is what Linux distributions ordinarily do.
if you only write the bridging code and/or distribute the module source, you may be quite protected. but if you distribute a patch from the mainline to include zsh, or a mixed tarball of Linux+zsh, or even compiled Linux kernels complete with a loadable zsh module, you may be in breach. all of these clearly derive from multiple sources with incompatible licences.
> Furthermore it is profoundly curious to claim that a cross platform filesystem that began life on Solaris and existed for years as a stable complete work before being ported to several OS including Linux becomes by virtue of a bridge a derivative work of Linux. It's magical thinking.
It's not magical thinking. A work can be a derivative of multiple other works. It's completely plausible that adding Linux stuff to ZFS can make it a derivative work of Linux in addition to being a derivative work of whatever else it is already a derivative work of. There's room for debate about how much Linux stuff (and of what nature) something like ZFS can incorporate before it becomes a derivative work according to copyright law and the GPLv2, but you cannot dismiss the idea outright.
Magical indeed. Except that public opinion counts for nothing in copyright law; you have to fight it out in court. Luckily, Sun isn't the kind of company that would file questionable lawsuits against Linux users just to... wait.. wasn't Sun bought by another company?
Well, whoever owns the copyright now, you're probably reasonably safe unless they're a hyper-litigious corporation with deep pockets and an army of lawyers, who see lawsuits about software licensing as a potential source of income....
> Furthermore it is profoundly curious to claim that a cross platform filesystem that began life on Solaris and existed for years as a stable complete work before being ported to several OS including Linux becomes by virtue of a bridge a derivative work of Linux. It's magical thinking.
Let's say I make a movie M. It's a nice movie, but it doesn't have music. Someone else wrote some music S that is quite lovely. If anybody takes my movie M and mixes in music S, then the new movie is a derivative work of both M and S.
For a scary few days he was considering completely breaking the ABI of getrandom(2). If he decides to break interfaces, I don't know that anyone can stop him. It is a downside to the BDFL model.
Edit: Breaking ZFS is not a great example — as sibling comment points out, that is a kernel module using kernel-internal APIs, not userspace syscall ABI. The Linux kernel notoriously considers all kernel interfaces unstable. GregKH has in the past characterized this as a feature, not a bug.
> The Linux kernel notoriously considers all kernel interfaces unstable.
“You think you want a stable kernel interface, but you really do not, and you don't even know it. What you want is a stable running driver, and you get that only if your driver is in the main kernel tree.”
I think that it is a upside of BDFL model, unless it's someone incompetent. If you dig deep in the threads you'll find that so far his judgement on what to break or not has always been stellar. As long as you have someone as competent as Linus making the call, BDFL model is perfect.
RE getrandom(2): As Linus said several times, we don't break userspace rule does not apply to security bugs. If there's no other way to fix security bug other than breaking userspace, then userspace has to be broken, for obvious reason. If you don't understand that reason, you shouldn't really have a say in any such critical decision.
Unfortunately, Linus is dangerously incompetent at cryptography. As was the case with getrandom(2).
> RE getrandom(2): As Linus said several times, we don't break userspace rule does not apply to security bugs.
I don't think you understand or correctly recall the scenario in which Linus nearly broke getrandom(2). The bug Linus was attempting to fix by breaking getrandom(2) was not a security bug. It was an availability bug, in userspace, caused by broken userspace code deadlocking itself.
> Unfortunately, Linus is dangerously incompetent at cryptography. As was the case with getrandom(2).
I beg to differ. Of all people I know of, Linus is one of the most competent person when it comes to cryptography, more so than any of those Twitter celebrities. I suggest you dig into LKML archives. Exactly what in the case of getrandom(2) do you think makes him incompetent?
> I don't think you understand or correctly recall the scenario in which Linus nearly broke getrandom(2). The bug Linus was attempting to fix by breaking getrandom(2) was not a security bug. It was an availability bug, in userspace, caused by broken userspace code deadlocking itself.
I recall correctly. Optimizations in ext4 unveiled broken design of previous getrandom(2). To keep backward compatibility an easy choice could have been to supply early userspace with possibly predictable random numbers, which, of course would be a security issue. Another would have been to keep the bug hidden and revert any optimization that lowers early entropy pool. In the end, the solution at which the community arrived is best compromise given the constraints. The kernel is one of the few places where you can real software engineering to take place.
I think your comment is driven more by personal hatred for a great mind rather than technical merit.
> Of all people I know of, Linus is one of the most competent person when it comes to cryptography
This statement strikes me as bizarre because it's just so off-base. It seems you relate to Linus as some kind of personal hero, which is fine, but then use that as a reality-distortion field, which is probably unhelpful for you. (It's just confusing to others.)
To the best of my knowledge, Linus has not done any significant work in cryptography, nor is he regarded by any practitioner as an expert in the field. I would love to learn about any work he has done; I am just not aware of any, and I have some experience in the field.
> Exactly what in the case of getrandom(2) do you think makes him incompetent?
getrandom(2) is the only good random number ABI in Linux. It is the only way to request real random numbers that blocks until initial seeding and never blocks afterwards (i.e., BSD /dev/random behavior). It has a flag that can be used to avoid blocking.
Linus seriously proposed changing the ABI for getrandom() with flags specifying "please block until real random numbers are available" to silently ignore those flags if the routine would block, and instead return garbage. This would break all existing software that used getrandom() correctly and would be especially egregious for anything using getrandom(2) for cryptographic keys — which is a good and correct use of the interface.
His proposal was fundamentally similar to, for example, changing the ABI of /dev/random to silently return /dev/zero output instead of blocking.
> I recall correctly. Optimizations in ext4 unveiled broken design of previous getrandom(2).
1. The design of the getrandom(2) API was and continues to be fine. It is exactly what was missing on Linux, and has room for future expansion via additional flag bits. It is not broken.
2. The getrandom(2) ABI had been in the kernel for five years at the time Linus proposed breaking it.
3. The bug was in GNOME Display Manager, which ran early in boot on the particular system that lead to the report, and GDM managed to deadlock itself by incorrectly invoking getrandom(2) with a blocking request for entropy that it did not need.
4. (Broken user code can also deadlock itself by calling, say, sleep(99999).)
5. Yes, an ext4 optimization lead to slightly reduced initial entropy on that particular machine and the sighting of the underlying deadlock condition in GDM. But that latent condition was always present.
> To keep backward compatibility an easy choice could have been to supply early userspace with possibly predictable random numbers, which, of course would be a security issue.
This (1) does not keep backwards compatibility, and (2) is exactly the approach Linus initially proposed and had to be talked down from. This is the proposal that was dangerously incompetent. From the thread at the time:
Andy Lutomirski:
There are programs that call getrandom(0) *today* that expect secure
output… We can't break this use case. Changing the semantics of
getrandom(0) out from under them seems like the worst kind of ABI
break -- existing applications will *appear* to continue working but
will, in fact, become insecure.
Matthew Garrett (various emails):
We've been recommending that people use the default getrandom() behaviour for key generation since
it was merged. Github shows users, and it's likely there's cases in internal code as well. …
The semantics many people want for secure key generation is urandom, but
with a guarantee that it's seeded. getrandom()'s default behaviour at
present provides that, and as a result it's used for a bunch of key
generation. Changing the default (even with kernel warnings) seems like
it risks people generating keys from an unseeded prng, and that seems
like a bad thing? …
In one case we have "Systems don't boot, but you can downgrade your
kernel" and in the other case we have "Your cryptographic keys are weak
and you have no way of knowing unless you read dmesg", and I think
causing boot problems is the better outcome here.
The getrandom(2) saga was a long thread of Linus repeatedly ignoring the knowledge and concerns of the his own security deputies.
> In the end, the solution at which the community arrived is best compromise given the constraints.
Yes, catastrophe was avoided and in the end a happy resolution was reached — in spite of Linus, not because of him.
> I think your comment is driven more by personal hatred for a great mind rather than technical merit.
Alas most people are incompetent when it comes to cryptography. It's another of those situations when if you don't know why something has to be done a certain way, you shouldn't be included in that decision making progress.
It's difficult to take him seriously when he picks and chooses when to enforce his supposed "never break userland" ethos.