It seems to me that several circumstances have changed since the idea of dynamic linking first came about:
- A whole lot of software is now open and freely available
- The code-changes-to-binary-updates pipeline has been greased by a couple orders of magnitude, from internet availability to CI to user expectations around automatic updates (for both commercial and non-commercial software)
- Disk space, and arguably even RAM, have become very cheap by comparison
Given these, it seems worthwhile to re-evaluate the tradeoffs that come with dynamic linking
> Given these, it seems worthwhile to re-evaluate the tradeoffs that come with dynamic linking
Perhaps this is just a Linux distro thing, but as someone who closely monitors the Void packages repo, dynamic linking burdens distro maintainers with hundreds of additional packages which need to be individually tracked, tested and updated.
(Packages which could otherwise be vendored by upstream and statically linked during the build.)
Dynamic linking also adds complexity to distro upgrades, because dependant packages often need to be rebuilt when the libraries they dynamically link to are changed or upgraded. For example, Void’s recent switch from LibreSSL to OpenSSL borked my install script which wasn’t aware of the change. It resulted in a built system whose XBPS package manager couldn’t verify SSL certs. On Arch, AUR packages were notoriously prone to dynamic linking errors (back when I still used it).
Personally, I don’t find the bandwidth efficiency and CVE arguments for dynamic linking to be all that convincing [1]:
Will security vulnerabilities in libraries that have been statically
linked cause large or unmanagable updates?
Findings: not really
Not including libc, the only libraries which had "critical" or "high"
severity vulnerabilities in 2019 which affected over 100 binaries on
my system were dbus, gnutls, cairo, libssh2, and curl. 265 binaries
were affected by the rest.
The total download cost to upgrade all binaries on my system which
were affected by CVEs in 2019 is 3.8 GiB. This is reduced to 1.0
GiB if you eliminate glibc.
It is also unknown if any of these vulnerabilities would have
been introduced after the last build date for a given statically
linked binary; if so that binary would not need to be updated. Many
vulnerabilities are also limited to a specific code path or use-case,
and binaries which do not invoke that code path in their dependencies
will not be affected. A process to ascertain this information in
the wake of a vulnerability could be automated.
> Perhaps this is just a Linux distro thing, but as someone who closely monitors the Void packages repo, dynamic linking burdens distro maintainers with hundreds of additional packages which need to be individually tracked, tested and updated.
> (Packages which could otherwise be vendored by upstream and statically linked during the build.)
The effort is approximately the same for distro maintainers. Shared libraries might mean more individual packages but the same number of projects need to be actively maintained because if a vulnerability is patched in libssl (for example) you'd need to rebuild all projects that are statically linked rather than rebuilding libssl.
Upstream vendoring doesn't really help here. If anything, it add more issues than it solves with regards to managing a distro where you want to ensure every package has the latest security patches (vendoring is more useful for teams managing specific applications to protect them against breaking changes happening upstream than it is at a distro level where you can bundle different versions of the same .so if you absolutely need to and link against those)
Additionally I would say at least `dbus`, `gnutls`, `libssh2` and maybe?? `curl` are this kind of libraries which count as "widely used system libraries".
I.e. the few for which Linus thinks dynamic linking still can make sense ;=)
I think none of those (at least two, nearly surely) might qualify for that exemption. On the vast majority of systems I bet libcurl, for example, is probably linked against by 1 other piece of software - curl(1). “Really useful” isn’t the qualifier here. I bet ~same for libssh. dbus and gnutls I’d have to see more examples to understand.
I think that the kind of "knowledge" you are correcting is informing much of the discussion here. And it seems like the motivation is "I want to make MY thing as easy to ship as possible!".
There are hundreds of binaries in a Linux system and no one wants to rebuild all of them when an important library is updated.
That's with shared libraries and debug symbols stripped. If I upgrade libc, why would I want to also waste a massive amount of time rebuilding all 5616 binaries?
Of course, even if we ignore the insane buld time, I don't have the source for some of those binaries, and in a few cases the source no longer exists. In those cases, should I be forced to only use those programs with the (buggy) versions of the libraries they were originally built with?
1.3 GB? It's 2021, I can read that from SSD into memory in less than the blink of an eye. Hard disk is fast and cheap enough that it's worth keeping builds around to incremental rebuild against new libraries. Also, typically tests require a build tree, and you are going to rerun tests on the relinked binaries, aren't you?
Libcurl is the only sane way I know to access the web from a C or C++ environment. I expect that most C programs needing to do web requests will link in libcurl.
Thank you! I had noticed the same problem, but was not aware of this excellent solution. I just applied it, and it works perfectly... and I removed the dozens of Haskell libraries that were there just for Pandoc.
For anyone else interested, the same solution applies to shellcheck - just install shellcheck-bin from AUR.
I discovered that quite quickly after they switched.
More importantly (for any haskell developers using Arch), there are also static builds for stack, cabal and ghcup too although you then need to maintain your own set of GHC and libraries.
I can't imagine what it must be like to use the official packages since the haskell library ecosystem is quite fast moving.
The Haskell maintainers in Arch have an almost fantatic fascination with dynamic linking. Dynamic linking is otherwise not the norm in Haskell.
Haskell has poor support for dynamic linking. The Haskell (GHC) ABI is known to break on every compiler release. This means that every dynamically linked executable in the system needs to be recompiled. Any programs you compiled locally against dynamic system libs will break, of course.
This stance makes it much harder than necessarily for Arch users to get started with Haskell.
This is due to the maintainers on Arch and I've complained about it before too. They also update them almost daily so they just add noise to system updates. I just end up running the official Pandoc Docker container.
I'm assuming they already are using the package manager. The issue is that there are so many dependencies for a single package, and almost every system update consist of updating a bunch of Haskell packages which just clutters maintenance.
Distribution does it job: it distributes upstream changes. It's like blaming a zip file, because it contains many small files instead of a few big, so it clutters maintenance. If you see this as a problem, then fix upstream. For me, it's important to receive all upstream changes in one update.
No, I didn't. But, before uninstalling Pandoc, I was wasting time with frequent useless updates of minor Haskell dependencies that should be statically linked in the first place as no other package, at least in general, make use of them.
As someone who runs a Linux distribution, please don't vendor your dependencies for the benefit of distribution maintainers, it makes it much more difficult to build a coherent system. Running different versions of common libraries causes real annoyances.
It's a double-edged sword, though. I find that with Debian I often can't have up-to-date versions of programs I care about, because some program that nobody cares about won't build against an upgraded library version that they mutually depend on.
The requirement that every binary on the system depend on the same shared library basically led to things like app stores and Snap, which I think we can all agree that nobody wants.
Yeah, and I get my free ride by just downloading the binary from the project's website. Linux distributions add a bunch of arbitrary rules that I don't care about and that don't help me.
Yes this can't be emphasized enough. static linking is fine, I don't really care either way. But please, please don't vendor.
The point of software is to be composable, and vendoring kills that. Vendoring will cause free software to drown in its own complexity and is completely irresponsible.
It's one a package contains it's dependencies, and often their dependencies too (transitive deps).'
packages are supposed to be nodes in a dependency graph (or better, partial order), and there are nice composition rules. But one the nodes themselves represent graph/order closures (or almost-closure) we use that nice property.
People that are very application oriented tend to like vendoring --- "my application is its own island, who gives a shit". But the distro's point isn't just gather the software --- that's an anachronism --- but plan a coherent whole. And applications who only care about themselves are terrible for that.
The larger point is free and open source software really should be library-first. Siloed Applicationns are a terrible UI and holdover from the economics of proprietary shrink-wrapped software which itself is a skeuomorphism from the physical world of nonreplicable goods.
> Siloed Applicationns are a terrible UI and holdover from the economics of proprietary shrink-wrapped software which itself is a skeuomorphism from the physical world of nonreplicable goods
Siloed applications are UI neutral and represent decoupling, which enables progress and resilience. Excessive interdependence is how you lose Google Reader because other services have changed requirements.
What? Getting rid of Google Reader was a business decision.
> Siloed applications are UI neutral and represent decoupling, which enables progress and resilience.
The medieval manorial economy is resilient. Is that progress?
What happened to unified look and feel? and rich interactions between widgets? The smalltalk and Oberon GUI visions? All that requires more integration.
I get that we need to make the here and no work, but that is done via the en-mass translation of language-specific package manager packages to distro packages (static or shared, don't care), not vendoring.
> What? Getting rid of Google Reader was a business decision
It has been widely reported that a significant factor in that business decisions was Google’s massively coupled codebase and the fact that Reader had dependencies on parts of it that were going to have breaking changes to support other services.
> The medieval manorial economy is resilient.
It’s not, compared to capitalist and modern mixed economies, and unnecessary and counterproductive coupling between unrelated functions is a problem there, too.
> Is that progress?
Compared to its immediate precedents, generally, yes.
Fewer and fewer people are interested in the benefits provided by "coherent whole" distributions. And more and more people are interested in the benefits provided by "it's own island" applications.
The future is statically linked and isolated applications. Distros need to adapt.
1. Have the users been asking for everything to be a laggy electron app? I don't think so.
2. Within these apps are languages based package managers that don't encorage vendoring, it's just one people go to package for distros that they vendor away. Distros do need to make it easier to automatically convert language-specific package manager apps.
The future is making development builds and production builds not wildly different, and both super incremental, so one can easily edit any dependency not matter how deep and then put their system back to together.
Again, I am fine with static linking. My distro, NixOS is actually great at that and I've helped work on it. But vendoring ruins everything. I hate waiting for everyone to build the same shit over and over again with no instrumentality, and I don't have time to reverse-engineer whatever special snowflake incremental dev build setup each package uses.
The number of people who consider the system/distribution the atomic unit rather than the application, is probably about equal to the number of people who "edit dependencies and put their system back together" -- they are in total statistically zero. The overriding concern is the user need for a working application. Everything else is a distant secondary concern.
I'm not trying to convince you of anything, here, I'm just describing the facts on the ground. If you're not into them, that's fine!
The number of people who have a "theory of distribution" one way or the other is pretty low.
But
- many people seem to like unified look and feel
- many people complain about per-app auto-update
- many people love to complain software is getting worse
Are these people connecting these complaints to the debate we're having? Probably not. Can these views be connected to that? Absolutely.
---
I work on distros and due edit the root dependencies, I also contribute to many libraries I use at work during work, finally, I use the same distro at work on on my own and everything is packaged the same way. So yes, it's a "unified workflow for yak shaves" and I quite like it.
I hope there can be more people like me if this stuff weren't so obnoxiously difficult.
Yeah, sorry, I saw that later. (May have seen it earlier too, but not connected it to your name as I was replying.)
But that still doesn't make my comment a strawman (because I was talking about this one specific comment), or AFAICS yours less of one: Why would you jump to "bloated Electron apps"? Sure, they may suck, but the comment you replied to was about statically linked apps; no mention of Electron at all. Unless you're saying there was, originally, and had been edited out before I saw it? If not, your reply was... OK, more charitably, at least a non sequitur.
The "coherent whole" is more in demand than ever before. Just look at the Android and iOS ecosystems with super hard rules how things have to behave and look in order to be admitted. They just put the burden on the app dev instead of a crew of distro maintainers.
If you define "demand" as hard orders from the warden of your walled garden, yes. But that's not how the concept is normally used.
Personally, for instance, I'd have been perfectly happy if at least a few apps had stayed with the Android UI of a few versions back[1], before they went all flat and gesture-based. There was no demand from me, as a consumer, to imitate Apple's UI.
___
[1]: And no, that's not outmoded "skeumorphism". That concept means "imitation of physical objects", like that shell on top of Windows that was a picture of a room, with a picture of a Rolodex on top of a dedktop etc etc. In the decades since ~1985 a separate visual grammar had developed, where a gray rounded rectangle with "highlights" on the upper and left edges, "shadows" on the lower and right edges, and text in the middle meant "button" in the sense of "click here to perform an action", not any more in the original skeumorphic sense of "this is a picture of a bit of a 1980s stereo receiver".
> packages are supposed to be nodes in a dependency graph (or better, partial order), and there are nice composition rules. But one the nodes themselves represent graph/order closures (or almost-closure) we use that nice property.
A) "Lose", not "use", right?
B) Sounds like a lot of gobbledy-gook... Who cares about this?
C) Why should I?
> People that are very application oriented tend to like vendoring --- "my application is its own island, who gives a shit".
No, people that are very application oriented tend to like applications that work without a lot of faffing around with "dependencies" and shit they may not even know what it means.
> But the distro's point isn't just gather the software --- that's an anachronism --- but plan a coherent whole.
A) Sez who?
B) Seems to me it would be both easier and more robust to build "a coherent whole" from bits that each work on their own than from ones that depend on each other and further other ones in some (huge bunch of) complex relationship(s).
> And applications who only care about themselves are terrible for that.
As per point B) above, seems to me it's exactly the other way around.
> The larger point is free and open source software really should be library-first.
Again, sez who?
> Siloed Applicationns are a terrible UI
WTF does the UI have to do with anything?
> and holdover from the economics of proprietary shrink-wrapped software
[Citation needed]
> which itself is a skeuomorphism from the physical world of nonreplicable goods.
Shrink-wrapped diskettes or CDs are physical goods, so they can't be skeumorphic.
That's easy to say, but sometimes I need to fix bugs in downstream packages, and I am not willing to wait for 6 months (or forever in some cases) for a fix to be released.
Then Linux distro a year out my patched censored version and link to the buggy upstream, and I have to keep telling bug reporters to not use the distro version.
A useful thing to do when forking software is to give it a new name, to make it clear that it's a separate thing. It sounds like you copied some specific version of software you depend on, then forked it, but left the name the same -- which caused confusion by package builders at the distribution since it's a bit of work to determine if you forked the dependency or are just including it for non-distribution user's convenience.
> (...) dynamic linking burdens distro maintainers with hundreds of additional packages which need to be individually tracked, tested and updated.
To me this assertion makes no sense at all, because the whole point of a distro is to bundle together more than hundreds of packages individually tracked, tested, and updated. By default. That's the whole point of a distro, and the reason developers target specific distros. A release is just a timestamp that specifies a set of packages bundled together, and when you target a distro you're targeting those hundred of packages that are shipped together.
Also compilers nowadays are smarter and can perform link time optimizations. Meaning that if of a library you only use a single function, in the final executable you would only get that single function. In reality code that use static linking could be more efficient than code that uses dynamic linking.
And you have to consider some performance penalty when using shared libraries. First you have a time penalty when loading the executable, since first you have to run the interpreter (ld-linux) and then your actual code. But also for each function call you have to make an indirect jump into it.
One tradeoff is security. If you're patching vulnerabilities, then just a single .so needs to be patched. With static linking every binary needs to be investigated and patched.
You can also argue that it is impossible to update dynamic libraries because they are used by multiple applications and you can't afford that any application breaks. So instead of being able to patch that one application where the security is needed, you now have to patch all of them.
> You can also argue that it is impossible to update dynamic libraries because they are used by multiple applications and you can't afford that any application breaks.
That's where maintenance branches comes in. You fix only the security issue, and push out a new version.
Isn’t this especially true in the world of containerization? We literally ship entire images or configurations of OS’s to run a single application or system function.
Although, I have mixed feeling about containers, because I fully appreciate that Unix is the application and the software that runs on it are just the calculations. In that world, sure, a shared library makes sense the same way a standard library makes sense for a programming language. Thus, a container is “just” a packaged application.
Regardless, this concept is so out of the realm of “performance” that it’s worth noting that the idea of trying to reduce memory use or hard disk space is not a valid argument for shared libraries.
> Disk space, and arguably even RAM, have become very cheap by comparison
CPU frequency and CPU cache are remaining small, so smaller binaries, which fit the cache, are running faster, and use less energy, and wastes less resources overall.
> Given these, it seems worthwhile to re-evaluate the tradeoffs that come with dynamic linking
Create your own distribution and show us benefits of static linking.
To underscore the gp's point, I have had to work with a 3rd party-provided .so before. No sources. A binary. And strict (if unenforceable) rules about what we could do with the binary.
I dealt with a situation like this for a ROS driver for a camera, where the proprietary SO was redistributable, but the headers and the rest of the vendor's "SDK" was not. The vendor clarified for me repeatedly that it would not be acceptable for us to re-host the SDK, that it was only accessible from behind a login portal on their own site.
The solution we eventually came to was downloading the SDK on each build, using credentials baked into the downloader script:
The vendor was fine with this approach, and it didn't violate their TOU, though it sure did cause a bunch of pain for the maintainers of the public ROS CI infrastructure, as there were various random failures which would pop up as a consequence of this approach.
I think eventually the vendor must have relented, as the current version of the package actually does still download at build time, but at least it downloads from a local location— though if that is permissable, I don't know why the headers themselves aren't just included in the git repo.
To be honest, I haven't really tracked it— the product I work on dropped stereo vision in favour of RGBD, so I don't really know where it's landed. I suppose it's not a great sign that the current generation SDK still requires a login to access:
And at least one spinnaker-based driver seems to have inherited the "download the SDK from elsewhere" approach, though who knows if that's due to genuine need or just cargo-culting forward what was implemented years ago in the flycapture driver:
The "proper" approach here would of course be for Open Robotics (the ROS maintainers) to pull the debs and host them on the official ROS repos, as they do for a number of other dependencies [1], but that clearly hasn't happened [2].
I think a lot of hardware vendors who cut their teeth in the totally locked down world of industrial controls/perception still think they're protecting some fantastic trade secret or whatever by behaving like this.
I think this was precisely the point. Shared libraries create seams you can exploit if you need to modify behavior of a program without rebuilding it - which is very useful if you don't want to or can't rebuild the program, for example because it's proprietary third-party software.
- A whole lot of software is now open and freely available
- The code-changes-to-binary-updates pipeline has been greased by a couple orders of magnitude, from internet availability to CI to user expectations around automatic updates (for both commercial and non-commercial software)
- Disk space, and arguably even RAM, have become very cheap by comparison
Given these, it seems worthwhile to re-evaluate the tradeoffs that come with dynamic linking