There was a strange and mutually self-supporting pair of ideas in the Plan 9 community at the time:
1) "Shared libraries are bogus."
2) "Anyone who likes normal-looking user interfaces rather than plain boxes with text in them is a poopipants."
Both of these propositions are contentious to say the least, but what bothered me was that two propositions were mutually supporting while being (to my mind) orthogonal. The obvious and most compelling example of a shared library on the Unixen at the time were all the various UI libraries (Motif and various other abominations; all of them huge and unwieldy). It seemed necessary to accept that these libraries were obviously Completely Unnecessary to buy into the Plan 9 idea that shared libraries didn't do anything worth mentioning.
I'm sure it's possible to design a better UI library (or maybe even a wacky user level file system for user interfaces; in fact, my honours project in 1993!) but at the time the way people made interfaces that looked vaguely like what other people expected computer programs to look like was to use big-ass shared libraries on Linux.
This was also the way (dragging in one of those godawful blobs like Motif etc) that anyone might have exerted themselves to port across a (not completely pitiful) web browser, but the degree of aggressive disinterest in supporting Other Peoples Code was extreme (Howard Trickey's "ANSI POSIX Environment" or "APE" didn't get much love as far as I could tell).
It was quite undignified to watch people struggling with text-based web browsers or firing them up on non-P9 boxes because of the inability to support the web at the time.
For the most part, MFC was and is a thin wrapper over the user32 and comctl32 libraries that ship with Windows. So the true Windows equivalent of statically linking Motif would be to statically link user32 and comctl32, and that's impossible.
Really? I have no numbers to back it up, but it felt to me that Visual Studio (and Visual C++/Basic before that) were more popular than the Borland tools.
Back in those days, Visual Studio was priced a lot higher than Borland offerings - this goes all the way back to DOS, actually. Borland had always priced its developer tools very aggressively compared to the state of the market, and they reaped the benefits. Thus, Turbo Pascal or Turbo C++ used with Turbo Vision was by far the most popular DOS TUI (as opposed to, say, VB for DOS), and OWL/VCL was the most popular Win16/32 framework, if you discount raw Win32 API (of which it was a wrapper).
Visual Basic ate a considerable chunk of Delphi's lunch in the enterprise area, where price was less of an issue, and large dev shops would buy an MSDN subscription anyway. But it was never more popular than Delphi for desktop apps.
"Was there not a version of Motif that could be statically linked?"
The GP seems to be confused about the meaning of "shared library", and seems not to be aware of the existence of static libraries -- which would (as you point out) be perfectly usable in the case of Motif.
'Perfectly usable' is a bit of a reach given how huge Motif was at the time. I shudder to think about the size of a directory with a bunch of X applications that each snarf in all their dependencies; remember we're talking 1990s.
No, GP is “grandparent”, that is, the post two posts upthread from the post in which the term is used (or, in some contexts, the user who posted the post two posts upthread.)
Similarly, GGP is “great grandparent” (3 posts up), and so on.
OP is “original post” or “original poster”, which is (in the general case) a different thing than GP.
I think I first encountered it on Slashdot a decade or more ago, but I've rarely seen it outside of Slashdot or, more recently, HN, so I think it's a Slashdottism that migrated to HN with Slashdot users.
Though I could be wrong, as my exposure to internet message boards is weirdly eclectic, but far from comprehensive.
It seems to me that the issue isn't between shared code and no shared code but between statically and dynamically linking that code. It's a question of bundling dependencies. You can generally statically link a GUI toolkit; you just have to deal with the extra disk and network usage. You can dynamically link the same library but then you just have to deal with all the familiar dependency issues you've probably dealt with. (DLL hell and related issues) It's a trade off, but in either case you're still sharing code.
DLL hell happened because Windows used to lack a package manager, leaving apps to bundle their dependencies and overwrite other versions (without semver names!) in common directories.
Yeah, Windows has traditionally been worse than Linux in this capacity, but my point is that dependency management is a non-trivial problem. My experience is that package managers usually work but not always (I've had occasional issues on Arch) and the fact that you need a package manager, and how sophisticated they often have to be, shows how tricky the problem can be to solve.
To be even more specific, it happened because apps dumped DLLs into shared system directories instead of bundling them alongside the app binary. Since on Windows, each app has its own directory, it's easy to bundle in a way that doesn't mess up other apps.
I mean, if all apps avoided writing to the global folders (i.e. reserve it for OS-managed components), there wouldn't be an issue even with the old arrangement.
But not even Microsoft's own products respected that in practice.
But then, you still have to put up with defining potentially hundreds of <bindingRedirect>s in config files to even get an application to run correctly, even though they mostly don't share their DLLs.
The only thing it takes is Nuget dowloading a newer version of some dependency another library depends on, even if it's backwards compatible with the old API. Which, of course, it should always be if you haven't switched to a new major version.
Edit: Also, I find it very disturbing that the resulting errors always happen to be crashes during runtime instead of up-front. If that happens with a plugin or through reflection: fine. But for most of .NET code, dependencies should be able to be loaded during startup.
Static libraries are a potentially bigger problem when your library is tightly coupled to a particular version of some other process (eg an X windows server).
The real distinction in my view is between tightly coupled components and loosely coupled ones. Loosely coupled as a design paradigm is harder to design and maintain for the programmer and in some cases has a performance impact, but a lot nicer for the admin and feels ‘cleaner’ to me, at least.
As a UI person, I think shared UI libraries are bad for usability in many ways. They lead people to design UIs that happen to be made of the widgets that are available, rather than designing something truly suitable to the user task.
This is one of the reasons the web eclipsed native applications. The web only provided the most basic common widgets, so designers were forced to reimplement the UI toolkit, but also given the freedom to have their own take on it.
I personally would prefer to see a UNIX-like UI library made of many composeable parts. With idependant codebases.
In that world, having a single giant dynamically linked UI blob doesn’t help.
I’m not saying standardization is bad, just that forced standardization at the architectural level is bad.
The theory of UI libraries is that users can take their knowledge between applications. When you start using a new app, you already know mostly how it will behave, because it shares the UI vocabulary with other apps.
The web has been terribly violent to this idea. Native UIs are expected to support keyboard navigation, keyboard shortcuts, a menu bar, type select, drag and drop, accessibility, scripting... And in any given interaction there are advanced modes: multiple selection, modifier keys, etc.
Hardly any of this works on the web; even if it did you wouldn't think to try it. Does type select works in GMail's custom context menu? Can you command-C copy a message in one folder, and paste it in another? Would it even occur to you to try those things?
That stuff is ancient I know, and it would be one thing if the web were pioneering new UI vocabulary that displaced the old. But it's not. There's nothing new that has taken hold, no replacement for what is lost. Gmail has its janky, app-specific keyboard shortcuts, which is at least something, but there's no mechanism for it to spread.
We're in a Dark Age. Every web page now has its own custom CSS-style menu that only supports the most basic hover-and-click, and the bar for good UI is just lying on the floor.
As a person who has to use those web apps that supposedly "eclipsed" native applications, I hope this stage will pass, and we're back to the sanity of consistent UI across apps and frameworks.
Just so I'm following properly, do you mean something like how you have {React | React Native | React VR} + a cornucopia of community-supplied custom components? imo it's a system that works well - you have a common system + easily extensible custom bits.
(take my experience with a grain of salt, I've only used React on side-projects, never anything complicated or that I was forced to develop on because of work, and never ran into any perf issues)
Not the person you replied to, but that's what I imagine he/she is getting at. What I think of when someone mentions a Unix UI library is something like Qt, which provides a number of conventions, such as:
- Standard layouts
- Standard input and display components
- Standard event and message handling
- Standard user interactions
- Standard color themes
Which is great for developers to set up an initial UI, but makes every application look the same and takes a lot of work to customize in a way that's consistent with the defaults.
On the other hand, building a complex, feature-rich UI requires a lot of skill both in terms of implementation and design, which is less than common. So that Qt UI will be functional and predictable to users, even if it was not very well thought out, and the broken web UI will be useless to everyone.
This is exactly why I like to use these UI toolkits, and despise most web "UI" frameworks. That uniformity is why these applications are usable and predictable, and yes, boring. Deviating from the defaults takes effort, and that's a subtle hint that it's something to reconsider. You're deviating from the norm, and that might indulge a developer's whims or the demands of a manager who wants their application to stand out from the crowd. But that superficial difference is something I've despised ever since we have WinAmp and XMMS skinning the UI like many media players, purely to look different at the expense of usability.
Even after reading lots of UI guidelines, and developing several tools and applications, it's taught me that I'm terrible at it, and should let an expert do that part while I stick to the other parts!
Personally I like consistency. I've actually moved away from GUI apps in general because the terminal is more consistent, so I feel like there's less to learn with each new tool.
Yes, I should’ve said UNIX-philosophy or something... what I said was pretty ambiguous. I meant UNIX-style in terms of single purpose tools that work together over a common bus without being tightly coupled.
I found during my ten years in Plan9 it was the most usable OS I have experienced in my 35 years of computing. My only problem with it is every other platform is now ruined because I gnash my teeth and say "would have been easy in Plan9".
or as relevant at the time: how did you communicate via msn/aim/yim/icq? I hope you're not one of those condescending Unix types that only had online interactions on irc :P
No need to be rude, and not everyone uses chat software.
Googling for the web browser options online, it seems Links[1] and Abaco[2] are the best options right now. There are others listed in Wikipedia[3].
I wonder if there's someone out there that has put in the effort to improve their Plan9 environment. I wonder how difficult it might be to port something like Chromium, Firefox, or Webkit to Plan9. Maybe it's not as tremendously difficult as I imagine it, but maybe I'm dreaming thinking this.
So what's your favorite operating system? Is there one you find ideal or do you like different ones for their different trade-offs? What do you wish was the most widely used OS in the world?
I ask these questions thinking of the OS foundations and ignoring that OSes like Plan9 and Inferno are lacking many things only found in more popular OSes by virtue of being popular.
Plan9 was far superior as a developer's system. It's text based tools are the best, bar none. But really it owes that to Project Oberon.
"Everything is a file" as an abstraction enables you to build really powerful through composition. e.g. running TCP/IP over a serial connection just by mounting files in certain places.
Your end user program doesn't need to know anything about networking, it just reads from files.
It's not perfect, of course, it had problems with throughput - we were forever discussing at conferences how to improve the protocol to implement streaming.
I describe it as a racing car, not everyone can drive it, you need to work to keep it going, you might kill yourself but when you're out front, it feels amazing.
Speaking as someone who's used IRC for nearly a decade and who has developed extensively for it, it most definitely has not stayed simple, what with all the nonstandard extensions every network has brought with it. And while I dislike Discord for many reasons, its moderation tools is not one of those reasons; I'd like to see a good, open chat standard come along that adapts features such as invite tokens, rather than depending on accounts or IP addresses or someone manually inviting you into a channel.
To have a functional, modern IRC client, you have to not only implement what there is in the RFCs (which incidentally, no IRC daemon follows anymore, not even the original ircd2) but you also have to pull in IRCv3, SASL, ISUPPORT parsing, CTCP parsing, a TLS library, ... the list goes on. And for a good experience you also want scripting and the ability to abstract away the differences between services implementations (Atheme and Anope, mainly). Not to mention you have no clientside encryption option on IRC short of any number of buggy OTR implementations.
IRC is not great. It might be okay, but by no stretch is it great.
Memory usage, extreme lag, the bugs (apparently I need firefox 37+ to access the voicechat feature, I am at 65 though), centralized (a big issue considering the censorship that they apply), no way to use it over tor (as far as I know at least), you are not allowed to make/use a 3rd party client, the official client is closed source, no e2e encryption or authentication, etc.
Discord wastes a few seconds of my time every time my Mac boots up because it can't just open at login without stealing the focus once or twice for no good reason.
Netsplits, hilarious lack of security (nickserv identify uses plain text passwords sent over an unencrypted TCP connection), you can't retrieve channel history, no support for threads, images, basic markdown formatting... Shall I go on?
Eh, most of these seem like features to me. Slack's threading support is enraging, and I consider plaintext a courtesy. One of the fallouts of the lack of centralisation for things like chat logs was a thriving bot culture (years before it became cool and VC fundable).
Some of these are already in place, eg. IRCCloud can render simple markdown, supports push notifications, can show inline images, supports display pictures, Slack-like threads, emojis, etc...
Hopefully longer term we won't need to use IRCCloud/a bouncer to keep chat history. AFAIK an IRCv3 server should be able to offer that natively (to an IRCv3 client).
As for unencrypted TCP... a lot of IRC networks seem to offer TLS these days.
Not saying this is perfect -- far from it, but people are working on improving IRC!
Very little of that is a problem in practice, and proprietary systems like AIM had more than their fair share of problems too.
>no support for threads
That one's just wrong. You can make a threaded client if you want one, although it's not totally clear why you would since the CPU load of IRC is going to be very low in any reasonable situation.
Edit: Oh I see, threads as in forum threads.... just open a new channel. Otherwise, the concept seems to map poorly to ephemeral real time discussion. This is the first I've ever heard of AIM,etc of having forum threads.
Slack-style threads work very well for real-time discussion. It's far easier to follow a channel with them than without.
Opening a new channel wouldn't work for fairly obvious reasons (nobody would be in it, you'd effectively have to do it for every single message, etc.).
SASL auth and TLS are fairly common nowadays, threaded discussion is not really mandatory for a good modern chat (Telegram, Discord for example don't have anything like that,) image unfurling can be done by the client, and there is actually formatting and some clients will autoconvert basic markdown to it for you.
IrcCloud is an example of a client that tries to solve pretty much all the issues and you can even plug it into a desktop IRC client and get most of the functionality including history. It's not necessarily better than modern chat or anything, but I love having a modern chat experience while being able to discuss things with people and groups who virtually only use IRC.
If you aren't a fan of hosted services you can run your own BNC. Some IRC servers have built in BNC services that give you things like better security and channel history.
Let me reaffirm that I am not of the position that IRC is perfect or modern. But you are misrepresenting it a fair bit, even in 2006 I was connecting with TLS (sometimes self signed certs though.)
How is no channel history a feature? I think if you take a look at your reasoning you'll see that you are just defending IRC because you like it. Which is fine, but it is intellectually dishonest to pretend that flaws are not flaws.
I’ve assumed the 2018 version of the dynamic library view is bad is that non-operating system dynamic libraries are bad (and operating systems should provide a GUI library), use a web browser, or use the command-line.
Starwman, nobody said that. Shared libraries make sense when a program wants to load additional code at run time. A classical example is loading and unloading plugins.
For everything else, not so much.
2) "Anyone who likes normal-looking user interfaces rather than plain boxes with text in them is a poopipants."
Most people I know use a tiling window manager with terminals. So plain boxes with text in them seems to make sense for many people.
It's hard to recall conversations from 1994, but I came away with the overwhelming sense that the Plan 9 guys thought shared libraries were bogus - certainly that they were bogus for the non-plugin use case (which is the one that pertains to this discussion; libraries like Motif were not plugins). So you've accused me of raising a strawman (actually a "starwman" - "he'd like to cwome and mweet us, but he thinks he'd blow our mwinds") by raising a point absolutely not germane to the discussion, but you when refute my point with this devastating rebuttal:
"For everything else, not so much".
Most people I know also use the web and menus and buttons and dialog boxes and so forth and expect all this stuff to look vaguely like other computers do. The fact that the Plan 9 folks were trundling over to non-P9 machines to read the web seemed to suggest that they also liked seeing things that weren't just text in plain boxes.
The point remains that these two propositions are both at the least contentious, and only by combining these unrelated points could anyone really seriously take Plan 9's approach to shared libraries (at the time) seriously.
If Plan 9 guys really thought shared libraries were bogus, they wouldn't have created the follow up version heavily based on dynamic packages for Limbo.
Your "most people I know" comment is bizarre: even hardcore terminal users need to browse the web, and it's been years since lynx or w3m could be claimed to be adequate for most web tasks.
But the Plan 9 folks consciously made it difficult to write or port web browsers and other consumer software. I think I remember Tom Duff stating that writing a web browser was a "fool's errand", which I guess captures the sense of heightened seriousness, privilege, and lack of concern for the average mainstream user that informed Plan 9's design.
There's more discussion in this file between Howard Trickey (who refers to himself as an "erstwhile fool", having written Charon for Inferno), Duff and other Plan 9 enthusiasts.
The displayed level of knowledge and engagement with the requirements for building a web browser isn't exactly off the charts. I think it's fair to say that Duff didn't have a lot of patience with the project. He saw it as a moving target that probably couldn't ever be hit. For Plan 9 the classic web browser is a culturally alien type of software: complex and to some degree necessarily monolithic.
Here's what Duff said:
> how come mothra bit the dust?
I wrote it. It was not in good condition when I left
Bell Labs. Understantably, nobody else wanted anything
to do with it, so it died.
Its biggest shortcoming (other than its general
internal hidousness) was that its document imaging
model was fixed before <table> entered the picture.
Deep down it believed that documents were running text
with embedded line-breaks and indent changes, meaning
there's no good way to get tables or frames to work.
Also, if your browser doesn't closely match Netscape
and Microsoft, people will believe that it just doesn't
work, regardless of how good a job you do of meeting the
published specifications.
On the other hand, I still think its idea of how to handle
navigation (mostly the panel with an LRU list of pages
visited) was better than anything else I've seen.
Writing a web browser is a fool's errand. The
specification was changing faster than I could type, and
still is.
And here's Rob Pike:
> Surely somebody must be trying to port Mozilla to Plan 9?
While that would be better than no browser at all, Mozilla
is just the sort of stand-alone monolith that we're trying
to argue against. And of course, everyone else in the
world is trying to turn their system into a giant web
browser. I'd rather see web access be one aspect of a
system in which the pieces work in concert.
-rob
I think most people who've given the web browser platform any serious thought would agree that the way Duff and Pike are talking is indicative of not really having taken the problem seriously. The discussion continues in that speculative vein.
They had the luxury of being able to choose (effectively) to reject the web user experience and they took that option. Tom Duff probably wouldn't claim that he was speaking about the intricacies of web browser development with great authority!
There's more discussion in the 9fans archive. And see also Abaco, another attempt at a graphical browser for Plan 9: http://lab-fgb.com/abaco/
On the subject of web browsers, I wonder if Plan 9 aficionados would appreciate edbrowse [1]. As the name suggests, it's a combination ed-like editor and web browser. It includes the duktape JavaScript engine. The project even distributes a statically linked binary for Linux using musl.
In theory, that could even be ported to some versions of Plan 9. I remember that the Jehanne OS dev had said something along the lines of "Netsurf (another Duktape browser) could be ported soon".
> Most people I know use a tiling window manager with terminals. So plain boxes with text in them seems to make sense for many people.
As my human-computer interaction prof said back in undergrad: You are not normal. There is a reason that no modern operating system outside the "Other Linux/BSD" bucket ships with this as a default.
Most here seem to know that the motivation for adding DLLs to unix was to make it possible for the X windowing system to fit in the memory of a computer of that time, but many comment writers here seem not to know something that the participants in the discussion that is the OP all knew:
Plan 9 has an alternative method for sharing code among processes, namely the 9P protocol, and consequently never needed -- and never used -- DLLs. So for example instead of dynamically linking to Xlib, on Plan 9 a program that wanted to display a GUI used 9P to talk to the display server, which is loosely analogous to a Unix process listening on a socket.
Problem with that is that it is less efficient than a shared library but also susceptible to the same issue, namely that if you change the interface then things break.
Edit: It does of course get you extra functionality, like being usable over a network connection. Trade offs.
Directly dispatching a function via a call per the ABI of your computer vs producing a serialized message stream, sending it to another process, deserializing a stream and then calling the function.
Yes, it's less efficient.
The better question might be "did it matter in practice?".
I once replaced an export operation that depended on COM calls to Excel with some text manipulation code and brought the run time down from 30 minutes to an hour to just a few seconds. So in at least that case it did matter in practice.
There are examples of very efficient IPC, just not in UNIX-land. If your IPC system is integrated with the scheduler, the difference between a dynamic library call and IPC can be minimized into irrelevance.
I am repeating stuff I learned over the years from internet discussions, e.g., on the 9fans mailing list, rather than from direct experience in writing GUIs in Plan 9 and in X. I think when the decision was made to add DLLs to Unix, Xlib, the library a program would use to talk over the socket, was itself too big to fit in memory if a separate copy got statically linked to every program that displays a GUI. (The Wikipedia page for Xlib says that one of the two main aims of the XCB library, and alternative to Xlib, were "reduction in library size".)
I'm not advocating for removing DLLs from our OSes, BTW. Nor am I advocating for Plan 9.
COM uses vtables and normal calls via function pointers, unless the caller and the callee are in incompatible contexts that require marshaling (remoting, different processes, or different threading apartments in the same process).
SunOS before 4.0, when it still used SunView¹ instead of X11, still did not have dynamic linking. Hence this email rant by John Rose titled Pros and Cons of Suns from 1987 (as included in the preface of The UNIX-HATERS Handbook²):
[…]
What has happened? Two things, apparently. One is that when I created my custom patch to the window system, to send mouse clicks to
Emacs, I created another massive 3/4 megabyte binary, which
doesn’t share space with the standard Sun window applications
(“tools”).
This means that instead of one huge mass of shared object code running the window system, and taking up space on my paging disk, I
had two such huge masses, identical except for a few pages of code.
So I paid a megabyte of swap space for the privilege of using a
mouse with my editor. (Emacs itself is a third large mass.)
The Sun kernel was just plain running out of room. Every trivial hack
you make to the window system replicates the entire window system.
Oh my, each app is going to be 20 mb bigger! This mattered 30 years ago, but now I would say we have a huge problem for end users with all of these "Package managers" and "dependency managers" getting tangled up because there are 5 versions of Perl needing 3 versions of Python and so on... I would be a much more happy Linux user if was able to drag and drop an exe. 100mb be damned
This seems easier in both Windows and OS X. On both, a native application gets its own directory, and will first look in there for any shared libraries it needs. It gives you a nice middle ground between static linking everything and dynamic linking everything that still avoids the "we have to choose between pervasive system-wide dependency hell and Dockerizing everything" situation that seems to exist on Linux.
True for Windows, but not true for macOS. See DYLD_FALLBACK_LIBRARY_PATH in https://www.unix.com/man-page/osx/1/dyld/ -- the dynamic linker will look for the library at the proposed absolute or macro-expanded path (specified in the load command), and failing that look for the leaf name under a few fallback paths:
$(HOME)/lib:/usr/local/lib:/lib:/usr/lib
You'll notice that those fallback paths do not include any location relative to the application. It's actually pretty difficult to get the object code to lookup libraries relative to its own directory. Explanation here:
The program sets LD_RUNPATH_SEARCH_PATHS which bakes a list of paths into the binary. This can include relative paths as well as @loader_path and @executable_path. eg a plugin bundled with an app can specify @executable_path/../../Frameworks to reference the parent bundle's Frameworks directory. Libraries can also add search paths to help locate their dependencies. @loader_path will expand relative to the library that actually has the dependency.
Any linked library with a DYLD name of @rpath/ will be searched in those locations.
At build time the linker checks each library for its DYLD name and bakes that name into the final binary. Making sure all of your non-system dependencies are @rpath and relative to your app bundle is what makes it a "Relocatable" application.
Out of curiosity, what stopped you from setting the -install_name flag when you built the libraries you are bundling with your juicysfplugin plugin? That's the standard mechanism for building app-relative libraries and wasn't listed at all under "Alternatives to manually rewriting dynamic links"
You're thinking of "application" as a single binary, which is unnecessary. You wrap up all of core-utils into a single environment and launch the shell.
If you want the space saving characteristics of shared libraries but your OS simply doesn't support that, you could probably do the busybox trick of rolling a bunch of binaries into one and changing which codepath gets executed based on which name the monolith binary was invoked with. Of course there are obvious downsides to this approach.
Doubt that, looking at my list of applications - I have 124. So, 100mb * 124 = 12.4 gb and I have half a terabyte of storage... Windows 10 requires 16 gb of storage. Heck most phones have 64 gb these days.
We need to stop living in the past - this very issue has probably contributed to the scourge of Electron crap being thrown at our faces since it doesn't suffer from dependency hell they just throw a bunch of javascript into an encapsulated instance of chrome and call it a day.
I have 971 binaries in /usr/bin alone. If they were 100mb each, I'd be looking at 94GB of space on a 250GB laptop ssd. 94GB that I'd have to re-download every time there's a security patch to a common library (e.g. libc). I'll keep living in the past and use shared libraries until download speeds and disk space increase by a couple orders of magnitude.
1. most files in /usr/bin are much smaller than 100mb when statically linked; 100mb is for e.g. gui applications
2. Even in an absurd world where each coreutils executable required 100mb of libraries, a busybox-like delivery would already shave ~10GB off of that. Other improvements can be made: binary deltas for security updates, performing the final link step at package install time, probably others.
3. libc updates have introduced security issues; shared library updates in general break things. I can take a statically linked executable from 1998 and run it today.
Lastly, this is totally unrelated to the question because 971 statically linked command line applications will be well under 1GB, but a 250GB drive? The last time I had a drive that small was a 100GB drive in a PowerBook G4. Linux (and p9 from TFA) are OSes used primarily from the developers (at least until the mythical year of linux on the desktop). Springing $200 for a 512GB SSD upgrade seems well worth it if you are being paid a developers salary anywhere in the western world.
Too late to edit, but gnu coreutils statically linked is 8.8MB total for 105 executables versus 5.5MB for Ubuntu's dynamically linked version.
The largest executable is ptx at 272kb vs 72kb for the Ubuntu binary.
For the smallest, false is 48k statically linked vs 32k for the Ubuntu binary.
If all 970 executables in /usr/bin average out to 100kb of extra space, that's less than 100MB overhead.
[edit]
Stripping the binaries decreases the size to about 7MB total or byte sizes of 34312 vs 30824 for a stripped false binary and 251752 vs 71928 for ptx.
For download times, a tar.xz is a good measurement and it's 819k for the 105 statically linked files or 1015k for the full installation of coreutils including the info files and manpages.
[edit2]
Some proponents of static linking talk about performance, I think it's a negligible win, but as I have it handy I thought I'd measure:
>For the smallest, false is 48k statically linked vs 32k for the Ubuntu binary.
Lol, you have to do some kind of stripping or GC sections or what not for this to be a fair comparison. A proper version of false is 508bytes on my machine.
Clearly most of those 971 binaries wouldn't be that large as they only use a subset of the shared libraries functionality and therefore everything else could be left out.
This is actually an area of very current research. We have implemented a form of software multiplexing that achieves the code size benefits of dynamically linked libraries, without the associated complications (missing dependencies, slow startup times, security vulnerabilities, etc.) My approach works even where build systems support only dynamic and not static linking.
Our tool, allmux, merges independent programs into a single executable and links an IR-level implementation of application code with its libraries, before native code generation.
I would love to go into more detail and answer questions, but at the moment I'm entirely consumed with completing my prelim examination. Instead, please see our 2018 publication "Software Multiplexing: Share Your Libraries and Statically Link Them Too" [1].
How does your tool handle the dynamic libraries loaded on demand during the life of the program? Specifically, where the application depending of the user input dynamically loads only one out of the set of shared libraries which all are made to be linked with the main application and use the same interface but are designed to be "the only one" loaded? That is, both the application and each in the set of the libraries expect to have only 1-1 relation (only one library loaded at the time)? Edit: OK, reading further your article, I've found: "our approach disables explicit symbol lookup and other forms of process introspection such as the use of dlsym, dlopen, and others."
If you'd manage to implement that too then it seems that really big projects could be packed together.
In Linux, if libssl is compromised, you install a new libssl. In Plan 9, if libssl is compromised, you re-install Plan 9. That's static linking for you.
Yeah that works if your app is in the Debian's software repository (and Ubuntu's and Red Hat's and Gentoo's and etc. etc.). If that is the case it is trivial to update all apps that depend on libssl anyway, even if they use static linking.
In practice it is far easier for a lot of software to distribute Windows-like binaries where all but the most basic dependencies are included (e.g. Flatpak or Snappy). In that case dynamic linking doesn't help at all.
Modern Ubuntu systems rely on Snap or Flatpak for a lot of software. What these systems do (as I understand it) is package a large amount of the dynamic libraries that would be provided by the operating system, and stick them in a compressed file (or virtual file system, whatever).
So what you essentially get is a 200MiB 'binary' without any of the benefits of dynamic linking (being able to swap out a library given a vulnerability without recompiling) OR static linking (a single file, with the extraneous code removed, etc. etc.).
With flatpak, you can update the runtime (a bundle of libraries) independently of the app, so you can swap out a library with a vulnerability without recompiling.
Flatpak has a concept of runtimes, shared among multiple applications, which are basically a well-defined bundles of libraries. So yes, dynamic linking helps there.
The entire plan9 system takes less than 10 minutes to compile from scratch on one core of my 11 year old laptop. OpenSSL alone takes 2-3x that on two cores.
On the original RaspberryPi it took a minute to recompile the kernel from scratch and 4 minutes to recompile all the standard programs. In comparison it took 10 to 11 hours to recompile the Linux kernel. Cross-compiling the plan9 kernel on a 2009 era amd64 computer took 20 seconds computer. And rebooting took few seconds.
I think it's silly to dismiss static linking like TFA seems to do but I don't think your point is very fair. Assuming that you have a proper package manager upgrading all applications that link to libssl would definitely be a much larger download that merely libssl.so but it could be handled automatically and without too much fuss.
> One of the primary reasons for the redesign of the Plan 9 security infrastructure was to remove the authentication method both from the applications and from the kernel. Cryptographic code is large and intricate, so it should be packaged as a separate component that can be repaired or modified without altering or even relinking applications and services that depend on it. If a security protocol is broken, it should be trivial to repair, disable, or replace it on the fly. Similarly, it should be possible for multiple programs to use a common security protocol without embedding it in each program.
> Some systems use dynamically linked libraries (DLLs) to address these configuration issues. The problem with this approach is that it leaves security code in the same address space as the program using it. The interactions between the program and the DLL can therefore accidentally or deliberately violate the interface, weakening security. Also, a program using a library to implement secure services must run at a privilege level necessary to provide the service; separating the security to a different program makes it possible to run the services at a weaker privilege level, isolating the privileged code to a single, more trustworthy component.
The paper goes on to explain how the various cryptographic services are exposed as a file server. This is the Plan 9 way of doing things: have lots of small programs that talk to one another.
The new model of One Version, all updated together is interesting in this context. Examples are iOS, Chrome, Firefox and node_modules. All super complicated with many dependancies. Update everything, fix broken stuff. Only maintain the one blessed dependency graph.
If you report an iOS or Chrome bug where you tried to revert a library upgrade and something broke, they'll just mark it "Won't fix: omg never ever look at this".
The dependency graph when everyone isn't updating all at once is brutal. Half of Unix life is/was "well I need to update X, but can't because Y depends on old X. Now we'll just create this special environment/virtualenv/visor/vm with exactly the brittle dependency graph we need and then update it, um, never."
We complain about One Version/Evergreen, and should, but it's got huge advantages. And might be an indicator that testing surface is the real complexity constraint.
One Version's success a good indication that Plan 9 was at least not totally wrong.
Arch Linux's approach is similar. The only version of the OS that is blessed is the current version. Every package install should come with a full system update. Package downgrades aren't supported.
In the case of an irreconcilable "X requires Z v1 but Y requires Z v2" they fork package Z.
Shared libraries are a pain for sure. They also have a lot of really nice advantages, including:
- You can upgrade core functionality in one location
- You can fix security bugs without needing to re-install the world
- Overall, they take up less disk-space and RAM
- They can take much less cache, which is significant today
The cache aspect is one that I'm surprised not to see people talk about more. Why would I want to blow out my CPU cache loading 20 instances of libSSL? That slows down performance of the entire system.
It's not without cost, though. Moving messages around causes CPU data cache pressure and CPU cycles you wouldn't otherwise have spent if you merely referenced shared memory that's mapped into your process.
Right. I didn't mean that to be why it is correct. Just giving some comment to what was actually suggested. Static linking is not the only alternative to dynamic linking.
> How is this functionally different from a shared object?
Because it's a separate process, so when it crashes your application can put up a "SSL failed, reconnecting" message and carry on. Also when your application has read-something-somewhere security vulnerability, it can't compromise your SSL keys.
Nitpick: it's not one process, but one executable; you can have multiple SSL daemons using the same (read-only) binary, so a attack on one only gets one set of keys. (The same attack will probably work on every instance of the same version of SSLd, but shared objects don't fix that.)
It isn't really. The actual big problem with shared libraries is that developers cannot be trusted not to change their interfaces or behavior and break things. That's true regardless of what mechanism you use to share code.
This is far too narrow a view. It's not a question of whether to dynamically link or not, but WHERE and HOW to dynamically link.
Think about it: If you were to force absolutely everything to be statically linked, your KDE app would have to include the ENTIRE KDE library suite, as well as the QT libraries it's based on, as well as the X window libraries those are based on, etc etc. You'd quickly end up with a calculator app that's hundreds of megabytes.
But let's not stop there, because the linkage to the kernel is also dynamic, which is a no-no. So every app would now need to be linked to a specific kernel, and include all the kernel code.
Now imagine you upgraded the kernel. You'd have to rebuild EVERY SINGLE THING on the system to build with that new kernel, or a new version of KDE or QT or X or anything in between.
The kernel membrane is a form of dynamic linkage for a reason. Same goes for IPC. Dynamic linkage is useful and necessary; just not to such a microscopic level as it once was due to size constraints.
The key is not to eliminate dynamic linkage, but rather to apply it with discretion, at carefully defined boundaries.
> Think about it: If you were to force absolutely everything to be statically linked, your KDE app would have to include the ENTIRE KDE library suite, as well as the QT libraries it's based on, as well as the X window libraries those are based on, etc etc. You'd quickly end up with a calculator app that's hundreds of megabytes.
That is false. Static linking only links in the the parts that are actually used, which would be a small fraction of the total.
Almost all parts end up being used though - for the vast majority it's not possible to prove that it's never used.
KCalc would for instance pull in HTML rendering for the Help dialog, which would also add all available picture format plugins and also PDF writing support etc.
And as you begin to flesh out some kind of calling convention, you pretty quickly end up with what could very easily be called dynamic linkage of sorts, even if it's not strictly the linking of object files, etc.
Yeah, I figured that'd be a response, but if you're gonna call mime type handlers dynamic linking we may as well give up on using the term at all.
I just don't think "I need to let the user open an HTML document, though not as any core part of my program's functionality" is a strong case for "well I guess I'll have to embed an entire HTML rendering engine if I statically link my deps".
It feels like your missing his point. OP is pointing out that static linking everything is necessarily unworkable. You're arguing that static linking isn't unworkable because you can just dynamic link. It doesn't respond meaningfully to the OP.
I may be totally misunderstanding asark, but the model I thought they were suggesting was something like how COM works in Windows. You can, for example, talk to other COM components and pass data back and forth using an efficient binary message passing protocol while still letting them live in their own process space.
It allowed some interesting things. For example, you could write applications that could read, display and modify the contents of Excel files, except they didn't do it directly; they delegated all the actual work of opening, reading, and modifying the *.xls file to Excel itself.
I wouldn't personally consider consuming a COM interface like that to be a form of linking.
That said, getting back to the broader context, I've no idea how you'd make something like that work for a GUI toolkit. But WinRT is supposedly based on COM, so maybe they got something figured out?
No, I'm arguing that including a manpage doesn't mean your program is now dynamically linked to the man command, or that receiving a .doc attachment in my email client doesn't mean my email client is now dynamically linked to a word processor. If you're going to call that dynamic linking then we've officially reductio'd this out of the realm of usefulness.
Maybe. But for an application opening html documentation, the convention already exists.
xdg-open [URL]
Opens the URL in the user's default browser, which is a wholly reasonable and expected way for a program to behave. Moreso I'd say than using shared libraries to pull up some sort of kludgy KHTMLPart thing.
No, by default, everything is included, at least in GCC. Go ahead and statically link against some unused .a of your choice and run objdump if you don't believe me. There are some special compiler flags that can be used to prune unused functions, to some extent, but they aren't on by default.
I mean static linking is not on by default either so you have to flip a flag as well. Adding -ffunction-sections -fdata-sections to the compiler and --gc-sections to the linker is not hugely difficult, although of course you have to make sure that the static libraries were built that way as well.
That being said I agree that even with that you'd still end up with a massive binary because of all the code that's actually used in one way or an other and all of the code that the compiler and linker can't statically mark as dead. In particular anything being called indirectly through a function pointer is almost certainly going to end up in the final binary regardless of whether it's used or not.
And even if a lot of the code gets removed it's a bit silly not to reuse the shared code in RAM instead of allocating pages containing the same thing over and over again. It's wasteful and cache-unfriendly.
> Think about it: If you were to force absolutely everything to be statically linked, your KDE app would have to include the ENTIRE KDE library suite, as well as the QT libraries it's based on, as well as the X window libraries those are based on, etc etc.
no, because we had working LTO for a long time - every KDE app would contain only the exact code that it needs, down to the individual member function. Sure, all KDE apps would have a copy of QObject / QString / QWhatever - but most C++ code nowadays is in headers anyways .
> You'd quickly end up with a calculator app that's hundreds of megabytes.
The DAW I'm working on (340kloc), which can be built statically linked to Qt, LLVM, libclang, ffmpeg and a few others, is a grand total of 130 megabytes in that case.
It's actually heavier when distributed as set of dynamically linked things (unless shipped by linux distros of course, and most of that size is useless LLVM things that I haven't found how to disable yet) :
only accounting for the dependencies I have to ship, it would be 176 megabytes - plus 17 megabytes of the actual software. I'd argue that in that case dynamic linking actually takes less disk space overall because most people don't have LLVM installed on their machines and don't need to.
The code needed is more than you might think. The transitive usage graph is surprisingly dense. FYI I'm one of the original Qt developers, and I remember how it was to try to slim down Qt for embedded use.
Java is like that too. The smallest possible program (just a single line to exit) requires things like java.time.chrono.JapaneseChronology. Because argv is a String array, and making the String array requires calling a function or two that can throw exceptions, which requires the Throwable class, which has a static initialiser that requires... the chain is long, but at the end something has a SomethingChronology member, and so AbstractChronology.initCache() is called and mentions JapaneseChronology.
A friend of mine tells a story about how how he accidentally FUBARed the tests, and then discovered that running one unit test gave 50% line coverage in Rails.
We have big libraries nowadays, and we use them...
This is zone of the things I like more about HN. You read a random answer to a comment and end up discovering he's one of the original authors of QT. Thanks for the nice work!
Regular LTO for large sized projects (think Chromium sized), in my experience is by far the biggest bottleneck in the build process. It's partially the reason so much is being invested into development and improving parallel linking as well as techniques like ThinLTO that all aim at reducing the link times since often, LTO linking takes around 60-70% of all build time combined despite the heavy use of C++ code (although with no exceptions or RTTI).
Unless you have build servers capable of rebuilding all Qt, WebKit etc. and performing an LTO link (which pulls in all build artifacts in form of bitcode archives/objects) in a reasonable amount of time (big reason buildlabs exist - it takes a long time), LTO is not likely to be suitable, it's an extremely expensive optimization that essentially defers all real compilation until the link step at which the linker calls back into libLLVM/libLTO and have them do all the heavy lifting.
At the very least you need a workstation grade machine to be able to do that kind of stuff on regular basis, you really can't expect everyone to have that. And there's a reason libLLVM.so is usually dynamically linked, it cuts a massive amount of time spent on builds, which is especially useful while developing and it's a middle ground between building all LLVM and Clang libraries as shared objects and having to wait for static linking of various LLVM modules into every LLVM toolchain binary (which tends to result in the toolchain being much much bigger). The build cache with shared libLLVM.so for Clang/LLVM/LLD builds is around 6-7GB (Asserts/Test builds). Statically linking LLVM modules blows that up to 20GB. God forbid you actually do a full debug build with full debug information with that.
That's a terrible argument against dynamic linking. That's not to say static linking is bad, in fact, recently it's been making a comeback for exactly that reason - LTO and next-generation optimizers. But saying LTO makes static linking viable for everyone including consumers is somewhat far fetched.
> At the very least you need a workstation grade machine to be able to do that kind of stuff on regular basis, you really can't expect everyone to have that. And there's a reason libLLVM.so is usually dynamically linked, it cuts a massive amount of time spent on builds, which is especially useful while developing
I of course do not argue doing LTO while developing, it seemed clear for me that the context of the whole thing is about what's released to users.
> your KDE app would have to include the ENTIRE KDE library suite, as well as the QT libraries it's based on, as well as the X window libraries those are based on, etc etc. You'd quickly end up with a calculator app that's hundreds of megabytes.
If you look at the transitive dependencies and then only link in the code that is actually reachable, I doubt that static linking has a significant impact. (I don't know to which degree this is possible with today's toolchains, but I suspect that many libraries have some inefficiencies there due to missing dependency information). As an uneducated estimate (from someone who has written compilers and GUIs entirely from scratch) I'd say 100K of additional statically linked object should be enough for almost any desktop application.
> the linkage to the kernel is also dynamic
Not an expert here, but you don't link to the kernel at all. The interface is via syscalls. At least on Linux, to help decrease inefficiencies at the syscall boundary, there is some magic involving the so-called VDSO, though - which is indeed dynamically linked, but shouldn't be very large either (On my system it seems the [vdso] and [vsyscall] mappings (both executable) both have size 8K. There is also a non-writeable and non-executable [vvar] of size 8K and I guess it links to live variables written by the kernel).
> As an uneducated estimate (from someone who has written compilers and GUIs entirely from scratch) I'd say 100K of additional statically linked object should be enough for almost any desktop application.
This is completely misguided. For a modern UI library it is very very hard to figure out what code is actually used.
Say we load the ui description from an xml file. What controls get used in the app then? Maybe the xml has a webview control in it? Then we can't drop the webview dependency.
Then your app needs to load an icon, which typically goes via some generic image loading API. What types of loaders are there? Can we get rid of the gif loader (and dependencies)? If any single image file reference is not hardcoded in the binary, again we can't.
Want to script your application in lua/python/js, and let it call into the UI library? Then we have to assume that basically any code can be reached.
Yes I agree. I does not work with deep nested dependency chains where some choices are only decided at runtime (after loading additional text files).
Maybe it's possible, though, if we remove a little flexibility and flatten the dependencies, such that the application hardcodes a little more of what actually happens (e.g. specificies the file formats for image loading, the format of the ui description, etc).
This is the way I've done my own GUI application, where I simply coded what should happen, and created widgets as I needed them, and all the important decisions were known statically. But maybe this amount of specification is not what people expect from a standalone GUI library.
It is possible to do it. And it probably happens naturally for a handrolled toolkit. However when the (one) developer of the GUI is different than the (many) app developers using it things typically end up different.
For instance you end up with things like ui designer tools that produce manifest files, where you just expect that some non-programmer can edit it and load it into your app without having to rebuild the toolkit.
Statically linking the OS is quite common nowadays [1], better known as containerization.
For quite a while there has been a movement that every program or "executable" is statically linked.
In your example, the "OS" is a program on it's own, it's controlled by a single vendor and released/updated at the same time. So dynamic linking it fine there
For a downloadable program statically linking Qt would seem prudent however, otherwise an incompatible version on the OS would render the program unusable.
You see this design decision nearly everywhere where legacy is not too big a concern: Go and Rust statically link its runtime and dedendencies in executables, JVM consumer apps are usually bundled with a specific JVM, containerization etc.
[1] edit: masklinn correctly pointed out that usually the kernel is still shared between containers and the host OS
> Statically linking the kernel and OS is quite common nowadays, better known as containerization. For quite a while there has been a movement that every program or "executable" is statically linked.
Containers specifically do not statically link the kernel, the entire point is to have multiple isolated userlands use the same underlying kernel. That's why they're so cheap.
Ok, yes, but that's just the kernel, and it's cheap only relative to a full VM. A container may or may not be cheap depending on the language runtime and whatever helper apps you're bundling. Bundling the python interpreter or a Java VM into every app isn't cheap.
This reminds me of early web servers and CGI's. A CGI can be relatively cheap to run as long as you write it in C and don't link in too much; not so much for a scripting language.
Redundancy can happen at multiple layers, so deduplicating at one layer might not be enough, depending on the situation.
> Fair enough, although it is the case if the host OS is different then the container, VM would be a better name then I guess.
If the host OS is different than the container's you need either an intermediate hypervisor / VM (pretty sure that's what happens for docker on OSX, and why it's much more expensive than on linux) or for the host to provide an ABI compatible with the VM's OS/expectations (WSL, smartos's branded zones).
Either adds even more "dynamic linking" to the pile.
> So maybe not kernel, but the entire userland/OS of the app is still statically linked with containers.
Yes, a container's userland should be statically linked.
Do you have any evidence to support your claim? My understanding is that containerized processes are theoretically the same as any other process on the system in terms of runtime performance.
In plan9 they are exactly the same, not theoretically. Isolation happens on a file system level (everything is a file, but truly, not a "special device file").
While it's practically not possible to compare performance, just the fact that this is a part of the initial design and a core of Plan9 vs adhoc feature (actually ported from Plan to Linux) is enough to speculate on performance.
>Linux namespaces were inspired by the more general namespace functionality used heavily throughout Plan 9 from Bell Labs.
> While it's practically not possible to compare performance, just the fact that this is a part of the initial design and a core of Plan9 vs adhoc feature (actually ported from Plan to Linux) is enough to speculate on performance.
It really doesn't. For all we know, because the plan9 version has to be more general it is much slower than the ad-hoc version.
> Statically linking the OS is quite common nowadays, better known as containerization.
Shipping prepackaged images with self-supplied libraries is functionally equivalent to shipping a blob linked statically. But the implementation is different, in a container, each program still loads the self-supplied libraries using the loader, and this difference matters. Only dynamically-loaded libraries with PIE executables can enjoy the fully benefit of Address Space Layout Randomization. But statically-linked executables cannot, and they leave fixed offsets that can be exploited directly by an attacker.
> Security measures like load address randomization cannot be used. With statically linked applications, only the stack and heap address can be randomized. All text has a fixed address in all invocations. With dynamically linked applications, the kernel has the ability to load all DSOs at arbitrary addresses, independent from each other. In case the application is built as a position independent executable (PIE) even this code can be loaded at random addresses. Fixed addresses (or even only fixed offsets) are the dreams of attackers. And no, it is not possible in general to generate PIEs with static linking. On IA-32 it is possible to use code compiled without -fpic and -fpie in PIEs (although with a cost) but this is not true for other architectures, including x86-64.
Unless we stop writing major programs in C one day, and/or formally verify all programs in the system and provide a high-assurance system that most people would use, using exploit mitigation techniques is still the last line of defense to limit the damage created by a memory-safety bug.
After thinking for a while, I think should change my stance, and support containerized applications such as Docker or Flatpak despite it's an ugly blob. A buggy and ugly container with prepacked applications linked dynamically to vulnerable blob with PIE and ASLR, is less of a evil than a buggy and ugly executable, linked statically to vulnerable blob with fixed addresses.
You are conflating a lot of terms that have nothing to do with each other.
Static linking would never cross process boundaries, would optimize for modules that are actually called from the program that the libraries are being linked to and has absolutely nothing to do with IPC.
Dynamic linking is useful, but not always necessary and not always appropriate and does not result in 'calculator apps that are hundreds of megabytes', it might be 10Mb or so, which is still large but not comparable.
If I'm not mistaken shared libraries are also shared (for the ro sections) in RAM. Saving 10MB on disk is mostly negligible nowadays but 10MB in RAM less so, and 10MB in cache is very much significant.
I really think that the linked post throws the baby out with the bath water, library versioning issues can be handled by the package manager to become effectively non-issue. DLL hell is a thing on Windows because it lacks a proper package manager. On a case-by-case basis applications that can't be packaged properly (proprietary binaries for instance) can be statically linked.
In my experience that's how it works on Linux and for the BSDs and as far as I'm concerned it hasn't caused many issues. Maybe I've just been lucky.
Shared libraries are shared in RAM between processes. But I believe the cache for each process is not shared. If three processes currently have the same shared lib code (or any shared memory) in their working set, they will each work it into cache independently. This sounds inefficient but is rare in practice and maintains process isolation. The added complexity of cache sharing would not be worth it.
Kcalc doesn’t use the whole qt library, the whole X library, the whole
kDE library, or the whole C library. When you statically link, only the functions that are actually called are included. This can trim a lot of space.
> When you statically link, only the functions that are actually called are included.
This is not true in general.
For this code:
#include <stdio.h>
int main(void)
{
puts("hello world\n");
return 0;
}
both GCC 7.3.0 and Clang 6.0.0[1] on my system generate an 828 KB binary when compiling with -static. The binary includes a whole bunch of stuff, including all the printf variants, many of the standard string functions, wide string functions, lots of vectorized memcpy variants, and a bunch of dlsym-related functions.
That's great, but can you now demonstrate the same with Qt and KDE, as you claimed above?
FWIW, I believe (but could very well be wrong) the reason that this works this well with MUSL is that every function is defined in its own translation unit. I doubt that Qt and KDE do that.
The library/project has to be designed right. Musl has dummy references that make static linking work without pulling in a bunch of junk. This is very much a language/project issue and I wish more languages actually looked at the huge mess of dep graphs that are needed for simple programs.
I think you will have better luck static linking against a different libc, like uClibc or musl libc. Statically linking against glibc doesn't work very well, and isn't supported by upstream.
uClibc and musl libc have much better support for static linking, and you will be able to make a much smaller binary with them than you would static linking against glibc.
It's not that complicated. -ffunction-sections -fdata-sections arguments to gcc coupled with --gc-sections argument to the linker will get rid of most of the fat.
> The key is not to eliminate dynamic linkage, but rather to apply it with discretion, at carefully defined boundaries.
Question: In the right system, does discretion even matter? Eg, NixOS - where linked libraries and dependencies are defined by content addressing. Should we use discretion with linked libs in a system like that?
When I started developing in golang I used to prefer static binaries without cgo 100% of the time. Now that I have moved to NixOS and each binary can either use the one in the system or provide their own libc if it is different I feel much better about it. I don't think we need to use discretion in a system like that.
Several comments have already pointed out many ways that this take is incorrect, but I'll also mention that software like KDE is the antithesis of plan9 design and I bet I'll be 6 feet under before we see it ported.
> But let's not stop there, because the linkage to the kernel is also dynamic, which is a no-no. So every app would now need to be linked to a specific kernel, and include all the kernel code.
> Now imagine you upgraded the kernel. You'd have to rebuild EVERY SINGLE THING on the system to build with that new kernel, or a new version of KDE or QT or X or anything in between.
You could though, that's pretty much what unikernel systems do.
You'll still likely have dynamic linkage of sort to the hypervisor or (ultimately) the hardware.
As a converse, its very "nice" to have programs in /sbin and some in /bin to be statically linked.
And when I say nice, I mean absolutely essential. Ive had glibc go south before... And then nothing works. Its a "hope you can boot with rescue media and hope and pray the old glibc is in your apt/rpm cache".
But having the rest of the system being dynamic linked makes a great deal of sense for the reasons you stated.
This isn't so much an answer for 9p as a description of why GCC symbol versioning is confusing.
> The symbol versioning breaks assumptions users have about how shared
libaries work -- that they provide a link to one version of a function and
if you replace the library all the programs get fixed. I've seen this
problem in practice, for both naive users and very non-naive sysadmins.
I guess I am heretic for breaking my project to a collection of DLLs. Ironically I am doing it over a 2 mil lines of code statically linked code base. The statically linked code base takes 150 seconds to build and my much smaller project only 4 seconds.
I have also designed it that way to do live coding in C and I have made a similar library for live coding python .
The idea was that individual programs would be small and loosely coupled. That is, rather than drag in a giant library, abstract out its code in a separate server and talk to it via some protocol. This worked remarkably well on the whole. So the idea of not wanting to have 100 statically linked programs all weighing in at many megabytes kind of misses the point.
For what it's worth, I said no to syntax highlighting three years ago and never looked back. Anecdotal, but I'm making much less errors and paying more attention to the code, now.
I think a lot of code highlighting is basically ineffectual, though, given that it (almost always) highlights based on keyword-based regex, rather than the semantic meaning of the code.
I use it as visual feedback from the parser. That is, if I’ve messed up a bracket placement while moving code around, all the code after that will be colored “oddly” and I can instantly tell both that I’ve made a mistake, and often where exactly that mistake is.
I contest that they highlights based on keyword-based regex too. I haven’t formally analyzed TextMate grammars, but I believe the scoping behaviors take them into context free grammar territory.
I don't think I would care one bit if my syntax highlighter stopped making keywords blue and type names green. But damn, would it get frustrating not having dumb quoting mistakes stand out right away. I like fixing those based on immediate visual feedback as I type, vs looking at a surprise compiler error next time I try to build.
Which of the below string expressions (C# as example language, many are similar) has a mistake?
1.
"Most strings are " + easyOrHard + " to read and " + concatenate + " with variables
2.
"Some, like \"" + onesWithQuotes + "\" are trickier"
3.
"Your friend \" + friend.Username + "\" has joined the chat"
4.
@"In C# multiline string literals denoted with @, \ is a legal character on its own and """" is used to escape the "" character"
5.
@"You're probably a jerk like my buddy """ + name + "" if you mix syntaxes like this"
"Anecdotal, but I'm making much less errors and paying more attention to the code, now."
Could be correlation rather than causation. Presumably, you're a more experienced programmer now than you were three years ago and you might be making fewer errors and paying more attention regardless of whether you turned off syntax highlighting.
Granted there isn't much to be gained by making your code look like an angry fruit salad. But I do find it useful in some cases, e.g. to make comments stand out from the rest of the code. Also, unterminated block comments or strings will immediately show up as well. And if you are not very familiar with a language, it might help to highlight keywords, so you don't accidentally use them as variable names.
It’s actually quite nice. Much less distracting. But I haven’t stick to it because it’s a nightmare on languages with block comments. It’s hard to know if you are reading a big piece of commented code.
That's why quite a few languages don't have block comments. But then it gets very tedious to comment out a section of code without a language aware editor. For me, having comments and strings highlighted is essential.
Seems like the Plan 9 guys haven't heard of these? If done properly, dynamic linking is almost always better than static -- key word here is "properly", which doesn't always/often happen.
What a load of FUD about updating dynamic libraries not actually fixing the code without rebuilding the dependent executables.
Symbol versioning does not break things in this way. If you've replaced your only libc.so with a fixed one, there is no other libc.so for dynamically linked executables to link with at runtime. If the new one brought a needed fix, the bug is fixed everywhere using libc.so.
It's not like the new library bundles old bits to fulfill old versions of symbols, that's madness.
The only way I can see someone getting into a situation implied by the article is when they install multiple versions of a shared library simultaneously, keeping vulnerable versions around, and have executables continuing to use the old copies. This is an administrative failure, and says nothing of dynamic linking's value.
The Plan 9 fans keep forgetting that the end of the road was Inferno not Plan 9, developed by the same devs, with all Limbo packages being dynamically loaded.
So, if you don't have dynamically linked libraries and you need a security patch in one of the libraries, how exactly is the system admin going to patch the system. I assume I would need to find every program that contains that library and recompile them?
The whole point of containers is that they are all-in-one and prepared each time from some build mechanism. Tracking down every binary on an OS is a whole different thing, thus the creation of containers in the first place.
site is down now... but I hope it will be honest and just explain that "they did it just so they could do that one demo of tar'ing one process from one machine, 9p'ing it to another box, untar it and the process graphical UI would resume on the new host as if nothing had happened"
1) "Shared libraries are bogus."
2) "Anyone who likes normal-looking user interfaces rather than plain boxes with text in them is a poopipants."
Both of these propositions are contentious to say the least, but what bothered me was that two propositions were mutually supporting while being (to my mind) orthogonal. The obvious and most compelling example of a shared library on the Unixen at the time were all the various UI libraries (Motif and various other abominations; all of them huge and unwieldy). It seemed necessary to accept that these libraries were obviously Completely Unnecessary to buy into the Plan 9 idea that shared libraries didn't do anything worth mentioning.
I'm sure it's possible to design a better UI library (or maybe even a wacky user level file system for user interfaces; in fact, my honours project in 1993!) but at the time the way people made interfaces that looked vaguely like what other people expected computer programs to look like was to use big-ass shared libraries on Linux.
This was also the way (dragging in one of those godawful blobs like Motif etc) that anyone might have exerted themselves to port across a (not completely pitiful) web browser, but the degree of aggressive disinterest in supporting Other Peoples Code was extreme (Howard Trickey's "ANSI POSIX Environment" or "APE" didn't get much love as far as I could tell).
It was quite undignified to watch people struggling with text-based web browsers or firing them up on non-P9 boxes because of the inability to support the web at the time.