Over the years, I've been leaning more and more this way. Designing, writing, developing software using the Unix Philosophy [0], the suckless philosophy [1], cat -v [2], Write Small and Fast Software [3], DJB and libdjb derivatives [4][5] as sources for inspiration, IMO, will lead me to writing more comprehensible, more performant, more secure software. I mention these inspirations when ever I can. And I think software is overdue for a reboot on how we design, write, develop software.
Another excellent resource is "A Philosophy of Software Design" [0], from John Ousterhout (known for Tcl/Tk, Raft, RAMCloud, Log-Structured File System). Like Niklaus Wirth, he is striving for designs that fight complexities and yield simple and powerful abstractions. This book is so much better than overrated books like "Code Complete" and "Clean Code" for a fraction of the page count.
I don't think more than a tiny fraction of programmers are capable of writing secure software under such an approach; unix's reliance on unstructured interfaces means misunderstandings and "confused deputy" style attacks are inevitable. IMO our industry fetishizes performance to an absurd degree in proportion to how bad we are at correctness and basic security. I do think we're due a reboot (and to a certain extent we already have one in the (typed) functional programming revolution), but one that focuses on writing software that does what was intended, rather than obsessing over shaving pennies-worth of cycles off program runtimes.
> IMO our industry fetishizes performance to an absurd degree in proportion to how bad we are at correctness and basic security.
I completely disagree. Our industry (or at least parts of it) have absolutely no concern for performance or power consumption (electron is one of the main culprits here, but the issue existed with Java/swing before,and im sure it existed before that too). For the most part, our industry is just as bad at security though, you're right.
I disagree about correctnesd though - correctnesd isnt a binary thing, it's a scale. Some things matter more for correctness (it's ok if we miss an animation transition, it's not ok if we have bad rounding in our financial software). Overall we have a decent grasp of the severity of issues and in prioritising what to ensure is correct.
I think we need an increase in software refunds and lawsuits due to security exploits in cloud and IoT deployments for companies to start paying attention.
I find the bloat of recent software comes mainly from the web development area which consists of far too many kids who want to make cat web sites cause they now know "computers and stuff". This trend has led those with small businesses to think anyone can slap together a web site in two weeks and "What does it matter if it's a 10MB download?".
This has led to people like me who are strong followers of such philosophies and methods who want more time to analyze the real needs of the business and the technical needs of a web site to get beat down by managers who claim they've never known anyone to take more than two weeks to do anything. (A recent true story.)
That's unfortunately the harsh reality of a business. They always want it done yesterday, never mind arguing the time needed for writing unit tests / E2E or performance & load testing as those don't give "business value". oh, don't even mention refactoring.
* can this button do action A? yes move on to B. etc... (don't even look at A anymore)
I have been following the development of Oberon-07 (and submitted minor fixes to Prof. Wirth) over the years. The compiler's source code is the fruit of refinements and simplifications accumulated over 30+ years (more if you include his compilers for previous languages). His approach for compiler writing, and system design in general, is outstanding. For more information about his compilers see "Compiler Construction - The Art of Niklaus Wirth" [0]
If you are interested in the Oberon-07 language, I made a port of Wirth's compiler for the JVM, see [0]. Several other implementations exist for different platforms [1].
Wirth has reportedly told the joke that, because Europeans pronounce his name properly ("VEER-t"), while Americans pronounce it as "nickel's worth", he is called by name in Europe and called by value in America. --- https://en.wikipedia.org/wiki/Niklaus_Wirth
It's not even possible for a single human to understand how a modern processor works in full detail, let alone how all the software running on top of it works. The same is true for almost all complex machinery (and not just because most things contain little computers nowadays).
You have to distinguish between inherent complexity that is necessary to solve the problem at hand and needless complexity that arises from not having the resources for building a simpler solution.
Uhh no.
Systems are so insanely complex that we have to use analogies to explain how they work to other people in our field. Its just not possible to truly understand from high to low level every single component and how they operate. That being said, I truly appreciate and agree with the premise that we should strive for brevity and simplicity _where possible_.
And yet everyone is voting this claim up from their Web browsers which do not come anywhere near this standard, on operating systems of which the same can be said.
If by toy languages you mean HTML then that's precisely why I dislike those things.
I want my browser to be served marked up text with the occasional embedded hypermedia, that's it.
I don't need fancy layout, I don't need transition effects and I don't need updated data without making another request to the server. If I wanted all those things I'd install an app like I do on my phone.
Of course, that spells doom for advertisers and those that rely on advertising dollars to generate content. But I'd rather pay for content if I think is worth paying for, like the Wall Street Journal for example.
I agree but what I has in mind is the horrible complexity of css, with 50 different ways to do somethings, making really hard to do simple things like middle align, etc.
Forgive me for being obtuse, but I can't see how the result would be any less complex or easier to understand than one application with reasonably modular design. You're just moving debugging into the interaction of services instead of the interaction of application components.
What I mean is that 'one person can understand the whole system' doesn't mean that everybody has to understand it in the same way. It can be understood using different abstractions.
A single system? Absolutely not. Even with a lifetime of study it might not be possible for an individual to really understand everything that goes into a modern web browser.
A single module with well-defined interfaces that abstract away its internals for its consumers? Yes, that metric is appropriate for a module like this.
I don't know if you're quoting this to show the absurdity of the statement or if you really believe it.
Nobody understands how a processor is made. Hell, I bet there's almost nobody that truly understands how a plastic cup is made. Do you know how complex the equipment for exploration and drilling oil is, refining, polymerizing, injection molding etc? Do you really think you can find a single man that's able to do this from scratch? And that's an easy case.
No system can be understood in it's entirety by anyone. This is because there's no absolute truth to be known, only approximations of varying degrees.
I am not sure if this makes sense. How would you build anything of some complexity? At some point you have to use components that are probably understood by someone else but I don't think a single person can understood the whole system.
How would you build a big rocket or an OS that way?
EDIT: A good principle would be that every software author should understand his own code in detail. This is often not the case.
Not sure. A lot of the world system from basic to very high level need coordination and modular ... you need some principle of operation which hopefully can be evolved over time so we are in the same page from time to time.
But no. You cannot ask for all to be understood like that.
Another attempt at minimalism in software was Alan Kay's http://www.vpri.org/ , which aimed to develop an OS and applications in less than 20k lines of code [1]. It operated from 2001 to the start of this year. It is also associated with YC's harc [2].
To be fair, he's countering articles that do focus on those exact things (and "Eight Megs and Constantly Swapping" suggests these attacks have a long pedigree).
To take his example of Word, I don't think it really affects users negatively that there are tons of features they don't use. If all you want to do is use it like Wordpad, nothing's stopping you, and many users do just that. But if you want to start automating features with VBA, doing extremely complex mail merges, building a bibliography, or all sorts of other great features, they're there for you. And any reasonably-designed software should be designed so that you don't have to know the whole thing inside and out to work on one feature.
When I think of software I use often, quite of a lot of it is very complex, fully-featured software, and I'm sure there are many features I never use (e.g., IntelliJ, Chrome, Office, JDownloader). In this respect I really don't believe in the "Unix philosophy," at least for GUI programs.
About Eight Megas And Constantly Swapping, I would note that emacs 24 (X11) is 14 MB on a 64-bit linux, 46 MB if you count the associated files (/usr/share/emacs), while the web browser that started at a few MB on NeXT, is now at 163 MB on same 64-bit linux (/usr/lib/firefox).
So on one hand, a full-featured extensible program whose size barely changes over 35 years, and one the other hand, a single purpose application, somewhat extensible, whose size grows exponentially over 25 years (and it's very much NOT limited to the 163 MB on disk space, since if you want to show or run anything in a browser window, you have basically to download it from the Internet, and since the least web page is 300 KB download size...).
At one point, we have to address the question of the programming language. C++ with its templates, and its baroque syntax and typing rules, multiplies the the size of the sources and the object files. Even the lighter C, when used with modern programming techniques (the older ways would lead to too many bugs and security holes), involves a lot of (by-hand!) code duplication (even with _Generic in C11). On the other hand, emacs written (mostly) in lisp uses a programming language that is basically generic. Since types are not checked at compilation-time, but at run-time, any function is essentially a generic function by default, and this leads to a lot of reusability and avoids a lot of code duplication. In addition, the use of a garbage collector (and in some cases, run-time checks for immutable data) means that you can avoid a lot of data duplication; this is not possible in C or C++, therefore you can witness a lot of data copying and duplication in those programs, to ensure manually memory and "const" safety.
"Eight megs" was supposed to be a hyperbolic claim about how much RAM it used, not the size of the application on disk (hence the "constantly swapping" part -- i.e., it's constantly going to your HD's swap partition because it wants to use so much RAM).
... or rather, doesn't take up, resulting in software with tons of broken behaviour (including vulnerabilities) because the developers don't really understand what they are working with sufficiently.
>So I’m starting to suspect that fretting about bloatware is more of a mental health problem than a software problem.
I wouldn't call this style of writing 'pretty good'. Asserting that anyone who worries about something he doesn't worry about has a mental health problem is pretty arrogant.
His point is clearly that they are devoting an extraordinary amount of energy worrying about a paltry amount of resources and thus to the extent there is a problem it is largely a figment of their imaginations.
It's a pursuasive and fun article and feels convincing, because that's just how Joel writes. But it's kind of funny to imagine the same arguments in any field but software. "We could have optimized and improved the car's design, but you would have had to wait two more months and that cost would have been perceptible. Anyway gas has gotten cheaper, so in 'real' terms the car is 'more' efficient! Fretting about elegant car engineering seems like more of a mental health problem." Not saying Joel is necessarily wrong, but it would be nice if software design had some sense of craft....
Perhaps, but the engineering of physical goods is a poor analogy to software, considering the relative ease of changing things in software if you decide they aren't working.
(Also to bdhess) just a note that I'm not trying to say the right thing for a car is also the right thing for software, just that the contrast between the design philosophies is funny (haha and/or peculiar).
To be fair, you did argue that it would be nice if software engineers showed "craft," which suggests that you think the way software is done is the wrong way.
I don't think there actually is a contrast in philosophies. The philosophies are the same for each: do a cost/benefit analysis, and then go with whatever makes the most money.
It just happens that the cost of fixing an automotive defect (i.e., with a recall) is generally way higher than the cost of fixing a software defect (i.e., by shipping an update). So the automotive industry invests more heavily in eliminating defects.
Certainly the philosophy when you're developing software for medical devices or rockets is more rigorous and careful, but the speed would be pretty much unacceptable for, say, Facebook.
Yes. And there's the inherent problem in arguing with analogies.
Here's another analogy:
"This car seats twice as many people, and accelerates twice as fast, but sadly will require twice as much motor oil when it's time for an oil change. We could have optimized and improved the car's design, but you would have had to wait two more months and it would've added $10,000 to the unit cost. It would've been madness to do."
We could sit here all day arguing about which analogy captures the actual situation correctly, or we could just talk about the actual situation.
> "We could have optimized and improved the car's design, but you would have had to wait two more months and that cost would have been perceptible. Anyway gas has gotten cheaper, so in 'real' terms the car is 'more' efficient!
If gas did get cheaper as fast as memory and storage, that would be a correct argument.
> Not saying Joel is necessarily wrong, but it would be nice if software design had some sense of craft....
I'd actually say the opposite: too much sense of craft is the problem with a lot of software development these days.
Interesting analogy, given that this attitude is pretty reflective of the industry as a whole. US manufacturers lobby against fuel efficiency regulations, and stonewall any newcomers to the industry. European manufacturers fraudulently subvert efficiency requirements to make the product seem more efficient than it really is. Presumably some of those resources could have been devoted to actual engineering...
I find an interesting contrast between Niklaus Wirth and Bjarne Stroustrup.
Wirth made a bunch of languages that were based on the "right" way to do things: Pascal, Modula, Modula-2, Oberon. The languages either were used and became massively fragmented (see Pascal) or were not used much. Other than Borland (Turbo Pascal and Delphi) no other large company really embraced any of the Wirth languages.
Stroustrup, on the other hand, was more pragmatic, and released a language aimed at professional programmers. In addition, instead of creating new languages, he has spent decades evolving C++. As a result, C++ has been in the top 5 of most used programming languages for years (decades?) and forms the critical infrastructure of many of the largest tech companies. In addition, I bet the evolution between C with Classes, and C++17 is pretty comparable in scope to that between Pascal and Oberon.
Apple embraced it, and used Pascal to write the LISA OS and the Mac ToolBox (not the MacOS "kernel" which was written in assembler for speed and memory optimization). Until they switched to OpenStep and Objective-C.
PalmOS was also basically a simplified clone of MacOS; I don't know if it was itself written in Pascal, but it kept the same internal Pascal data structures IIRC.
That's a very cheap way to dismiss C++. I think it's more that Stroustroup has the ability to develop a decent language, promote it, make it usable for real-world use and improve it long term. I would say Linus is in the same category. Despite all his antics he knows how to push a project forward while still keeping his old users. Both have a very good combination of vision and pragmatism.
While I agree with you, and enjoy following Bjarne, the rise of VMs and scripting languages helped keeping C++ around.
Had Java and .NET do the proper thing, by following up on Oberon, Modula-3 and Eiffel, by having proper AOT compilation since day one, and on Java's case value types as well, developers would feel less pressured to turn to C and C++.
Basically if they had been released the same way Go was.
> I have to ask - does anyone actually use half of the C++ features? Exactly how much of C++'s 1600-page specification can be attributed to its success?
do you also use all the classes of Python or Java's standard library ? C++17's spec is ~1400 pages (http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n471...) but the actual language spec stops at page 406, the remaining 1000 pages are the specifications for the standard library.
And in these 400 pages of specs, there are a lot of examples, grammars... just look at the chapter on templates: it's ~80 pages but more than half are valid & invalid code examples.
This was written in 1995, just after the failure of the $3.7 billion attempt to modernize software at the FAA. That was the most expensive failed software effort ever. The amount of bloat on that project was astonishing. It’s worth reading about that failure to get some historical context against which this author was writing:
But this continue still todays, with hundreds-of-million dollars or euros projects being thrown away (or that should be thrown away, given their poor state).
Small Anecdote: A great quote Prof. Wirth made during a talk at the IBM European Networking Centre in Heidelberg somewhen in the late 90s, which I found spot-on already then and very memorable: "Software is getter faster slower than hardware is getting faster." The audience was cheering.
I have to commend Google here for bucking the trend. To give two examples:
1. They recently rolled out an update to GMail that they had worked on for some years. Were these years spent figuring out how to make it fill available memory and processing? No. They were spent figuring out how to streamline.
Today, if you visit mail.google.com, the page loads in about the same time as a ping: that is because they got the login page down to just 56 bytes after compression. That is the size of a ping packet. This is an astounding achievement.
After logging in, 56 bytes isn't enough to present much of a view, but it does the next best thing: in the space of the absolute limit for a single TPC/IP packet, which is 64KB, the interface is rendered enough to see the latest unread emails, and to make the Compose button active. After a single packet! In my tests this took less than 7 milliseconds. This is enough to render both of the main things one would like to do: compose a new mail, or view the latest mail. In other words, after a single packet is received, a fully functional inbox is presented, and a fully functional Compose link. The TCP/IP packets that follow then load some javascript, but you wouldn't know it from the interface, which stays the same as it loads. I've tried manually throttling the interface after the single packet, when it was bare HTML, and could still click to request a view of mail or compose new mail, getting the result instantly. In all, it is possible to enter and exit gmail while consuming a grand total of 2 KB. To put this into perspective, this represents approximately 0.016% of 1 second of a 100 megabit connection. Wow.
2. The other example I have from Google is Google Chrome. In an era of heavyweight desktop apps like Firefox, Adobe Creative Suite, or text editors like Atom (which uses up to 3 GB just to open a text file)[1], Chrome does the insanely impossible: it uses less than 2 megabytes of memory to fully open, and less than 1 kilobyte of memory on each open tab. Since Chrome has a 32-bit version, I tried running it on an old Pentium I that I had running windows 95. It is a 300 Megahertz computer. While not fast, it was able to open a hundred tabs in about 15 seconds. That is 0.15 seconds for a new tab on a Pentium I. On a modern quad core computer, a new tab is ready by the next screen refresh. And it scales: opening ten thousand tabs takes less than 4.5 seconds and ended up eating 10 megabytes. Wow.
To think about the low-level engineering that must have gone into this is just astounding. Firefox chokes on a few dozen tabs. Windows can't even render a window that fast - they must have literally written more efficient low-level code than the built-in window manager for Windows.
In a world where software struggles to somehow use up all precious 16 gigabytes of RAM that you have, and eat up every cycle like it's some buffet, it's nice to know that there is some serious capability put behind behind small and lean.
EDIT: three people read what I wrote literally (even though I linked an archive.org link). I was kidding. Unfortunately, I don't have access to a magic GMail. I have to wait 15-45 seconds for the new GMail to load. It is literally the slowest site I use on the entire Internet. The above represents my expression of what I know Google engineers would be capable of. They're smart enough. I believe in them. They can do it!
I suddenly got a new Gmail version since a couple of weeks, it is an order of magnitude slower than what I've used for a decade and loads with a pointless animation that, by itself, takes seconds before I can even begin to see my mail. How does one activate this magical Gmail experience you have? If there's any way.
If page load is that fast according to your description, how come can I still have the time to click on "Load basic html". By the way, that one is truly lightning fast.
It's one thing if they bloat their mail app beyond recognition. It's another thing entirely if they brittle Web standards (through whatwg funding and Chrome de facto behaviour) ad absurdum.
The Web used to be such a simple and elegant platform (you might disagree) that was immediately useful and transparently working for what people wanted to use it for, including simple static or almost-static web mail apps. Why was/is it necessary to make it so fscking complicated that the world's digital heritage is at stake, and information is monopolized in very few hands whose business model is spying on their users? With "hackers" standing by and cheering?
What? The new Gmail has so far been a nightmare. Dumps me to a loading screen and eventually loads my inbox. Once it loads, don't rush too quickly to a message. Because it isn't ready. :(
Come on, you can't serve 1.2 billion users without having the most talented people. It's just not possible. They don't need to bring people back from different divisions, I'm sure that at a division level GMail has the talent within it today.
Reality is not some immutable force. We all take a role in shaping it.
In my current role I work on a very large build chain for a very large software product. I have pushed and chipped away at that build chain over the last three years, and at current measure a clean build now takes an hour and a half, where it used to take eight hours.
Why do I care so much about things being lean and small? There are a plethora of tangible benefits, but... maybe eight years ago or so I stumbled across Niklaus Wirth's (author of this article) works, and they struck a chord with me. But I had questions. So... I emailed him.
I'm nobody special. I wasn't expecting a reply. But he wrote me back a wonderful reply to my questions, and that, too, has stuck with me over the years.
So, I'd like you to keep in mind that that 'plea from a purist removed from reality' had a strong impact in at least the reality for one large software product.
> How else would you explain the rise of HTML/JS/Electron and the fall of C++/Qt in the UI world?
Does this "fall" really exist though, or is it just the byproduct of extremely intense marketing from the HTML/JS/Electron software authors ? I'm pretty confident that there are more desktop software being written today in Qt than in Electron - but most of the time they don't end up plastered all over the interwebs for the simplest todo-list app.
Are mediocre C++ devs that much more expensive and rare than mediocre web-devs? Certainly in the field I work in (civil engineering) you're probably far more likely to run across someone who can cobble together a simple GUI tool in C++ than you are someone who could cobble together the same thing in Electron.
My bet is on there being less friction at the onset. How quickly can you get a cross-platform prototype up as a web app vs Qt?
I'd also wager there are more C++ pitfalls than JS pitfalls (or at least the JS ones are less severe/obscure). A less experienced engineer would be able to better maintain the web app than the C++ one IMO. This increases the bus factor.
The people who make the decisions care more about those things than the layers and layers of expensive abstractions that prop up web technologies, and how much electricity/battery life that consumes in aggregate.
I'm working in game dev. Most of our tools are native UI or in-engine. The heaviness of web apps frustrates me a great deal, but I get why things exist the way they do.
[0] https://en.wikipedia.org/wiki/Unix_philosophy#Doug_McIlroy_o...
[1] http://suckless.org/philosophy/
[2] http://harmful.cat-v.org/software/
[3] https://www.fefe.de/dietlibc/diet.pdf
[4] https://www.fefe.de/djb/
[5] https://www.fefe.de/libowfat/