For people involved with technology I think this is a perfect response to "how hard can it be" - no matter if they're discussing how to keep toilet paper stocked or why cURL can be replaced with five lines of shell script - it emphatically shows how complicated something apparently simple can be once you start diving into it.
More often than not a problem that looks simple and has insane complexity once you dig a bit deeper is based on some faulty assumptions. Assumptions that either changed over time or never made sense to begin with. The wast majority of software systems out there are significantly simplifiable. However, not all developers are willing to analyze and question fundamental assumptions about systems they deal with. Most, in fact, are far more comfortable learning to jump through arbitrary hoops. They then brag about their "expertise" in the "domain" (which often doesn't even exist outside of software).
> The vast majority of software systems out there are significantly simplifiable.
I think exactly the opposite, actually.
Most programmers don't just write lines of code for no reason. They generally stop when the problem is solved.
So, you might be able to simplify new programs, but I wouldn't put a lot of money on it. The programmer would have had to miss something architecturally up front for that to be the case. It happens, but it's not that common.
So, you might be able to simplify production programs. Maybe, but those have a lot of bug fixes already implemented. And they encode the political state of the entity that built it (see: microservices). So, a program that has existed a while may not encode the current political system, but I don't think fixing that is making things simpler.
So, you might be able to simplify legacy programs. Perhaps, but those have a LOT of business state encoded in them--much of which is still used occasionally but you won't know that until you do a deep dig on the code. That rule for disbursements to Civil War descendants is there because the system still needed it up until a couple years ago.
Oddly, the best way I have found to simplify computer systems is to give the humans more power. Humans are good at exceptions; computers not so much. This is, sadly, anathema to modern programming and modern political/corporate structures.
>The programmer would have had to miss something architecturally up front for that to be the case. It happens, but it's not that common.
It is very, very common indeed. Seldom is software created with all the requirements and complexities of the problem known upfront. In fact, very often the process of development of the software itself reveals all the different ways in which the original specifications were imprecise and not completely thought through. And new features and requirements get added onto later all the time.
Software is created in an iterative process of feedback and development. At each stage, the programmers takes the shortest, most straightforward approach to solve the problem. When the programmers are not careful (or just lazy) they inadvertently add dependencies that shouldn't exist and make the whole thing more complex. Of course, any single infraction seems innocent enough, but eventually you almost always end up with software that is much more complex than it needs to be.
Here is a talk on how software is too complex all the way from 2006. I am more than confident that the problem has only gotten worse in last 15 years.
> At each stage, the programmers takes the shortest, most straightforward approach to solve the problem.
Unfortunately, this is the often the best case. There’s a point in the life of most good programmers where they can’t help but to massively over engineering everything they make, adding pointless interfaces and abstractions everywhere, to chase the dream of reusability. This mindset will utterly bury your capacity to iterate.
I once saw a Java method trail to initialise something which was 19 levels deep - each level just making one method call to the next abstraction. You couldn’t just trace it in the IDE - lots of those calls called an interface method or something, so you had to hunt down the implementer. But it got worse. There was also a method trail alongside it for cleaning up that context. It had the same 19 levels deep trail, but after all that work the final callee was an empty function.
> Most programmers don't just write lines of code for no reason. They generally stop when the problem is solved.
If only. Over-engineering is definitely a thing. And programmers definitely miss something more often than not, even when over-engineering. Or even: especially when over-engineering.
You must have above average programming peers. It's my experience that architectural decisions that could cut future work by orders of magnitude are quite commonly missed.
Alan Perlis' old motivational-poster slogan "Fools ignore complexity. Pragmatists suffer it. Some can avoid it. Geniuses remove it." has a fair bit going for it.
I love that apart from the "Some can avoid it" bit — feels out of place with the rest. Who's "some"? Pithy slogans often come in three parts rather than four; I love the overall message, I think I'd just prefer it with that 25% removed.
I think this is a funny thing on several levels - one you see the part that doesn't fit so you remove it making you the genius - in this way it's a nice motivational poster level thought.
But you could be a some. But if you were a some it would mean that there was no possibility of actually removing the some from the statement because it actually did describe accurately that there were somes that could avoid it.
No, it doesn't, and no, it isn't. As much as some people like to pretend it doesn't exist, accidental complexity is a thing, and reducing it is part of our job. Also, there are ways of removing complexity of processes before they are turned into software, but that requires talking with people and agreeing. Not every program has to do everything.
>As much as some people like to pretend it doesn't exist, accidental complexity is a thing
Yes, and it's often the best feature of the software, as far as the sale teams is concerned, because it allows them to sell concultancy fees or because it satisfies the customer by matching 100% it's crappy requirements (which they don't really need).
Of course it is, in fact it is pretty much all-pervasive in our field, even in very successful software companies - which makes me question whether removing it is as valuable as people make it out to be.
Yes, and I think you provided a lot of the answer.
You can spend 5 years customizing off the shelf ERP software to do insane things because John in accounting and Jane in HR won't budge from weird processes they built over the decades... Or you can use that opportunity for business transformation, simplify their processes and their program, save money, reduce errors, improve responsiveness and timeliness, and generally make it a win win win.
Not all complexity can be reduced of course. I can imagine self driving cars, rocket science, etc are all about critical edge cases.
But when it comes to business, and particularly back office processes (where a huge amount of IT goes) , a lot of complexity is unnecessary. Each company each department each person is certain they are special, unique, and their requirements are paramount and unchangeable. But if you move around a bit you'll come to believe otherwise.
Of course it's is valuable. Complexity makes software expensive to make, so removing it is literally worth a lot of money that can be better spent making features. Nobody buys software because it's accidentally complex, they buy it because of features, quality (which is something that's also affected by complexity!), etc.
The only reason complexity is prevalent is due to a shortage and geniuses (and also a shortage of "Some" too, if we're going by Alan Perlis' quote).
Most things in life and programming exist on a trade off curve of goodness vs simplicity. Making the thing better means giving up simplicity. Making it simple means removing features people want.
Geniuses invent new ways to get both. Those ideas permanently move the trade off curve outwards.
In programming, some abstractions which have done this are: Operating systems (Abstracting away the hardware), Grace Hopper’s invention of program libraries, high level languages and compilers, HTTP and JSON, tcp/ip (replacing custom transmission protocols), and there’s lots more. Calculus and the Arabic numeral system are examples in mathematics. (It’s insanely difficult to multiply using Roman numerals!)
I once read a book called "simplicity" by de bono I think it was...
He made the point (I think) that you do want simplicity, you don't want simplistic. That is: it should be able to do all the things you want to, but in an easy to use manner.
To me, simplicity means that there's clarity of purpose for each component and how it interfaces with other components. A large system can still be simple if the interactions and interfaces between the different components are reasonably well understood and justifiable. I think thoughtful UI design is how you prevent complexity. The UI for developers and software is the API.
When you start adding features and instead of generalizing you simply (heh) add a special case to allow two unrelated components to communicate, and then another, and another, until the two components become dependent on each other such that neither can do anything without the other; then you have a complex system, and the "UI" for the developer is worse because it's full of seemingly random elements in random places that only do one very specific thing.
>Operating systems (Abstracting away the hardware), Grace Hopper’s invention of program libraries, high level languages and compilers, HTTP and JSON, tcp/ip (replacing custom transmission protocols)
How many of us get to work at such lofty problems though?
>Calculus and the Arabic numeral system are examples in mathematics.
Eye roll. Of course reducing complexity is the very essence of mathematics. Let us not pretend that software engineers are mathematicians.
> How many of us get to work at such lofty problems though?
How many of us choose to work on such lofty problems? You can work on whatever you want, whenever you want. But lofty, unproven ideas don’t pay Google salaries. Your career is a choice, not a prison.
And of course most software engineers aren’t mathematicians. We’re talking about genius - and most of us are a long way from that. Most of us are lucky if we manage to invent a couple complexity reducing concepts in our entire careers.
But lots of great computer science ideas have “come down the mountain” from mathematics adjacent work. Functional programming wasn’t invented by C lovers like me. But I love closures, pure functions, and map / filter / reduce. This stuff makes certain problems lovely. And we wouldn’t have any of this without mathematically minded CS.
What? IME removing complexity removes costs. That's a good choice. I can't think of a case where it didn't happen. Of course this only works for the removable complexity, which is discovered through analysis (possibly iterative/recursive)
Time spent removing complexity is time not spent developing features. And your users/clients care more about features than complexity (which is completely invisible to them) or even speed. They don't even care if your app runs 10 times slower than it needs to, as long as it runs at an acceptable speed.
That is why software engineers are seldom rewarded at their companies for "removing complexity", only for finishing projects.
You're right that, given a set amount of programming time, working on features versus working on performance is zero-sum.
But that doesn't mean you should always work on features. If anything, it only means that feature-bloat is doubly bad, not only adding logic and binary size, but imposing opportunity cost.
Most users hate features. Nobody has ever sworn at their computer because they had to make a mess in Excel to use VLOOKUP() before XLOOKUP() was invented (even though I love XLOOKUP()). Plenty of swearing happens because the computer just won't respond or it's taking whole minutes to send each email when you're trying to leave work on Friday night.
> Plenty of swearing happens because the computer just won't respond or it's taking whole minutes
Of course, my point was that it is usually not beneficial to worry about performance until that happens. And when it does, improving the performance of the product becomes a project in its own right. Misbehaving parts of the software are diagnosed and fixed, usually by introducing even more complexity.
Sounds like our companies and clients are very different. As for the feature/maintenance trade-off, a reasonable CBA will show the answer. I don't think it's quite as absolute as you've described.
The number 1 assumption is that all text will meet the format you're expecting it in. Be that an encoding (everyone in the US speaks English, but utf-8 is dangerously compatible with ASCII), or a character range (do we block input of certain characters?)
Presentatuon is huge too, How do we render a pasted newline into a single line block? Where do you put the cursor if someone pastes a block of Arabic text into your multiline input? What do you display if your font doesn't have the character they've copied and pasted from another source?
There's also just the basic stuff of "every keypress/chord equals a new character" - I have an app that I use every day that renders Ctrl + backspace rather than deleting the word?
Then there's input considerations; Macos uses alt for per-word navigation, windows uses Ctrl. Do you support the OS input type, or do you support a popular editor bindings (emacs) and how do you differentiate between an input to display and an input to take a navigation from? What about mobile? Most boards support input gestures, and autocomplete suggestions. How do you know to modify your current context over appending it to your input?
Finally, what about non-renderable input? If you're modifying a rich text string, how do you escape from a block, or how do insert between two separate blocks?
This. My name is Kayodé Lycaon. Note the é... how many places don’t support it? Some people consider names to be sacred and changing the spelling is more than a little offensive. Even UTF-8 can’t represent all characters in use. I believe there are Japanese names that can’t be represented by its character set.
I get it’s technically difficult but so many places treat people who are different as edge cases to be optimized away.
UTF-8 can represent any Unicode code point. If there is a character that it can’t represent, then that’s because that character is not encoded in Unicode; it has nothing to do with UTF-8 itself.
Perhaps the assumption by Unicode that we need emoji modifiers instead of just more emojis. All ~2000 valid combinations are enumerated anyway and many require different images so they're effectively different isolated characters.
That text is a sequence that we edit by having a cursor into it as opposed to a specialized form of graphics, maybe? We've inherited the controls of a typewriter (which provided a very simple graphical model for a narrow range of languages), but maybe they aren't what we need?
For someone not overly ear to the ground culturally this is a rabbit hole. The Youtube one fascinates me. I've not heard of most but imagine there are hundreds of interesting stories behind every item. Interesting work and concept. Well done.
I clicked on directors to discover that I've seen at least one film by almost all the directors in all the tiers (my omissions being Denis Villeneuve, Nicolas Winding Refn, Gaspar Noé, and Tom Six.
Over the course of my life and in my thirties especially, I've watched a lot of movies.
Wasn't there a post or (most probably) a comment recently (within a month or so) with an iceberg chart? It looked very much like this site, but I can't find the right subject. Someone even commented that one of the best links in the chart was to a fictional story on the site itself. For the life of me I can't recall what the subject of the iceberg was, except that I think it was technology related, but I did spend several hours digging through all the links.
See! Irony at work: it wasn't good enough for you so you wanted a feature. That's where it all starts. I'm not dissing you, I'm pointing out how innocuous feature creep can be.
It most commonly afflicts organizations with management that insists upon KISS interactions from their direct reports, but then micromanages the details when the going gets tough (scrutiny from higher layers). Some of the more common phrases you will hear in these interactions is, "I don't understand why this has to be so complex", or "Give me the 50 thousand meter/foot summary". They elevate their mental model of the domain to a level where they cannot resolve the granular details.
Analogous to the lack of optical resolution of any telescopes (including Hubble) we have to see the American flag on the moon [1]. You need a very massive bandwidth (the telescope diameter) to resolve those details, but then you trade off for increased processing time (analogy breaks down here).
Surprised Pikachu when they drill down, and the requirements explode in complexity. With most successful, highly-refined systems, it isn't the primary requirement that should be the focus of planning (Microsoft Word is just a text editor, how hard can it be!?). It is all of the 1% of features 1% of the market/stakeholders want that deliver 80% of the value proposition of the system to them. Stacked together, all those "corner cases" add up the value of the overall system, and add up the complexity. It isn't just a heap of features binned together; the team has to make sense of how it all fits together.
In college we had a professor in software engineering class who had this wonderful saying: "no matter how complex a problem looks, if you look at it just right, you will see that it is in fact even more complex."
It took years of working as a software engineer to begin to appreciate just how deep that is.
I think there's some context that matters. I often don't need the full functionality of übertool and can implement the functionality easily because I don't have to worry about 90% of use cases with their associated edge cases. It's dead simple to keep toilet paper stocked in my house. But I would not use the method I use for my house to run inventory for a grocery store chain.
That's exactly the point. The "how hard can this be" comments are often applying an individual use case, where corner cases can be ignored or worked around, to an enterprise situation, where corner cases may hit some legal constraints (e.g. accessibility) or may no longer be corner cases once in enterprise-scale (e.g. multiple languages / currencies / time zones).
It depends how big you think. Dealing with status quo, things like text direction are necessary complication.
But at a higher scale it’s purely incidental complexity: you only need one direction for all languages. All languages are linear. No language uses direction to encode meaning. Direction in language is just “stuff starts one place and ends in another”.
Balkanization of the encodings is not a good thing. Imagine if every city had a different color for a traffic stop light. Why not? Maybe it’s a matter of cultural pride. What if every country had its own network protocols? Imagine what a nightmare internet would be.
But thankfully we had common sense in few cases. In many others we don’t.
And Unicode is full of incidental complexity itself. Did we really need skin color emoji. Who is yellow here? Anyone being brightly yellow perchance? No. We had the perfect abstract color for emojis.
It's not the last time major portions of the world fundamentally changed how they did things to align with the major powers at the time.
Even WITH all the Unicode support that currently exists in the computing world I don't know of a single computer language that isn't based on English in ASCII. I guess you could count APL there (or perl!).
Imagine if Europe was pre-metric system today. Merely proposing unification I'd be instantly downvoted, and called a bigotted racist who doesn't respect the cultural heritage of every nation.
Unfortunately those are the times right now. We, the most adaptable and intelligent species on the planet, have become used to bragging about not having to adapt or know a lot. We sure do have really strong opinions about everything, though.
I think that's the dividing line - each person decides if it's worth it to learn something new. For example, some people love foreign media so much that they trade fansubs of said media or even learn the language so they can watch.
Someone who wants to program computers will most likely learn functional English, just because of how much content on that subject is in English (the "lingua franca" of computers).
But someone in Mongolia who wants to text his mom has no inclination to learn a whole new language just to handle the limitations of ASCII. And if you want to sell him a phone you'll meet him where he is.
(This doesn't even begin to touch the "it's not fair" aspect that people like to point at.)
You PoV is understandable, but still do you believe the ongoing long-term pain of this fragmentation is LESS than the short-term pain of unifying?
This is why I mentioned the metric system, and what not. Every time throughout history when we decided to unify, the advantages VASTLY, and I mean VASTLY have outweighed the short-term drawbacks.
In fact I'd say, one of the biggest test for us as species is that we can fully understand, internalize and implement this concept. Because we always reach for the short term. Short term profits over long-term strategy in the enterprise. Short term economy benefits over long-term climate impact. And so on and so on.
Someone in Mongolia won't have to switch overnight. Someone in Mongolia may gain the option to have it both ways until they're used to it, and eventually there will be unification. That's basic change management.
Change management is a necessary art to survive and to thrive.
Languages aren’t paradigms of thinking. We borrow phrases and concept from each other at a higher level, they’re basically identical, unless you’re isolated on an island and don’t interact with anyone.
For proof, look up the etymology of a random set of words. They come from all over the world.
This is literally pride over encodings. It’s superficial, petty, and ignorance makes it seem profound and important.
You should study linguistics. Your're flat-out incorrect.
I speak two languages fluently and they really are two different modes of thinking. That's why translation is an art, not a science, and for literary stuff it's quite difficult.
I think if you click a link which points to a very detailed list of programming languages based on non-English languages, then decide to take an example from a different list and suggest that’s what OP was “reaching for”; then you’ve probably failed.
Can you honestly say you know of AxumLight / Geez#, Al-Khawarizm, Jeem or anything at all that's based on another human language? These are mostly obscure.
Sure, but not all people who use computers are programmers, and programmers make up a small minority of computer users. Much of the value in computers come from augmenting other workloads. There are no shortage of people composing e-mails and documents, or consuming content, in non-English languages. Probably more than there are fluent English users.
> ?srekaeps hsilgne-non lla rof sdrawkcab txet lla ekam ot si noitulos ruoy oS
So you just said all non-English speakers use right-to-left? No, the direction isn't what's confusing me, rather the fact you're trying to score a cheap outrage comment, but actually wrote non-sense no matter which direction you read it.
How about maybe educate yourself a little, and rephrase your question to reflect reality, and we can have an intelligent conversation about it.
Most of the world uses left-to-right and top-down (yes look around when you go out and notice all the vertical logotypes in English around the city). Right-to-left is relative minority. If you have democratic thinking and we gotta vote for it, the winner is obvious. Yet, I'd rather learn to read fluently English in right-to-left than the current mish-mash of "whatever".
Also read about Korean. It was a top-down right-left language, today it's written left-right. People are surprisingly adaptive, you know, it's supposedly our top advantage as a species. No one is born with "right-to-left" grafted in their DNA. We honestly gotta tone down the ignorance in these discussions.
My native language isn't English, by the way. And I don't mind adapting.
Okay, well sorry I was being snarky, so let me try to be a bit more serious.
You are proposing that we teach millions-billions of people a new way to write, rather than teach a few hundred thousand at most how to write computer programs that accommodate existing ways of writing.
Like maybe you underestimate how genuinely ridiculously hard of a problem it is to get billions of people to do anything! It would take literal decades of work by multiple governments and unimaginable amounts of money! Getting 10 people to agree on a favorite programming language is hard enough. Teaching a city full of kids a new way to multiply numbers takes 1000s of hours of prep, planning, syllabus development etc. by teachers. Just imagine how much harder this is.
Certainly I can see how nice it would be if the whole world used the same date format, same language, same metric system etc.
But from a practical point of view, making systems to accommodate the way people do things is 1000s of times cheaper and easier than getting them all to change.
I'm not who you're responding to, and obviously the commenter should have said "some" non-English speakers rather than "all".
But the main point is still obvious and true: how would you like it if another country was a dominant technological power and we all had to re-learn to read and write English backwards precisely like that to use computers?
No need to get snarky about it. Comments like "maybe educate yourself a little" are against HN guidelines. [1]
Also, Korean is different because the characters represent entire syllables, so legibility isn't impaired as much by direction. CJK scripts have historically been flexible in direction in a way that Latin scripts never have been.
> how would you like it if another country was a dominant technological power and we all had to re-learn to read and write English backwards precisely like that to use computers?
I'd actually like it.
You know, thanks to this we have Murican Internet that's the same stack everywhere. How would YOU like it if "Internet" was just your country?
We gotta disentangle this obsession with encodings as some kind of national cause for pride. Encodings are INCIDENTAL. Encodings are LEARNED. You weren't born reading right-to-left or left-to-right and you can switch both in surprisingly short amount of time, if you bothered.
Pride doesn't even matter here. It's simple economics.
It's far less effort and cost for developers to accomodate the world's existing scripts, than it is for the vast majority of the world's population to re-learn how to read and write a new script.
By orders of magnitude.
And obviously, it's not just direction that differs from English but a whole host of other aspects as well.
I see it the other way. It would be actually orders of magnitude less effort for the rest of the world to learn one standard system/language than to have every app/product/document adapted to every single language/script/whatever in existence
The percentage of the world population involved in script adaptation and language translation is really quite tiny. Consumers vastly, vastly, vastly outnumber producers.
Not to mention that we have things like automated translation that are "good enough" for many cases as well.
Whereas learning a new language to fluency, for most adults, takes about ten years.
>the rest of the world to learn one standard system/language than to have every app/product/document adapted to every single language/script/whatever in existence
But they don't. That's precisely why the text input handling is so difficult, because it has to solve it for all apps. How many apps have you worked on that has had to create their own text input system from scratch? I think for most developers that's zero.
Like the article said in the end:
>The necessary complexity here is immense, and this post only scratches the very surface of it. If anything, it's a miracle of the simplicity of modern programming that we're able to just slap down a <textarea> on a web page and instantly provide a text input for every internet user around the globe.
There is an issue with that just like with programming languages. Do we want to standardize on one programming language for all use cases? Because they have different expressiveness and core concepts, not all programming languages are the best fit for all situations (consider assembly versus TensorFlow). Similarly, different human languages have different core concepts, and express them differently (for example, English has a stronger emphasis on time branches, with all the different tenses, while Korean has a stronger emphasis on relations, with all the very specific words for different members of the family and politeness levels). If they are replaced with one language, that expressiveness would be lost.
> It's far less effort and cost for developers to accomodate the world's existing scripts, than it is for the vast majority of the world's population to re-learn how to read and write a new script.
That's the short-term analysis. But this is not a short-term impact change. And in the long-term, things turn around.
If as a species, we can't think long-term, we're basically doomed.
> But at a higher scale it’s purely incidental complexity: you only need one direction for all languages. All languages are linear. No language uses direction to encode meaning. Direction in language is just “stuff starts one place and ends in another”.
I think an interesting example here is Chinese which is traditionally written top-down-right-left but these days is just as if not more commonly written left-right-top-down. There seems to be almost no push for supporting vertical text while its just assumed that RTL is a must, why is that?
I'm not a native English speaker and have always used computers in English. Early computers simply didn't come with language options and still people were able to use them. In fact I find the localized Windows versions odd and contrived.
Not even just non English speakers, we in en-gb land had enough trouble getting £ to work 100% of the time, which was kind of a basic requirement for business use.
Extended ASCII was actually several different and incompatible encoding, "codepages". It was kind of a solution, but there were major issues that made it much worse than Unicode/utf8:
- Asian languages needed special encodings, 256 characters weren't sufficient.
- It was impossible to work with different languages in the same document, each codepage only covered some character/alphabet.
- There was no 'safe subset': Unicode codepoints 0-127 are exactly ASCII, which means that ASCII is compliant utf8. The same wasn't true at the time.
I thought it was fine really. It forced us to drop some of the quirkier characters sooner. Nobody in the Netherlands uses things like the paragraph sign anymore, or the 'ij' glyph. And accents on words are now pretty much optional.
> Jonathan Blow, in a talk about how software is getting worse, points to the example of Ken Thompson's text editor, which Ken built in a single week.
This kind of sentiment drives me crazy. Yes, internationalization makes things harder and more complex than they were in the old days. But, back then, only people who lived in one small part of the world were able to usefully use computers in their language. (Simply supporting the Unicode character set is not enough.) Nostalgia for the simplicity of the past ends up having ugly cultural implications. It's easy to say "let's go back to the time when things were simple"; it's a lot harder to say "folks in (e.g.) Israel shouldn't be able to type in their native language".
>Nostalgia for the simplicity of the past ends up having ugly cultural implications. It's easy to say "let's go back to the time when things were simple"; it's a lot harder to say "folks in (e.g.) Israel shouldn't be able to type in their native language".
This is a ridiculous mischaracterization of what Jonathan Blow was talking about.
Here is the full presentation. Very much worth watching in its entirety and thinking about:
> This is a ridiculous mischaracterization of what Jonathan Blow was talking about.
I watched the full presentation and agree with almost everything he said, but I can't find a part matching the conflict between "Simple text editor Vs. Complicated character encoding scheme / rendering / formatting". How would JB make a text editor that works with any kind of character?
I don't agree with almost anything he said. He doesn't know what he's talking about. He comes from games. I too came from games. I had all the same opinions as him. Then I worked on a browser on a team with hundreds programmers (browsers are huge). The article is a perfect example. He hasn't solved the actual problems being solved so he has no clue why the solutions used are the ones that were chosen.
Could some parts be re-written to be more efficient? Maybe. Could they be made efficient and still be easy for any of those hundreds of programmers to follow and modify? Probably not, even if all of them were of his caliber.
Games just don't have to deal with the same issues. Take Unicode fonts. Just the emoji font is 221meg! I'm pretty sure if you ask JBlo about it he'll give some flippant and asian bashing "emoji shouldn't exist in the first answer". He won't actually "solve the problem that's actually being solved", he'll solve some simpler problem in a simpler way in his head and then declare that's all you need.
He's made all kinds of ridiculous and conflicting claims. Example, he believes there should be only one graphics API. To put it another way, he believes in design by committee since that's the only way there would ever be one API. Yet in other areas he rejects design by committee (game engines would be one).
Another issue is security. AFAICT he's never given it a second thought. As one example his JAI language he pointed out he never runs into memory ownership issues so he doesn't want a language that helps with that. Memory ownership issues is one of the top ways security bugs appear. Again, pointing out he doesn't know what problems are actually being solved and is thinking only from his own limited experience.
agreed, I respect what he did but some claims are ridiculous and his "clan" of followers only makes it worse.
BTW: would you be willing to chat from time to time? Im doing something similar (hobby, mostly one-man show so far) to browser and I would really could use some help. I dont need any programming but pointing me in the right direction would be awesome.
I agree with the sentiment you're getting at about nostalgia for the past sometimes ignores the homogeneity of the people using those systems and how that reduced complexity. I do think there's an argument there that needs to be explored further to identify what is actually simple and hard/complex in computing.
However, I think this point about how JB specifically would make one with any type of character is not the best direction for exploring that. He's made games and his own presentation software (Used in the presentation linked above) which is able to do that. I doubt he would classify that as one of the actually hard aspects of computing.
Yeah I agree making an editor that supports any type of character is not the best way to explore this subject, but I doubt the result of this exploration can solve that editor problem.
I follow a similar mindset of JB and have being making / using my own tools, even my personal character encoding (I have Chinese as my first language and want to use Chinese in my works but Unicode is too messy and it's impossible to make a font that supports thousands of characters so been developing my subset of Chinese that fits in a byte), also my own game engine / renderers / programming language (similar to JB's stack) My experience is these softwares are too personal and not applicable the the real general public. JB also makes personal softwares and won't consider about generality. Personal softwares are inherently better, because the bloat and bad about current state almost all comes from derailed generalization effort. If we take the beautiful DOOM editor and make it general, it'll be real hard to not make it another Unity, and that's the real under-explored problem.
This is accurate in the same way as saying "a rock chiseled into a wheel with a stick as an axle is still useful today". Or maybe "a mechanical calculator and a book of logarithms is still useful today".
Sure, it still exists and still does what it used to do, and its derivatives surround us. But nobody does that any more because nearly everything else is so superior, it'd be ridiculous to do so, except as an art piece or when you literally have no alternative.
sed is basically the stream interface for ed and it's one of the most common unix text utilities, and even ed has its uses today. I remember using it on a machine I was remotely connected to. For some reason the terminal output was garbled and neither vi nor emacs would render properly, but ed worked fine.
Obviously you wouldn't use it day to day, and yeah, you wouldn't use if you have any alternative at all, either. But the fact that it still has some form of (admittedly very limited) competitive advantage over modern tools makes it useful to have /bin/ed handy, even if you actually use it once in a blue moon.
No more uncharitable than saying that modern developers suck at their jobs because they can't make Sublime Text in the same time as it took Ken Thompson to make ed.
> It's easy to say "let's go back to the time when things were simple"; it's a lot harder to say "folks in (e.g.) Israel shouldn't be able to type in their native language".
Is writing a text editor literally so hard that we can't do it twice (or three, or five times)?
There are literally trillions of lines of code in existence now. Instead of making a five hundred thousand line text editor because are terrified of writing one again, maybe we make five ten thousand line editors?
Of course, I'm exaggerating, but do you get my point? There are worse things than duplicated code. Narrowing the requirements for latin and cyrillic left-to-right languages so their code can be made radically simpler seems well worth it! The specialization necessary to make a text editor do Hebrew well, or Chinese or Arabic well is probably way different, and frankly, better done by experts in those languages. From their perspective, to completely flip it around, why should their editors be encumbered by the need to support English? Instead we end up with the proverbial "Jack of all trades, master of none". And we end up with radically complicated, hard to test, terrified-of-breaking-a-language-I-don't understand code that serves no one well.
Editors don't have have to completely diverge in features and feel just because they might have different code bases. Surely we can try to make copy&paste and select all and the common things operate in a fairly standard way across a few different programs.
People who work in multiple languages probably wouldn't be too happy with this. (Particularly not with dropping English since it's the most common second language.)
This also isn't going to work economically for companies selling consumer apps because they aren't going to want to limit their market to a one country, particularly if it's a small one. There's more money in targeting the world, so programmers just have to deal. (The support matrix is often complicated already with multiple platforms to support.)
I think it's fine in many cases to target limited audiences. Companies often have internal apps that aren't internationalized. But the popular apps that we all use have to be very complicated because they actually do try to be universal.
Conceivably we'd be better off with a compound-document standard, like the 90s attempts OLE and OpenDoc but somehow nicer. Then multilanguage documents, with specialized editors, would fall out as one use for compound documents.
This is what was done with codepages before the advent of the internet, basically. The problem that arose was transferring information between systems - even without the internet it was needed at areas that were near borders, etc.
Regarding the vertical cursor movement, back in the day there was an alternative behavior. You can see this in QuickBASIC/QBasic[1]. Here, Microsoft treated the entire screen as a grid. I always thought that was a great UI. It hides the fact that there are line endings (\n on Unix or \r\n for DOS). And why shouldn't it? That's an implementation detail. And as evident by DOS using CR+LF it goes back to the days where you had an actual physical carriage on a typewriter. Nearly every modern text editor you have to type out the spaces/tabs you want to reach the area you what to add text. In QBasic, you just move the cursor wherever you want on the screen with arrow keys and start typing. Because, unlike a typewriter, a computer screen is 2D. Vim has block mode but, AFAIK, can only work on selections. Of course, this is not universal and only works with monospace fonts in certain languages. But back in the day it made it easy to create ASCII/ANSI diagrams and art.
Modern text editing is a multilayered mess. Rendering, input methods, variable width fonts, and the special codes (markdown, etc.). On some sites it's not even self-consistent. Take Jira. Last I checked, they have one editor for creating a ticket and an entirely different editor for editing that same ticket[2] (one of these editors sucks more than the other, but I can never remember which... but neither one is remotely what I would consider a "good" interface). How they got there is truly the 8th Wonder of the World.
I think this feature is called 'virtual space'. I like it too and would always look for it and turn it on if available in any text editor I used. But it's not as common as it once was.
This feature conflicts with whitespace-at-line-end rules, because if your editor truncates ws at eol, it must account for multiline string literals in a generic way, and if it doesn’t, everyone else editing that file has to deal with ················’s scattered randomly all over the text/code/diffs. Especially when “·” renders as as “ ” instead, and pressing the End key turns into a lottery. I can see ways for this mode to remain tempting to someone, but it’s just incompatible with everything beyond that editing session.
I don't see why this is so. I like auto removal of trailing space, I don't see why you can't combine that feature and virtual space. Virtual space just means the cursor/caret moves strictly vertically as you press the up/down keys even as you visit lines that are shorter than the current offset from the left margin. If you alight at such a line and start typing text, the editor simply inserts sufficient spaces to extend the shorter line to the current offset before it inserts the text you are entering. If you are entering only spaces, that's fine too but unless you enter something else at some point all the excess space can be trimmed when you leave the line.
Auto-removal of trailing space is only possible if the trailing space is meaningless. Many programming languages have meaningful trailing spaces which cannot be removed. The most common example is inside multi-line strings.
Fair enough, I agree that virtual space is not useful or even possible if you can't or don't want auto trimmed trailing space as well. Fortunately there are many workflows and languages where auto-trimmed trailing space is a plus, and these overlap with my areas of interest.
Monospace is the more sensible option as an editing medium for symbolic content. It's in having "WYSIWYG" editors that do everything in one modeless widget that we start getting this particular kind of feature creep.
And considering how awful it tends to be to navigate a complex word processing task in Word or LibreOffice even now - mystery-meat selections abound - I'm not sure this is the right feature to support all of the time.
Proportional fonts are more aesthetically pleasing and legible (the entire reason for their existence).
When I'm editing "symbolic content", I like it to be easy on my eyes. If I'm constantly re-reading what I've just written, I appreciate the added legibility.
(With the exception of heavily punctuation-based computer code, of course, where monospace results in greater legibility.)
I rarely care about my position on the "page" unless I'm typesetting something (otherwise I'm just typing from X number of tabs in), and when I'm typesetting, I'm not going to use a monospaced font very often.
I liked doing ANSI art back in the 80s, but that's not really interesting outside of terminal connections to a BBS.
Fun fact: Python and QBasic both came out in 1991. Though QuickBASIC is a few years older.
> should the missing characters be tabs or spaces then?
I'm not sure what QBasic does. But it's irrelevant since they control the format and the display. All Microsoft needed to care about is whether the file displays the same as before when loaded.
But it's really no different than text editors that let you switch a tab character to insert 2/4/8 spaces. If you've ever had the misfortune of working on a code base that allowed both tabs and spaces (fake tabs) then you'll be keenly aware that plaintext has always been modal. Your editor either has to figure out how many spaces to display a tab character, or how many spaces to insert on the tab key. Even line endings, CR+LF vs. LF. I think Mac even just used a single CR once upon a time.
Makefile is another format that is particular about requiring an actual tab character. The first time I "discovered" this I had the urge to toss my computer out the window. That was probably the late '90s when that sort of information was a bit more difficult to come across.
Having touched adjacent technologies to this as well, I could not shake the feeling that text editing on the web, and WYSIWYG text editors especially, would be very well served by some sort of common understanding and agreement of output given certain inputs. As alluded to in the article, the devil really is in the details in this domain. When one knows what to look for, most rich text inputs really miss the mark in understanding intent and doing something reasonable on behalf of the users.
An interesting example is the recent trend to start interpreting selection of text + pasting of a link as "turn this text into a link using the URL in the clipboard". It just makes complete sense. When would I ever want it to be "please delete the selection, and instead paste the URL"? Probably not very often. This is just one tiny example, but there are lots and lots of those in text editing.
My gut feeling is a lot of the time this is all left to individual engineers to try to decipher the user intent; I would assume for the most part they either copy their peers or do a best effort guess.
What if there was some sort of test suite one could run instead? Like a large number of test cases built upon interactions with actual users.
>> the recent trend to start interpreting selection of text + pasting of a link as "turn this text into a link using the URL in the clipboard". It just makes complete sense.
Ugh, no no no no no. Modes like this are the opposite of making sense. Highlight something, and press paste - of course it is a replacement of the highlighted text, because that's what pasting does.
Now, we want highlighting something and pasting to contextually mean "insert markup specific to some markup language, after parsing what's on the clipboard and deciding that it looks like another part of that markup language." And it'll work one way on Jupyter notebooks and another on Slack and I can guarantee you that all of them will confuse my screenreader and I'll spend 5 minutes trying to fix whatever stupid web stuff just happened.
I agree, but that is generally not referred to as modes. A mode in UI is, when it switches and the inputs behave different. For example the "Insert" key is a mode. One could argue that Editing and Debugging (of code) are two different modes.
What you (and me) dislike are "assistants". It is when a UI decides upon something.
If you take close attention you can see clippy flash up and give a helping hand. /s.
If you want to delete the highlighted text, then why not press backspace or delete? Those already have the explicit action you're looking for.
>> Now, we want highlighting something and pasting to contextually mean "insert markup specific to some markup language, after parsing what's on the clipboard and deciding that it looks like another part of that markup language."
Not really sure what you're trying to say. You don't think they do this already? WYSIWYG-editors on the web almost all have normalization and sanitization built in when you paste something, and depending on where you paste the content will get merged with existing rich text blocks.
I think in general editors should be unsurprising.
If there's a general expectation of "if text is selected, and you do a WRITE_TEXT action, then the selected text is replaced with text"; and if CTRL_V is a WRITE_TEXT action; then doing CTRL_V after selecting text should replace the selected text.
Then again, the above rule already has exceptions; eg writing a parenthesis in some editors will surround the text with parentheses.
The problem here is “you can’t quickly make a link in our wysiwyg”. Screwing paste is not a correct answer to that. Let it detect that a clipboard text is a link and tooltip-hint a user that they can combine it with a selection by pressing ctrl-alt-v, or something similar. Nobody is against your easy editing, but not at the cost of a cognitive overload. I wish UI/UX designers read some books on how human mind works or how to do UI/UX at least.
>selection of text + pasting of a link as "turn this text into a link using the URL in the clipboard"
Seems like an interesting UX debate. It's a handy shortcut, but it makes "paste" modal depending on what's in the clipboard. Pasting might replace the selected text, or might link it. And because the behavior depends on whether the clipboard text "looks like" a URL, there's an element of DWIM (Do What I Mean) in there. Does it only handle http:// links? What about www.? Or mailto:? etc.
I haven't experienced the behavior described but reading it, I instantly hate it. It would differ from every other program I use.
I'd counter with there should be two different paste actions in every program: plain and smart. I almost always don't want to preserve rich formatting when I paste text. It looks like my cat walked over the keyboard while I was typing. The action to paste as plain text versus a fancier/DWIMy method is often different from program to program.
> all left to individual engineers to try to decipher the user intent
When it comes to a lot of text rendering, Unicode does specify most of it along with test suites.
While text input is really mainly two large teams: one at Microsoft and one at Apple. While they probably don't talk to each other much, they probably do wind up copying each other for the benefit of users.
I'm not really sure what open-source GUI's do, but probably mainly try to copy Microsoft+Apple.
recent trend to start interpreting selection of text + pasting of a link as "turn this text into a link using the URL in the clipboard". It just makes complete sense
I had to reread this 5 or 6 times to get the meaning, not joking. Please no, or at least use custom plugins for that and leave our generic text input alone.
A good article, which shows that even plain text editing (without any formatty) is already way more complex than a layperson may appreciate.
For the developer, there are some gems to be discovered when writing a text editor; recall that using regular expressions coverted to finite state machines saw the light as part of editing, and I remember being given a very cool paper (from the 1960s!) by my undergrad compiler construction professor that proposed an efficient algorithm for testing whether a string containing a line of text can possibly be a substring of any valid Pascal program (it was a piece of theory invented while and for a syntax-aware Pascal editor, possibly from IBM but not sure). Another thing is data structures: I learned of ropes, an alternative to strings for representing text (optimized for fast editing) in the context of text editing: https://dl.acm.org/doi/10.1002/spe.4380251203
On the emoji color (skintone) modifier... A normal person wants to select their emoji from a set. Only a programmer would allow the user to input a tiny program with a symbol and a color modifier and then interpret it.
If you want to make text editing a little easier, treat every character along with any style applied to it as a single entity. This way you avoid a lot of potential problems like in TFA as well as things like nesting of modifiers that may not be allowed to overlap - output those when all the characters are complete. In other words, don't track tags but generate them as a post-process when storing the data.
Yeah, the "Bad #2" example, how Chrome does it, makes perfect sense and seems correct to me. Modifiers get removed when the character they're modifying is removed. I would say leaving the modifier and applying it to the previous character would be certainly incorrect (unless you're in a hex editor, in which case combining the bytes as emoji at all is questionable). Imagine it were two different emojis and a skin color modifier after the last one (and somehow, through a rendering bug, the skin color modifier hadn't auto-combined) - OP's precedent for what happens when deleting the middle emoji would move the modifier to the first emoji! Clearly wrong, especially if it then decides to auto-combine.
I mean, the example is pretty contrived in the first place:
> Whoops, I didn't mean to put the a there.
How in the world are you typing emoji such that you have to add the skin color modifier separately that would even afford an opportunity for an errant character to get inserted between? Windows numpad alt codes?
I second that. Half of these “issues” arise from the fact that author is a programmer-y person and knows how unichars work. Normal people have no clue and completely no expectations on that matter, and don’t wanna have.
Neither do I. If I’d fuck my emoji up in such way, I’d just expect to backspace it and type it from scratch. No need to spend mythical man-epochs on a trivia nobody really cares of.
> This is why, on the web, text input and keypresses are separate events. Terminals conflate these two, causing problems.
Yeah, so please don’t create terminal TUIs please. Very disturbing to use TUIs that don’t work with CJK languages at all. Please, please create GUIs instead of TUIs. (I’m not blaming programs reading from stdin and printing data to stdout, but the ones based on ncurse and fill the terminal screen.)
Unfortunately the status quo on text editing didn’t improve since the last time this came up on HN. Unfortunately everything I said the last time[0] still applies…
Elsewhere here on HN people are complaining about those scientists who can't seem to replicate their results. But here I'm reading that all those smart programmers can't even handle text selection right.
So I guess I just want to say: things get complex, really fast, and a little humility goes a long long way.
Smart people love to solve complexity, but they often forget that “how it should work” is a complex knowledge by itself and the more people know, (1) the less they trust an implementation at hand, and (2) the more likely they have a different opinion. Sometimes you just have to kick the can down the road and accept that nothing and nobody is ideal. Then you get a working system that everyone loves (except chinese).
These days I make sure I am away making a cup of tea when these kind of topics are discussed at work. After writing a syntax highlighting editor in Delphi and a rich text editor in Flash/AS3 I try to avoid it.
You know, every text editing system I've seen that knocks off word (has those "bold", "italic", etc.) selectors has something anomalous in how selections are handled.
I think this overstates the general case a bit, although it has some great examples.
Basic text editing is not inherently hard, as the people here citing “ed” point out. Even text editing left-to-right ASCII in mouse-driven WYSIWYG style is not too hard -- I’ve done it, for old Mac apps where there was no good reusable OS widget, and it was tough but tractable.
What is incredibly hard is supporting editing in many different scripts. I think it’s hard for two reasons. First, there’s inherently just a lot of ground to cover -- multiple scripts, multiple encodings, multiple text directions, contextual rendering, emoji. Second, you can’t cleanly separate it into layers, as text input, text selection and text rendering all impact each other (I love the example of Chinese text input messing with vim’s command mode).
If you could just find a way to keep those layers separate it would go back to tough-but-tractable. But from a user’s perspective, having the layers mixed up is (arguably) what makes it easy to use! You don’t need to switch back and forth between different modes to do stuff, it’s all just WYSIWYG. (Although what with emoji selector popups and the like it’s not actually perfectly modeless, like it can be in the left-to-right 8-bit encoding case.)
I suppose the latex approach is one way to make some of this easier. The document is plaintext, just some letters snd symbols and all the formatting and fonts etc is specified explicitly through typed codes. So that solves the problem of invisible boldface characters and line breaks. I always thought it bothersome how latex worked but it does make some amount of sense to separate content from presentation.
Yeah text editing is hard. I've yet to find a Linux console editor that can do word wrapping smoothly yet preserve the normal line endings. Basically what the textbox on every GUI browser does right out of the box. It seems to be a very difficult ask.
I remember back in the early 90s wanting to write a custom IDE for LaTeX. Even in those 8-bit L-R text only monospaced font days, text editing was remarkably complex. Complex enough that I gave up before I started as I faced down the complications of being able to move from one line to the next.
> The emoji modifier is now modifying the newline character.
In that case, the modifier should be displayed at the end of the preceding line, as newlines are generally associated with the line they are ending, not with the subsequent line.
Good article but I would like to point out that I too looked into the Windows Text Services Framework years ago and it was literally the worst designed and implemented API I had ever worked with in my career. I got ahold of some engineers within Microsoft to ask wtf? and their response was that was written by some disconnected Asian office who understood Asian requirements well but were terrible architects.
That kind of post brings philosophical questions as well. Do we need to support 7000 languages? Should we? If we take a vision of United Earth a-la Star Trek, should there be support for any script other than “United Earth script?” Is in reasonable or unreasonable to expect countries/languages/people to upgrade their script to more simple/efficient things? Russian alphabet has had some number of simplifications and reductions throughout the history, can the same happen to Mandarin? Bunch of my Indian friends are more comfortable typing Hindi using Latin script rather than the native one. Not because of technical hurdles but because it’s cumbersome to apply all correct modifiers.
In other words, should computers adapt to people here or should people adapt to computers or some kind of mutual movement towards each other with finding a middle ground?
There’s a phonetic alphabet that contains all sounds from all languages (I don’t remember the name). It would be interesting to exercise a thought of a keyboard built to accommodate it and that alphabet serving as universal alphabet for the planet.
