It is more than just that it uses lisp. I do like that, and I think it is the correct choice. But it is more that even something as basic "move cursor down" is not tied directly to a specific key. And the help system will literally take you to the source for any command, if you ask it to. (Takes some setup for this to work down to the c source, but it is there.)
Is a bit like having a problem with "ls" and wondering if you could fix it by scanning the source real quick. In emacs, that is often just a keystroke away. In most systems, good luck.
I'd go further. Some of the world's best tooling is only usable by the world's best users. Examples abound on this. The best drivers are in vehicles that I would almost certainly crash. Our best pilots are in planes that I literally don't understand the controls on. (That is true for the cars, too, oddly.)
A really good guitar is easy to miss notes on. Precisely because good guitarists don't typically miss.
Now, I think you could reword a little bit. The best performers know their tools very well. So well, that they often "just disappear" for them. This does require some level of quality in the tool. But requires a ton of effort from the performer, too.
As you get better at something you become more opinionated at what you need your tools to do. You demand more specific and tailored things from your tools and so you start to lean towards things that are more adjustable.
There is also the case that once your entire livelihood depends on something, consistency and muscle memory matter a lot. Lots of world-class athletes, drivers, and performers probably use tools that closely resemble the tools they learned and trained with their whole lives so they would probably seem kinda anachronistic to a newcomer.
More generally, the key here is that the more magic you want in the execution of your code, the more declarative you want the code to be. And SQL is pretty much the poster child declarative language out there.
Term rewriting languages probably work better at this than I would expect? It is kind of sad how little experience with that sort of thing that I have built up. And I think I'm above a large percentage of developers out there.
I'm curious on the uptake of SIMD and other assembly level usage through high level code? I'd assume most is done either by people writing very low level code that directly manages the data, or by using very high level libraries that are prescriptive on what data they work with?
How many people are writing somewhat bog standard RUST/C and expect optimal assembly to be created?
I was heavily into assembly before I discovered C. For the first decade and half or so I could usually beat the compiler. Since then, especially when supporting multiple architectures I have not been able to do that unless I knew some assumption that the compiler was likely to make wasn't true. The 'const' keyword alone killed most of my hand optimized stuff.
In the end the only bits where I resorted to assembly were the ones where it wouldn't make any sense to write stuff in C. Bootloaders, for instance, when all you have to work with is 512 bytes the space/speed constraints are much more on the space side and that's where I find I still have a (slight) edge. Which I guess means that 'optimal' is context dependent and that the typical 'optimal' defaults to 'speed'.
I think this is talking past my question? I don't necessarily think "low level" has to be "writing assembly." I do think it means, "knows the full layout of the data in memory." Something a surprising number of developers do not know.
I've literally had debates with people that thought a CSV file would be smaller than holding the same data in memory. Senior level developers at both startups and established companies. My hunch is they had only ever done this using object oriented modeling of the data. Worse, usually in something like python, where everything is default boxed to hell and back.
I was really only responding to this part, apologies for not quoting it:
> How many people are writing somewhat bog standard RUST/C and expect optimal assembly to be created?
As for:
> I don't necessarily think "low level" has to be "writing assembly." I do think it means, "knows the full layout of the data in memory." Something a surprising number of developers do not know.
Agreed. But then again, there are ton of things a surprising number of developers don't know, this is just another one of those.
Similar stuff:
- computers always work
- the CPU is executing my instructions one after the other in the same order in which they are written on the line(s)
- the CPU is executing my instructions only one at the time
- if I have a switch (or whatever that construct is called in $language) I don't need to check for values I do not expect because that will never happen
- the data I just read in is as I expect it to be
You can probably extend that list forever.
Your CSV example is an interesting one, I can think of cases where both could be true, depending on the kind of character encoding used and the way the language would deal with such a character encoding. For instance in a language where upon reading that file all of the data would be turned into UTF-16 then, indeed, the in memory representation of a plain ASCII CSV file could well be larger than the input file. Conversely, if the file contained newlines and carriage returns then the in-memory representation could omit the CRs and then the in memory representation would be smaller. If you turn the whole thing into a data structure then it could be larger, or smaller, depending on how clever the data structure was and whether or not the representation would efficiently encode the values in the CSV.
> My hunch is they had only ever done this using object oriented modeling of the data.
Yes, that would be my guess as well.
> Worse, usually in something like python, where everything is default boxed to hell and back.
And you often have multiple representations of the same data because not every library uses the same conventions.
Totally fair. I had meant that to be only concerning complicated assembly. Either some of the more obscure instructions or SIMD, in general.
I think my more general idea, that I was vaguely grasping towards, is that the style of programming that produces optimal assembly is already a different style than most people use. Certainly a different one than most people talk about using.
And fair on the CSV points. I definitely meant that to be in numeric heavy CSV files. Strings are a whole other beast. I put https://taeric.github.io/challenging-my-filesize-intuition.h... together a few years ago when I had this discussion at work the last time.
Oh, another one: the byte length of my strings is the same as the character length of my strings. That one is so nasty, especially in a UTF-8 setting where if you naively test with ASCII strings the tests will all be written with that assumption in mind.
It's really only comparable to assembly level usage in the SIMD intrinsics style cases. Portable SIMD, like std::simd, is no more assembly level usage than calling math functions from the standard library.
Usually one only bothers with the intrinsic level stuff for the use cases you're saying. E.g. video encoders/decoders needing hyper-optimized, per architecture loops for the heavy lifting where relying on the high level SIMD abstractions can leave cycles on the table over directly targeting specific architectures. If you're just processing a lot of data in bulk with no real time requirements, high level portable SIMD is usually more than good enough.
My understanding was that the difficulty with the intrinsics was more in how restrictive they are in what data they take in. That is, if you are trying to be very controlling of the SIMD instructions getting used, you have backed yourself into caring about the data that the CPU directly understands.
To that end, even "calling math functions" is something that a surprising number of developers don't do. Certainly not with the standard high level data types that people often try to write their software into. No?
More than that: many of the intrinsics can be unsafe in standard Rust. This situation got much better this year but it's still not perfect. Portable SIMD has always been safe, because they are just normal high level interfaces. The other half is intrinsics are specific to the arch. Not only do you need to make sure the CPUs support the type of operation you want to do, but you need to redo all of the work to e.g. compile to ARM for newer MacBooks (even if they support similar operations). This is also not a problem using portable SIMD, the compiler will figure out how to map the lanes to each target architecture. The compiler will even take portable SIMD and compile it for a scalar target for you, so you don't have to maintain a SIMD vs non-SIMD path.
By "calling math functions" I mean things like:
let x = 5.0f64;
let result = x.sqrt()
Where most CPUs have a sqrt instruction but the program will automatically compile with a (good) software substitution for targets that don't. It's very similar with portable SIMD - the high level call gets mapped to whatever the target best supports automatically. Neither SIMD nor these kind of math functions work automatically with custom high level data types. The only way to play for those is to write the object to have custom methods which break it down to the basic types so the compiler knows what you want the complex type's behavior to be. If you can't code that then there isn't much you can do with the object, regardless of SIMD. With intrinsics you need to go a step further beyond all that and directly tell the compiler what specific CPU instructions should be used for each step (and make sure that is done safely, for the remaining unsafe operations).
I knew what you meant. My point was more that most people are writing software at the level of "if (overlaps(a, b)) doSomething()" Yes, there will be plenty of math and intrinsics in the "overlaps" after you get through all of the accessors necessary to have the raw numbers. But especially in heavily modeled spaces, the number one killer of getting to the SIMD is that the data just isn't in a friendly layout for it.
It is similar, except with a lot of fractals, the numbers being colored represent how many iterations are required to get outside of a set threshold (which indicates divergence).
Right, I just meant more that you plot more than just the equality to get some visualizations. Is also a common way to visualize game theory stuff, I thought. You want to know where the expected equilibrium is, but you also want to see, essentially, what the strength of getting there is.
I confess that picking yaml doesn't feel safe to me. As much as it annoys people, fenced formats are the way, here. And if you aren't careful, you are going to recreate XML. Probably poorly.
I will forever think of Bloom filters as "bouncer filters." Could go with concierge filter. Basically, it is the equivalent of every movie where the detective is asking the front desk various attributes of who they are looking for.
It is not hard to see how you could start asking the front desk to track every obscure attribute and to expect to fall over for various reasons.
I'm curious how this dream is superior to where we are? Yes, things are more complex. But it isn't like this complexity didn't buy us anything. Quite the contrary.
Totally depends on who "us" and isn't. What problem is being solved etc. In the aggregate clearly the trade off has been beneficial to the most people. If what you want to do got traded, well you can still dream.
Think about using a modern x86-64 cpu core to run one process with no operating system. Know exactly what is in cache memory. Know exactly what deadlines you can meet and guarantee that.
It's quite a different thing to running a general purpose OS to multiplex each core with multiple processes and a hardware walked page table, TLB etc.
Obviously you know what you prefer for your laptop.
As we get more and more cores perhaps the system designs that have evolved may head back toward that simplicity somewhat? Anything above %x cpu usage gets its own isolated, un-interrupted core(s)? Uses low cost IPC? Hard to speculate with any real confidence.
I just don't know that I see it running any better for the vast majority of processes that I could imagine running on it. Was literally just transcoding some video, playing a podcast, and browsing the web. Would this be any better?
I think that is largely my qualm with the dream. The only way this really works is if we had never gone with preemptive multitasking, it seems? And that just doesn't seem like a win.
You do have me curious to know if things really do automatically pin to a cpu if it is above a threshold. I know that was talked of some, did we actually start doing that?
> Was literally just transcoding some video, playing a podcast, and browsing the web.
Yeah that's the perfect use case for current system design. Nobody sane wants to turn that case into an embedded system running a single process with hard deadline guarantees. Your laptop may not be ideal for controlling a couple of tonnes of steel at high speed, for example. Start thinking about how you would design for that and you'll see the point (whether you want to agree or not).
Apologies, almost missed that you had commented here.
I confess I assumed writing controllers for a couple of tonnes of steel at high speed would not use the same system design as a higher level computer would? In particular, I would not expect most embedded applications to use virtual memory? Is that no longer the case?
But why do we think that? The complexity would almost certainly still exist. Would just now be up a layer. With no guarantees that you could hit the same performance characteristics that we are able to hit today.
Put another way, if that would truly be a better place, what is stopping people from building it today?
Performance wouldn’t be the same, and that’s why nobody is manufacturing it. The industry prefers living with higher complexity when it yields better performance. That doesn’t mean that some people like in this thread wouldn’t prefer if things were more simple, even at the price of significantly lower performance.
> The complexity would almost certainly still exist.
That doesn’t follow. A lot of the complexity is purely to achieve the performance we have.
I'm used to people arguing for simpler setups because the belief is that they could make them more performant. This was specifically the push for RISC back in the day, no?
To that end, I was assuming the idea would be that we think we could have faster systems if we didn't have this stuff. If that is not the assumption, I'm curious what the appeal is?
That’s certainly not the assumption here. The appeal is, as I said, that the systems would be more predictable and tractable, instead of being a tarpit of complexity. It would be easier to reason about them, and about their runtime characteristics. Side-channel attacks wouldn’t be a thing, or at least not as much. Nowadays it’s rather difficult to reason about the runtime characteristics of code on modern CPUs, about what exactly will be going on behind the scenes. More often than not, you have to resort to testing how specific scenarios will behave, rather than being able to predict the general case.
I guess I don't know that I understand why you would dream of this, though? Just go out and program on some simpler systems? Retro computing makes the rounds a lot and is perfectly doable.
This falls into the "people aren't rational/logical" trap. You really have to define what you would mean for it to be rational. And then you have to establish what the baseline ideas are from which you plan on establishing the logic/rationality. All the while, you need to really resist any "just so" stories that would explain anything.
Worse, this also falls into the trap of thinking you have to make a choice, and that an informed choice is better than chance. Reality is that you are often best trying to do both, and then having some sort of coin flip to disambiguate choices works far better than we'd like to admit. In large part for the simple reason that simply doing something is more than a lot of your competitors will be doing.
Is a bit like having a problem with "ls" and wondering if you could fix it by scanning the source real quick. In emacs, that is often just a keystroke away. In most systems, good luck.
reply