These are all tiny, targetted microoptimizations worth a percent or three of benefit in specific tests. They're worth doing (or at least evaluating) in any mature product and I have no complaint.
Nonetheless rustc remains really slow relative to other similar technologies, including C++ compilers. Is there any consensus as to why?
I mean, with C++, the answer is something to the effect of "template expansion happens syntactically and generally has to be expressed in headers, leading to many megabytes of code that has to be compiled repeatedly with every translation unit". And that isn't really amenable to microoptimization. We all agree that it sucks, and probably can't be fixed with the language as it's specified, and chalk it up to a design flaw.
What's the equivalent quip with rustc? I mean... is it going to get faster (in a real sense, not micro), or is it not? Is this fixable or not, and if not why?
> I mean, with C++, the answer is something to the effect of "template expansion happens syntactically and generally has to be expressed in headers, leading to many megabytes of code that has to be compiled repeatedly with every translation unit". And that isn't really amenable to microoptimization. We all agree that it sucks, and probably can't be fixed with the language as it's specified, and chalk it up to a design flaw.
Isn't this what happens with rust too ? It's called "monomorphisation" there but everytime you use a Vec<i32> in Rust I would guess that what happens is fairly similar to a template instantiation.
Most of the time of compilation is spent in LLVM since Rust generates a lot of IR. I haven't looked deeply into the issue, but it wouldn't surprise me that monomorphization combined with the functional programming patterns is one cause of code bloat. For instance, every time you use `map` (or another HOF) with a closure you end up with two generated functions per usage: One for the closure and one for the `map` monomorphed for that specific closure.
Compare this to a `map` function which is only generic over the return type:
fn map<T>(self, f: &FnMut(Self::Item -> T)
This function would only be instantiated once per unique T, regardless of how many times you use it with different closures. This halves the number of trivial functions that LLVM need to process and (usually) inline. The reason `map` isn't defined like that is that it uses dynamic dispatch instead of static dispatch.
However, I don't actually know how much a difference it would make, but it's my understanding that LLVM was never optimized for the use case where you have many, tiny functions (which is more common in functional languages).
The equivalent is doing more dead code analysis at the MIR stage before sending it to llvm. This is where rustc loses most of its perf. LLVM is usually 50-60% of execution time, mostly because rust sends teams of ir to chew through.
Serious question: is there any reason dead code detection would be any faster at the syntax level than the generated code? I mean, I'm no compiler expert but in a straightforward SSA representation "dead code" is just a register that gets assigned but never used, all of which can be detected and dropped globally with just one pass over the data.
I suspect the real reason is going to turn out more complicated, no? Like, the code is really "dead" yet because we can't know if it will be used later without solving the halting problem.
Which, if you think about it, is pretty much exactly the situation C++ is in. All those expanded templates aren't "dead" yet at the point of expansion either, you just don't know if the code will use them. And in practice it won't, so you wasted a ton of cycles to find that out.
> Serious question: is there any reason dead code detection would be any faster at the syntax level than the generated code? I mean, I'm no compiler expert but in a straightforward SSA representation "dead code" is just a register that gets assigned but never used, all of which can be detected and dropped globally with just one pass over the data.
That is only one type of dead code elimination. It is not necessary to solve the halting problem to eliminate larger portions of code than just unused register assignments. There are plenty of ways to eliminate beyond that, such as proving that entire classes, functions, declarations, branches, loop iterations, and data are unreachable. Especially so in a type-resolved format like MIR, some of these higher level optimizations become really cheap and would allow for LLVM to do a lot less work.
In general, you see inflation in code size as the representation becomes lower level and more explicit. Traversing a larger volume of code to detect dead code is generally more expensive than traversing a smaller volume of code. This might not be true if the deadness determination is really expensive, but it's almost always worth doing some kind of lightweight dead code elimination before each stage of lowering to save time in the subsequent stage.
> I mean, with C++, the answer is something to the effect of "template expansion happens syntactically and generally has to be expressed in headers, leading to many megabytes of code that has to be compiled repeatedly with every translation unit".
That's not really the case in C++. Most code in headers is not compiled repeatedly in every translation unit. The vast majority of it is dead code, which compilers are very good at discarding at an early stage.
> We all agree that it sucks, and probably can't be fixed with the language as it's specified
A lot of the overhead can be fixed, and is fixed, via precompiled headers, which have been around for decades.
> What's the equivalent quip with rustc? I mean... is it going to get faster (in a real sense, not micro), or is it not? Is this fixable or not, and if not why?
At this point, rustc and C++ compiler performance problems mostly consist of little things that add up. As engineers we always want there to be a magic bullet that will solve the performance problems, but in mature technology like compilers, those magic bullets often don't exist.
Ironically, C++ often doesn't suffer from the "too much code" problem as badly, because in practice programmers don't write "modern C++" nearly as much as C++ fans think they do. Industrial C++ uses raw pointers and C idioms all the time, and these tend to compile faster than the equivalent "modern C++" idioms. Rust, on the other hand, forces you into the corresponding "modern" idioms, and the compilation time tends to be slower as a result.
To this end, probably the most fruitful area we can explore in rustc is fast MIR-level optimizations to reduce the amount of code that we send to LLVM. In effect, this would desugar the "modern" style to the "raw" C style before sending it off to the heavyweight optimization and code generation framework.
In swift it's the type inference and operator overloading combination that causes long compile times. The operator + for example can have a lot of implementations, and since types have to be inferred instead of being declared upfront, the combo creates slow compile times:
Ex, there is this infamous error in swift:
"Expression was too complex to be solved in reasonable time; consider breaking up the expression into distinct sub-expressions"
Exponential performance cliff compiling associated types with equality constraints[1]. It was first opened in 2015 and was only recently addressed. Rust certainly has the potential for exponential times when performing large inference.
IME compiling C++ in debug mode (-Og) is decently fast; faster than Rust.
Are there any languages with fast compilers that aren't designed from the start for fast compilation? When I think of "fast compilation" I think of Wirthian languages.
This particular article is about a Firefox developer deciding to spend a little bit of time applying some of his knowledge on profiling and optimizaiton to the Rust compiler, without being a core compiler developer and involved in any of the big performance-related initiatives that are ongoing.
So it's going to focus on a few micro-optimizations that someone can find by doing a little bit of profiling and making some small, isolated changes.
I think there are a couple of major reasons why rustc is slow, and then a lot of ones which just boil down to "we haven't gotten to that yet."
One of them is that rustc was originally written as a self-hosting compiler while the language itself was still in flux. So for the early years, there was a lot of churn just due to keeping up with the language changes, and just implementing the language, not being able to really focus on performance and optimizations.
Once the language stabilized, the compiler was not really the best written piece of code in Rust because it had been written before some features existed, before people had figured out idiomatic ways to do things in Rust, and because it had been written with the assumption that it could just work with the AST and a high-level IR and translate that directly to LLVM and let LLVM handle all of the optimizations.
Shortly after the 1.0 release, a big project began to define and use "MIR", a mid-level IR between the high-level IR that is much closer to surface syntax and LLVM's low-level IR for generic optimizations and code generation. That project took a while to complete, but has finally been completed; that lays the base for new language features, improvements in existing language features, and being able to do better work on optimization; it has led to several major improvements in compiler performance that have already landed.
Another issue is, pretty similar to the C++ issue; Rust has generics, which offer many of the same compilation issues as C++ templates. They are constrained a bit more than C++ templates, and don't have the SFINAE philosophy, so I think it's possible for the compiler to be smarter with those, avoid monomorphization in some cases, and so on, but it is still likely that in heavily generic code, you may not be able to do much better than heavily templated C++ code.
There are also some issues that are just holdovers from the fact that language evolution, correctness, and ease of development was prioritized over compiler performance in the the initial versions; for instance, the de-sugaring of high-level constructs causes rustc to produce a lot of LLVM IR that LLVM winds up just optimizing away. Having rustc be smarter and produce less of this is a major optimizaiton goal.
A few major examples of improvements that are underway or being considered: sharing generic code between crates (if two crates in your dependency graph each instantiate Vec<i32>, you shouldn't have to compile that twice), sharing closures between generic instances (if there's a function that depends on some generic type T, but a closure in it that does not, there's no need to duplicate that closure for both instances of that function), doing some inlining in MIR (lots of high-level Rust features depend on inlining to produce "zero-cost abstractions", so doing that earlier in the process may reduce the amount of work that LLVM has to do).
And then there are improvements for allowing for better parallelization and incremental compilation, better link time optimization, and so on.
tl;dr: it will get faster, in a real sense, not micro, but in the end it will be constrained by language design to be closer to the C++ compiler end of the spectrum than the Go end of the spectrum
It would be nice to be able to use something like Keith Winstein (Stanford) et al's gg.[1] Sort of `make -j1000` for 10 cents on Lambda. Linux kernel cold compilation in under a minute.
Clang has a profiler for why your C++ compiles slow. Last I saw templates were not responsible for much in a typical codebase.
For Rust, though, the answer is obvious - the borrow checker. The language is designed around the idea of the compiler proving whether or not the code is safe and inserting appropriate destructors where they should be. I think it's expected that this will be slower than a language that just doesn't do that.
The borrow checker is clearly not the issue, if you look at the numbers. It does take some time but is far from the most expensive section in every profile I’ve ever seen.
The NLL version is slower than today’s, at the moment, though.
The query parallelization work has been coming along. Also it was recently discovered that building LLVM with a recent clang and doing cross-language LTO speeds up builds by ~25%.
What I've heard is that the output that the Rust compiler gives to LLVM is much more complex than it needs be, and optimising this is the simplest way to reduce the time spent in LLVM. This seems plausible to me as there aren't many complaints that other compilers using LLVM are slow.
> there aren't many complaints that other compilers using LLVM are slow.
Practically every VM that tried to use LLVM as a JIT gave up as it was too slow. WebKit eventually abandoned their LLVM backend and wrote their own, B3.
JITs are of course a specialized use case, but there are also known issues with compilation speed in LLVM. For example, LLVM is single-threaded, so compiling a single very large compilation module can be much slower than it needs to be - and this is an issue hit by Rust due to how crates are designed. But you are also very correct that Rust is generating overly-complex LLVM IR for LLVM to compile, which is another problem there.
Also worth noting that while once clang beat gcc in compilation speed, as clang's optimizations have caught up to gcc the difference has vanished, and on many benchmarks today gcc compiles more quickly.
> Practically every VM that tried to use LLVM as a JIT gave up as it was too slow.
And it doesn't even seem to work effectively as a target for JIT. It can do the machine code emitting part of a JIT well, but that wasn't really the problem and it doesn't seem to work well at the real problem of optimising code for a high-level language. You need to optimise yourself, by writing basically your own compiler pipeline, before emitting LLVM.
See for example the Rubinius implementation of Ruby.
Right, but compilation performance is not a race in which you have to be first, but a minimum threshold that you must meet. Compilation just has to be fast enough that it doesn't annoy developers most of the time. Once you achieve that they'll mostly stop complaining. I believe Clang et al are fast enough.
(This is true for performance and optimisation is general: fast enough is good enough.)
Compiling larger projects, I certainly do not feel that Clang et.al. are fast enough. It takes minutes to compile stuff at work with 32 Xeon threads powering through at full load.
Once it takes seconds, we can talk about "fast enough".
It's mixed C/C++, and the codebase does not contain a lot of "heavy" features like templating and header-only implementations (I'm looking at you, boost).
Not the commenter you're responding to, but at my previous place I dealt with compile times of around 20 minutes from scratch, and 1-2 minutes incremental using Clang/LLVM, on a 20ish physical core Xeon with 128ish gigs of RAM.
No, I'm talking about a normal incremental build (which usually touch many things—I don't usually get to only recompile a single unit). If the planets align, I can get an incremental build to take just 30 seconds or so, but that's rare enough that I might mark such an event in my calendar. And that's on a beast of a server.
With Go, on my laptop, 30 seconds is the time it takes to perform a complete, optimized build of a medium-sized application, all dependencies and the standard library from source.
It's always so... frustrating whenever I go from a Go project back to our large C/C++ mixed codebase...
Because Go doesn't do very much optimization. They've been fighting back performance regressions every time they try to improve the performance of the resulting binaries.
Neither does clang, gcc or rustc unless you explicitly tell it to. I was talking about a normal development session. Why would I make compilation time worse by making optimized builds?
Not to pick on you or the parent comment, but this is a such a common mistake that it's worth correcting.
"Clang" is a thing not a person, therefore it's appropriate to use et cetera, not et alii. The former is for things, the latter for people (usually limited to authors of academic papers).
I Googled it, and the results were about evenly split on whether "et al." primarily stands for "et alii" (masculine) or "et alia" (neuter). But they agree that it can stand for either of those, as well as "et aliae" (feminine). In Latin, neuter nouns don't generally refer to people, so whether neuter is supposedly the primary meaning or just an option, it seems to mandate that the phrase can be used for things. On the other hand, the converse isn't true: neuter nouns don't refer to people, but masculine and feminine nouns frequently refer to things. In fact, the neuter gender only accounts for a relatively small fraction of Latin nouns. Thus, even "et alii" or "et aliae" would often be the correct choice in Latin for lists of things, depending on the type of thing. (Of course, there's no Latin word for "compiler", so there's no way to say which would be most appropriate in this particular case, unless you refer to the Vatican's book of Latin neologisms or something.) The word itself just means "other", with nothing inherently limiting it to describing people. In English, substantive adjectives (adjectives without nouns) do tend to default to people unless a noun has been specified: thus it's natural to say "compilers including Clang, GCC, and others", but not just "Clang, GCC, and others". But Latin likes to use substantive adjectives much more liberally. Especially in the neuter, I'm pretty sure "alia" is perfectly good by itself where in English you'd need to say "other things".
Anyway, you could argue that the use of "et al." in English has a restriction that doesn't come from the original Latin. Many of the Google results I found do claim that there's a convention of using "etc." for things and "et al." for people, although that seems to conflict with the idea that "et al." primarily stands for "et alia". But they mostly don't claim that that convention is a hard rule. And I don't think it makes much sense to establish one, considering the original meaning.
You may be right in terms of Latin, but now that both words are being used as English expressions, the original Latin can only be a guide, not a rule. In formal writing I'd try to get it "right" but informally if it doesn't make it harder to read, it's fine imho
Language evolves. The original authors' intent was conveyed much better, in my view, by 'et. al.' then it would have been by 'etc.' given the shift in how that latter phrase is used.
I interpreted that as a deliberate and sassy way to write. It's as if I said with Python et al. eating its lunch, Clojure better light a fire under its own ass.
This is a genuine case of survivorship bias in action.
I guarantee you that there are people not using Clang or writing, let's say, C++ because of slow compile times. You just never hear about how much they hate using them, because those people no longer are using them.
But that threshold isn't a constant across developers. Coming from the world of dynamic languages I still find Go's compiler to be annoyingly slow at times.
That really depends on your definition of "work". They certainly spend more clock cycles and produce more heat.
They might have some more comprehensive optimization passes eating extensive cycles, but even rustc debug builds are extremely slow in comparison to the Go standard compiler.
I'm not sure what a definition of "work" would look like where the Go compiler is doing as much "work" as the Rust compiler, but I expect it would be extraordinarily contrived. Go deliberately prioritizes compiler speed, and definitely skips complex optimizations that Rust or LLVM do, and Go is just generally simpler to compile, because the compiler will do nothing even remotely resembling the borrow checker, nor does it have complex types, and so on.
(One of my benchmarks for "Go has really made it" is when someone starts selling an optimized Go compiler that does the expensive optimizations. You could still ideally use the standard Go compiler to develop, but then you'd test and deploy with the slow optimized compiler.)
Complex optimizations do not apply to debug builds, so we are basically only comparing borrow checker and the likes.
However, then we can draw on two facts to kill that this should be a significant part of the issue: The post states that for most projects, the majority of the compile time is spent in LLVM, not Rust. Second (my memory may fail me here, but this is easy to measure), for larger projects, Go also smoke clang/gcc, which do not have Rusts fancy features.
But, as I stated in my parent comment, these cross-language, cross-compiler comparisons are quite difficult to make. This is also why I attack comments talking about one compiler doing more "work" than another. It cannot be directly compared.
All we know for certain, is that rustc is far too slow for our liking.
If you are implying that it is the "optimizing compiler" I mentioned, I have not seen good performance numbers that show it as being any better than the official Go compiler.
Were I going to start the company to produce the compiler, the question of "why isn't gccgo about 2x faster than Go?" is one of the first I'd examine, though. I personally have zero idea what the answer is. Or, indeed, if it's even still true. However, I am active enough in the Go community that if good benchmarks were going around I'd expect to have seen them. (They'd probably make it to HN.)
I rarely hear much about gccgo than what I hear in the Go release notes (usually "gccgo is now feature complete at <old version>"). I wonder if it still uses the plain thread implementation of goroutines like it did early on (which provides quite different performance characteristics)?
I am frankly too lazy to set up gccgo for comparisons again, but if I recall correctly, it takes considerably longer to compile whilst providing similar quality output. Thus, I would ask the opposite question that you presented:
If gccgo isn't notably faster than gc, why is compilation so much slower?
If one compiler spends more time generating identical quality output with an identical input language, it is hard to argue against the fact that there are wasted cycles at play. It may of course partly be due to an immature Go frontend. Who knows.
>Go deliberately prioritizes compiler speed, and definitely skips complex optimizations that Rust or LLVM do
As the OP pointed out, Rust debug builds are still much slower than Go builds, so lack of optimizations can't be a big part of the story. The simplicity of the language and the deliberate design of the compiler for speed seem to be the main factors.
You should have added the next bit after where you cut the quote off, which was "and Go is just generally simpler to compile, because the compiler will do nothing even remotely resembling the borrow checker, nor does it have complex types, and so on."
I'm not claiming this is true by any means, but it wouldn't surprise me that much that merely the work to tell if a bit of Rust code in, say, Servo, is legal Rust in the presence of a rich type system and the borrow checker and all the other such things going on is more work than it would be to compile the roughly-equivalent Go module entirely. Compared to Rust, Go does not so much "cut corners" as cut entire dimensions out of the picture, and then cut some more corners for good measure.
I cut it off because I was only disagreeing with the bit before the 'and'. Indeed, Rust is a more complex language and it is not surprising that it would take longer to compile. Although, as another poster pointed out, Go code also compiles much faster than plain C code (with gcc or clang), and C is of a similar order of complexity to Go.
>They might have some more comprehensive optimization passes eating extensive cycles, but even rustc debug builds are extremely slow in comparison to the Go standard compiler.
Work is not just the optimization.
Having Rust track lifetimes and warn about ownership bugs, races, etc, is also productive work for the compiler -- and happens during the debug builds too.
The post mentions that the majority of the execution occurs in LLVM, especially for optimized builds. In other words, anything Rust related has to be a smaller part of the compile time.
I stated in my parent comment that cross-language/cross-compiler comparisons is tricky exactly because of the vast differences in both frontends (lifetime tracking, warning generation) and backends (code generation, optimization). So yes, I know there are different amounts of work related with different languages.
However, I believe the compile time differences significantly outweigh the compile time differences.
>The post mentions that the majority of the execution occurs in LLVM, especially for optimized builds. In other words, anything Rust related has to be a smaller part of the compile time.
A, true, by the time it hits LLVM Rust's lifetime's analysis has already happened...
Polly is considered one of the "primary sub-projects of LLVM."
That said, it's generally not seen as part of the main stack, which would be LLVM, Clang, compiler-rt, and maybe lld, libc++, and libc++abi. (The latter three being fully optional replacements where most people use the standard system components instead).
A recent good experience with rust: wanted to learn a little about machine learning. Found a crate called rusty-machine, copied some examples from the docs, and built them.
No problems, compiling all of the dependencies was fast, examples worked, and in literally about 2 minutes I had a neural net and was hacking on it.
So that you can easily rebuild and re-run each one with difference in build settings, instrumentation, etc, and get comparisons between the baseline and the modified version, without having to check out a different branch each time.
Sometimes it's just quicker to work with two separate working directories than switching branches in one.
I cannot speak for OP, but I work on the Pyret compiler. Building a bootstrapping compiler is a lot of work and creates many (intentionally) untracked files. Pulling the repo twice allows me to hack on my fork without doing a full-rebuild every time I need to try something or change something on master.
Total noob question here, but how does the hardware you're using affect compilation speed? What's the baseline hardware being used in this benchmark he runs? There's no way to recreate the benchmark environment unless we know this.
When someone like ajross says rustc is really slow, how slow are we talking?
Thanks, Steve, for the response. Certainly there are a lot of variables so I'd like to see benchmark reports actually disclose what's under the hood.
Let's say, for example, you're on a regular rebuild cadence for your work. What makes the biggest difference for speed in such a scenario: CPU cores, CPU speed, DRAM speed, SSD latency, SSD throughput, something else?
I only own one computer so I can’t really get empiric data here. We do track compile times on the same machine at perf.rust-lang.org, but I do t believe anyone has done those kinds of tests. It’d be neat to have though!
Awesome. I've been checking out the site on the wall time submissions especially. To the point of the original questioning, do you know what's under the hood on the machine you're using to test at perf.rust-lang.org? I'm not seeing it called out on the site so far in my searching...
We generally send our patches upstream but they can take some time to land. This will just always be the case. We test that everything builds with stock LLVM, you’ll probably just get some test failures from the unpatched bugs.
It is interesting because it advances the state of the art: it makes safe "systems programming" possible, which no production ready language previously provided.
It is interesting because it is accessible to many of the people who frequent HN. It's community is similar enough to the community of HN readers that many HN readers can use Rust and not feel out of place. The concepts it introduces are not such a departure from the concepts familiar to many programmers that a huge amount of learning is required. (Compare to, say, more research oriented languages like Idris).
> The concepts it introduces are not such a departure from the concepts familiar to many programmers that a huge amount of learning is required.
More specifically, I would say that it lands in the Goldilocks range: It's not so alien as to make it inaccessible, while being alien enough that you'll learn something from it.
Yeah, "production ready" was the wrong phrase. It's more a combination of availability of language compiler, tooling, training, and interest from developers.
If you start a new project using Rust there is a reasonable chance you'll be able to find libraries for common tasks and developers willing to work on it. You can make a case for it and not lose your job. With Ada, outside of a few niches that isn't going to be the case IMO.
It's been a long time since I looked at Ada, but I believe it's memory mgmt is more inline with modern C++ than Rust. So perhaps it's not truly comparable.
The thing that Ada and its peers lack that Rust has already tackled is finding a strong foothold in OSS. Doubtless there are more lines of Ada than Rust in the wild, but in 30 years of use the only OSS Ada project I can name is GNAT itself, whereas anyone using e.g. Firefox or VSCode is using Rust despite it being ten times younger. Given that most new developers these days seem to learn to program via OSS, and given that HN is largely OSS-focused, it's no surprise that Rust gets mentioned more.
You're missing the qualifier of "production ready". While I'd argue that Ada was indeed "production ready" language wise, I'd say that it's not with respect to libraries, of which, OSS contributes a whole bunch.
For some people, a good library selection is essential to being "production ready".
We must think about quite different things behind "systems programming" if you
need libraries for it. For me system implements and provides interface,
not uses it. Like in operating system.
And then, Ada had a selection of libraries for a long time. You just
conveniently forget about the paid ones, so you can call Ada "not production
ready".
I wouldn't even say conveniently forgot, I'd say it didn't even cross my mind. It is perhaps also representative of why people don't consider Ada when picking out a language.
That said, I agree that it'd be moving the goal posts to omit paid libraries.
--
I do think we have slightly different ideas of "Systems Programming", or at least what production ready means for that type of language. I think it boils down to the fact that people do a lot more in "Systems Programming" languages than just OS or implementing and providing interfaces. Specifically, people do it for performance and control, which I don't necessarily excludes the use of libraries.
I think the contention here is that people are thinking of production ready for these types of languages in terms of general use, not just the niche you describe. In retrospect, I do think it was a poor way to word it.
I guess I'd probably amend the original comment to say something like "Rust is the first production ready systems programming language that can comfortably (after a learning curve) be used at higher levels".
It's not irrelevant. Ada provided safe, production ready language for systems
programming long before Rust, so this part of explaination is simply false.
I wouldn't say it's simply false, just that it requires a bit more explanation. Ada is a safe systems language, but is more restricted in what can safely be expressed.
While you are right, it would also be good to learn the history of computing in a proper way, instead of re-writing it based on urban myths and perceptions.
Ada has memory management safety, even though it is not the same paradigm as Rust. Affine typed pointers might be a tad more powerful than Ada's scoped pointers and limited types, but they're still not powerful enough to express everything useful - cyclic structures in general are a very big hole in Rust's type system in that regard.
And then in both language, you can use unsafe when you need more power. So Rust might (and I say might, because I have no clear proof of that yet) be a tad more powerful in the safe subset, but it's not in a different class.
I've coded in Ada. Storage pools are not as safe as Rust's affine type system. For one it's a lot easier to leak memory with Ada, since they're mainly built off of reference counting.
You're totally right, but yeah, I tend to roll it in to the same category since I consider the denial of service associated with memory leaks to be as bad as stricter memory unsafety. (And to be very pedantic I said 'memory management safety', not just 'memory safety').
Ada uses a garbage collector for the free store (or just uses unsafe code if it's disabled). Ada's memory management model is not really different from any other typical language.
This is the first time I've ever heard of Ada being garbage collected. Neither
ARM, Barnes' nor Ben-Ari's books, nor GNAT manual say that there is supposed
to be a garbage collector (quite the contrary, the books say that GC is
allowed, but usually not present).
It's the first "new" language in a long time that is viable for embedded/bare metal/kernel space, high performance and real time applications.
The memory safety concepts are novel and unique to Rust but enough has been talked about them already.
Additionally, it brings a lot of features from "advanced" programming languages (ML, Haskell) to a more mainstream language. Sum types, pattern matching, powerful macros. None of these are new or unique, but are executed particularly well in Rust.
In addition to the new features and concepts, the Rust compiler and ecosystem are well engineered and have a good amount of support.
It's the first "new" programming language in a decades worth getting excited about. Ok, maybe I was a bit excited about Julia too, but their execution is not as great as Rust's (and they have made some questionable design decisions).
There's also the community. I've never seen another language community built with so much deliberate effort, rather than just accreting around a project. And for a language like Rust, I think this is necessary -- with so many unfamiliar concepts and new ways to fail, it would have been very hard to get any traction without a strong community push.
> Ok, maybe I was a bit excited about Julia too, but their execution is not as great as Rust's (and they have made some questionable design decisions).
Could you expound on this? The code I currently write and expect to write in the near-to-mid-term is numerical for engineering purposes. I have the luxury of being able to write greenfield projects, at least for now. My experience with OCaml has inculcated a deep appreciation for powerful type systems and functional styles, so Python, despite its awesome ecosystem, is not particularly of interest. Julia appears to be the natural choice in this space for new development, but I've been a little leery after reading about Dan Luu's experience a couple of years ago: http://danluu.com/julialang/
I'm interested in reasoned criticisms of Julia as it stands now, and what modern practical alternatives people have found.
Because Hacker news has a large population of systems programmers, and we are desperate for something to liberate us from C and C++. We have watched programming languages advance apace in other areas, helping engineers be more productive and produce a higher quality output, but in the systems space nothing has come along that provides enough benefit for a low enough adoption cost for it to be a worthwhile choice over C and C++.
Rust shows some promise of being that language, and so we hold out hope and enjoy reading about it.
To give a somewhat different reason than what I'm seeing, I don't care too much about the 'memory safety plus speed' thing. I'm often building services that would be "fast enough" in Java. Is it nice to be faster? Yes, definitely, and that's part of why I write Rust, but not the whole story by any means.
I like rust for a few reasons.
* I really like Rust developers - I've met many, talked with many, and we get along and they've helped me through many technical problems, both big and small.
* I found rust extremely easy to get started with. I knew I'd have to learn the language, I did not also want to learn build tooling, package management, etc. These barriers exist in many other languages, it's frustrating because I just want to focus on programming. Cargo is excellent and I am a huge fan of how rust manages crates.
* I find it very easy to express what I want in rust, to reason about the code, etc. I know when a function might error, I know when I need to handle some unforseen case, I can express how a variable is used, when it is usable, whether it can be mutated. I can cut down on boilerplate with generics and macros.
The last point is what keeps me coming back. I find rust extremely easy to write, because it makes the things I care about easy to express.
I find it interesting because of the novel approach to memory management. It is deterministic and zero-overhead like C, C++ and similar, while at the same time safe like garbage collected languages. I think of it like a compile-time garbage collection.
I wouldn't use it as an alternative to garbage collected languages since garbage collection is just simpler overall, but I would consider using it as a safer alternative to C or C++.
I wouldn't use it as an alternative to garbage collected languages since garbage collection is just simpler overall,
I do not disagree, but I think one could argue to the contrary as well.
Since most garbage collected languages do not guarantee that objects are actually (timely) garbage collected, you cannot tie the lifetime of other resources (file descriptors, sockets, locks) to object lifetimes. So, the burden is on the programmer to ensure that resources are correctly finalized. Whereas in RAII languages you can properly tie all finalization to object lifetimes.
Note: I am not arguing that GC-ed languages cannot do RAII. AFAIR D has a GC and supports RAII.
This is true - for example, C# has the `using` block to ensure resources are deterministically released when they go out of scope, not when they at some point later are garbage collected. I would still argue C# is much simpler than Rust since for most object you don't have to worry about this. But I guess there is a certain extra complexity in C# because you actually have to know if a class owns some external resource or not, while this distinction is not necessary in Rust.
In practice that's rarely an issue, because a) if it has a resource it'll implement IDisposable and b) my IDE screams when I don't dispose an IDisposable.
This is not as good as the Rust model, but it's not awful, either.
Yes but that is just the compiler offloading some of the static analysis to the IDE. The difference might not matter if you are always in an IDE anyway, but the language would be safer if this was an actual compiler error.
In most modern languages, there is some way to tie object cleanup to a scope, even for a GC'd value. Note I say "cleanup", as in "I closed the file handle" or something, not "finalization" which is a GC-specific term. It isn't necessarily as rigorous as RAII but it works in principle in much the same way. "with" in Python, for instance, "defer" in Go (less automatic but if used properly fits the 80/20 nature of Go), Haskell has a motley crew of solutions, most other languages you can put something together yourself if nothing is provided.
Where things get complicated is when you don't have a clear scope to tie lifetime to and so you can't use these, but then, that applies to RAII too.
I'm not saying C++ doesn't have a bit of an advantage here, but I think it often gets oversold as "C++ has RAII and other languages have nothing even remotely resembling it", which isn't true.
In practice this isn't a problem that I encounter in GC'd languages anywhere near often enough to justify even a slight preference for a "true RAII" language.
It isn't necessarily as rigorous as RAII but it works in principle in much the same way.
I would say that in principle they work very differently ;).
They work superficially in the same way in that if you tie a particular object to the current scope in a RAII language, the cleanup happens at the same point as defer, with, try-with-resource, etc. would. However, there are cases where you really want cleanup to be tied to the object's lifetime.
For example, I have a Tensorflow binding for Go. However, I cannot pass Go-allocated memory (e.g. memory allocated to a slice), because Go does not allocate slice memory on 32-byte boundaries (using Go memory would cause an allocation + memcpy in Tensorflow to align the memory). So, you allocate memory in C-land and have Go structs that wrap your pointers. However, now cleanup becomes interesting. You do not want to rely on finalizers, since they are not guaranteed to run. However, using a Close method is also an annoyance. For tensors that live for the duration of a graph run, it is fine (you can use defer), but other tensors live longer and are reused between runs, shared by models, etc. It becomes unclear pretty quickly who is responsible for closing the model.
I also use a Tensorflow binding for Rust, which is drastically more convenient in this respect. Since ownership is clear, the lifetime of a tensor is bound to the scope or object that owns it. If the owner is dropped, the tensor is also dropped. If you need to share a tensor, you make an Rc/Arc the owner.
Well, to be honest, my position has consistently been that using Go for scientific programming is not a good idea, and will never be a good idea because the Go team is never going to give you the features you need for it, and that you probably should just use Rust. (Or something else. Rust is not the only choice.) The Go answer is probably that, yes, you need a .Close method, and yes, I agree that in this use case that's really annoying and I would suggest this argues against using Go for this.
People tend to then get annoyed at me for expressing this opinion, but this sort of thing is the reason why. Go is really only merely adequate at interfacing with libraries in other languages [1] which scientific programming does a lot of, and Go has a type system that seems almost precisely tuned to get in your way if you try to program mathematical code in a typed manner but at the same time isn't so weak that you can pull something like a NumPy where at the Python level everything is just untyped so as long as you assemble it correctly up there, the C level can work it all out. Nor can you practically program in that manner, because while you can slather interface{} everywhere, you can't make it convenient to work with like a dynamically-typed language. I think Go is approaching maximally pessimal for scientific-type programming, personally.
I should clarify that when I say Go has "something like" RAII, I do mean pure-Go code only. And by no means is "defer" perfect. (I'm definitely in the camp that it should have been block scoped, not function scoped, and the performance hit can be quite annoying.) It's just that, as I said, it's not like the choice is "either RAII or you're in some manual-management only horrorland"... lots of languages have block-scoped constructs (not just Go) that can be used to 80/20 RAII. That last 20 may be important in some cases, but it's quite often a great deal less important than the 80.
(This post is brought to you by your friendly local "HN poster who has been accused of being unreasonably positive about Go".)
[1] Pretty much every modern language claims to have "great" interfacing with C, despite IMHO wild variances in difficulty. Go is "adequate" because it's not too difficult to simply call a C function, and with not much labor you can get binary-level-compatible structs between the two, which is a nice advantage over Python or Perl or something. But the semantic mismatch is pretty rough around memory management and threading model, and that manifests in slowness in the calls in addition to general semantic mismatch.
Rust is better than Go for sure (cough Generics) but it's already hard enough to fight the mathematics, statistics and machine learning, scientists will not want to fight the Rust syntax as well.
For me, the most promising compiled and statically-typed language for scientific computing is Nim.
Disclaimer: I am the author of a Numpy/Torch/Tensorflow-like library written from scratch in pure Nim, the look and feel is pretty similar to Python Pytorch + Keras for neural networks: https://github.com/mratsim/Arraymancer
Well, to be honest, my position has consistently been that using Go for scientific programming is not a good idea, and will never be a good idea because the Go team is never going to give you the features you need for it, and that you probably should just use Rust. (Or something else. Rust is not the only choice.) The Go answer is probably that, yes, you need a .Close method, and yes, I agree that in this use case that's really annoying and I would suggest this argues against using Go for this.
That's a good point. Go was my camping ground while Rust was still breaking every month and I didn't want to go back to C++. Although I don't use Go anymore, I have come to appreciate its simplicity and in a lot of scenarios I would definitely recommend it.
> So, you allocate memory in C-land and have Go structs that wrap your pointers. However, now cleanup becomes interesting. You do not want to rely on finalizers, since they are not guaranteed to run.
I did similar thing with C# more than once. Not with TensorFlow, but with my only C++ libraries that also used SIMD and therefore required aligned memory buffers.
It worked just fine. There’s IDisposable for deterministic cleanup, and finalizers as a safety net. Unmanaged interop, i.e. [DllImport], is supported on all platforms, e.g. on Linux it imports from *.so libraries.
> I would consider using it as a safer alternative to C or C++
I would, too. But there’re problems for which C or C++ is just faster because of language/compiler extensions like SIMD intrinsic or OpenMP.
Also Rust can’t be used for GPU code; strictly speaking C can’t either, but practically CUDA, C++ AMP, OpenCL are very close to C and/or C++.
Also if you have to deal with lots of pointer-based data structures (trees, graphs, etc)., and it’s not just on the lowest level you can abstract away behind a safe API, I don’t think Rust is safer. To get performance comparable to C++, it’s necessary to use unsafe rust for raw pointers, and IMO modern C++ is safer that unsafe Rust.
In Rust, the borrow check is still active even in unsafe blocks. In C++, however, the complex rules around when destructors are called, combined with references, are a constant use-after-free footgun that you can never eliminate.
Rust projects that use unsafe code are empirically safer than C++ projects. See, for example, the Servo style system as used in Firefox.
> the borrow check is still active even in unsafe blocks
When you deal with pointer-based structures (trees, graphs, etc.) you’ll use raw pointers for them. Raw pointers are not borrow checked, and the complete list of pointer-related potential bugs applies to unsafe Rust, you can use after free, double free, mess with pointer arithmetic so you’re out of bounds, etc.
> In C++, however, the complex rules around when destructors are called, combined with references, are a constant use-after-free footgun that you can never eliminate.
C++ ecosystem has lots of stuff that help writing correct code despite unsafe language. Debug builds use special version of heap that fills freed memory with a magic number, this help to catch most UAF bugs very early in the development. There’re runtime tools like valgrind and asan. There’re static analysis tools like PVS studio, clang static analyzer, and coverity.
Unsafe Rust has none of them. Not even a debug heap.
> Rust projects that use unsafe code are empirically safer than C++ projects.
Survival bias: people who need raw pointers and other performance-related features like SIMD and manual RAM layout don’t pick Rust for their projects.
Respectfully, you're missing one of the most important parts of Rust. Rust isn't just about reducing or eliminating the use of `unsafe`, it's the ability to build an API that might use `unsafe` internally but is safe to use externally. You can't do that with C++.
I also think many of your points over generalize. Finite state machines are graphs for example, but I've never needed to use raw pointers to achieve the performance required. Similarly, I definitely need SIMD, and I use Rust for that.
I can, and I do. The trick is to use some other safer language for less performance critical higher level code. C# often works well for me but there’re others, e.g. in data science community Python is quite popular for that role.
> I've never needed to use raw pointers to achieve the performance required.
Very likely, we work on different kinds of projects. One of my ongoing project is a CAD/CAM app for Windows, and I have implemented quite a lot of pointer-based stuff to optimize the performance. Technically they are mostly graphs, logically they’re multi-level hierarchical containers, LRU caches, quad-edge structures, scene trees, external indices, etc.
P.S. Why do you think Mozilla doesn’t use Rust for their DOM tree implementation, and instead relies on the JavaScript runtime and it’s GC? Don’t you think sometimes other people might also want DOM-like structures in their apps, be it for a web page, XML document, objects hierarchy in 3D space, or any other kind of data?
You're just moving the goalposts. Pretending an entire language barrier is the same as the barrier between Rust's safe/unsafe is kind of ridiculous. The friction for use is on a completely different scale, not only in terms of convenience but performance as well. For example, a regular expression library might use unsafe internally, but forcing every use of that library to operate in a higher level language to achieve safety is simply a non-starter.
> Very likely, we work on different kinds of projects.
I work heavily with finite automata. Similar libraries written in C or C++ use pointer based stuff quite heavily, but none of it has so far turned out to be necessary in Rust while also achieving comparable performance.
This continues to miss the point that Rust permits building safe abstractions over unsafe internals. Even if you need to use raw pointers in Rust, you're not only no worse off than you are in C++, but you can actually encapsulate that use in a way that is guaranteed by the type system without resorting to using a completely different programming language.
> Why do you think Mozilla doesn’t use Rust for their DOM tree implementation
I'm not involved with that project, so I wouldn't know, and I wouldn't speculate. You shouldn't either.
> Pretending an entire language barrier is the same as the barrier between Rust's safe/unsafe is kind of ridiculous.
I’m not pretending that. Just two points.
1. If you really need unsafe code, because raw pointers, or other reasons, C++ is safer than unsafe Rust.
2. If your project has two distinct parts, unsafe lower level, and safer higher level, you don’t need to use C++ for both. Real world software use multiple languages in the same project for decades already, e.g. QuakeC was developed in 1996.
> a regular expression library might use unsafe internally
I’m not saying Rust is useless. There are projects where it shines, and where I’d probably picked it myself. Like a web browser CSS engine, or your finite automata, or many other things.
I’m saying that there’re large problem areas out there for which, due to various reasons, other languages are still way better than Rust. I do realize Rust evolves fast; eventually these areas might shrink or disappear. But in its current state, my opinion is the applicability is limited to very narrow areas: no bare metal, no SIMD, no GPU APIs, very limited asynchronous IO, limited embedded options, very limited numeric libraries, no GPGPU…
I’ve only listed the areas where I have recently (last couple of years) developed substantial amount of code in any other language.
It just happened that I didn’t work on either finite automata, nor browser CSS engines.
> If you really need unsafe code, because raw pointers, or other reasons, C++ is safer than unsafe Rust.
I don't agree. I've told you why. I find speaking with you very frustrating, and it's not clear to me that you've actually understood my point unfortunately. :-/
> If a library wants to do that, it probably means that safe language is too slow
> P.S. Why do you think Mozilla doesn’t use Rust for their DOM tree implementation, and instead relies on the JavaScript runtime and it’s GC? Don’t you think sometimes other people might also want DOM-like structures in their apps, be it for a web page, XML document, objects hierarchy in 3D space, or any other kind of data?
Nothing prevents you from writing a DOM tree in Rust, but when in comes to the Web DOM you need to interact with JavaScript and with the JS GC no matter what you do. How is DOM managed in other browsers written in C++ ? I would be surprised if it wasn't also handled by the JavaScript runtime.
Yes and no, a lot of libraries have been built on top of the feature while it was unstable. It’s been in nightly for quite a while at this point.
Of course, more libraries are needed as well, but there’s already some higher level stuff. See https://github.com/AdamNiederer/faster for example. And const generics, coming to nightly near the end of the year, will be another step up. It's true overall that numeric stuff is a weakness, but we'll get there!
I wouldn't say zero-overhead. I would say O(1) overhead in the best case, and equivalent to C-style manual memory management in the typical (which again isn't zero-overhead and isn't even necessarily always O(1) overhead with heavy allocation and deallocation).
But, yes, it isn't GC'd with the attendant challenges GC poses (or benefits it brings).
Because it is has the potential to unseat C as the go to language for deeply embedded code. The safety aspect of Rust is quite compelling for Functional Safety applications in automotive, medical and aerospace domain. I can't wait for it to mature and be mass adopted.
Are there any studies done that looks at the relative amount of bugs in C code compared to Rust? Making the claim that Rust is better for those applications isn't convincing to me until I can see the statistics that backs it up.
I think you'll have a hard time finding conclusive research like this for any languages.
Many papers exist but often have conflicting results, or weird methodologies, or extreme artificiality. It's really expensive/ hard to generate good research on this topic that controls for the big variables - you basically need a company that's willing to throw away at on of developer time to build two versions of the same product in both languages, then a metric for measuring success, a way to ensure you take dev skill into account, etc.
So instead it seems a lot more worthwhile to not bother looking for objective research as it won't be found.
I think a better approach is to just use what we have - sense and anecdotes.
From a sensibility standpoint, let's take a step back from rust.
Do we expect fewer memory safety vulnerabilities in Java vs C? Probably yes - Java is a memory safe language with few escape hatches.
So what about Rust vs C? It's a bit less clear as rust has a more oft used escape hatch, but I think we can reason that it's likely to have fewer memory safety bugs.
Would we expect fewer bugs in general? That's much harder to discuss with that approach.
We looked at the number of security-critical bugs in one new component of Firefox—Web Audio—a few years ago in order to determine how many of them a memory-safe language would have prevented. The answer was 100%.
Apart from all the other things people are mentioning, I'd add that the development of Rust happens in the open. That means that a lot of very interesting debates are there on GitHub for everyone to read. For example, here's where arielb1 discovered that the original scoped threads design was unsound: https://github.com/rust-lang/rust/issues/24292 (which sparked a large debate about whether mem::forget should be allowed in safe code: https://github.com/rust-lang/rfcs/pull/1066).
The promises about memory safety was what initially made me interested in it. When it turned out that it was actually possible to learn, it was comfortable to write for someone with Python and C/C++ experience like me, and I saw Cargo and that the ecosystem of packages was quickly growing is when I fell in love with it.
Couldn't you achieve the same thing in C++ by using message passing between threads instead of shared memory? That way, you would get safe parallelisation without having to fight the borrow checker in every part of the program. I understand that having safety statically enforced at compile time is a real gain in terms of eliminating that class of bugs. However, the mental cost a Rust programmer pays for that trade (due to borrow checker) seems too high for me if you can get the same thing simply by shifting your programming patterns slightly.
Couldn't you achieve the same thing in C++ by using message passing between threads instead of shared memory?
I know that the parent post used multi-threading as an argument in favor of Rust. But there are many ways to blow of your foot in single threaded C++. For example, consider this single-threaded code:
std::vector<size_t> v;
v.push_back(1);
size_t *first = &v[0];
std::cout << *first << std::endl;
// The STL vector implementation will probably
// reallocate the backing array at least once.
for (size_t i = 2; i < 1000; i++)
v.push_back(i);
std::cout << *first << std::endl;
You wouldn't be able to introduce the same error in Rust. Once you borrow the an element of a Vec immutably, you cannot create mutable borrows during the lifetime of any immutable borrows. So the compiler would reject pushing new elements.
However, the mental cost a Rust programmer pays for that trade (due to borrow checker) seems too high for me
The thing is that most of the errors that the borrow checker will point out are potential programming errors in C or C++ as well. However, current C/C++ compilers will put most of the responsibility completely on you (plus tools like Valgrind). So, to me C++ has at least the same cognitive load if you want to program responsibly.
I agree with the general principle, but programming in Rust is not "program in C++, then fix errors". You really need to lay your code in away that will satisfy the checkers. That or jump into unsafe mode.
A good example of the challenge is that there's unsafe inner code in rusts own container code. If you try to build your own custom containers (or things like arenas) you'll likely work with unsafe code... Or rely on rusts container code (not a bad thing of course).
My feeling is that "C to Rust" is close to "Java to Haskell" more than anything else. The extra layers are important enough that they need a new thought process. (I say this as a proponent of rust)
Same thing is also true in C++, the stdlib also uses "unsafe" pointers inside, but you can use unique, shared etc.
> You really need to lay your code in away that will satisfy the checkers.
There are some patterns that are common but problematic but are being addressed by non-lexical lifetimes. Other than that it's almost "program in C++, then fix errors".
You really need to lay your code in away that will satisfy the checkers.
But that becomes a second nature after a while. I think all the edge cases that still exist make it more problematic. E.g. I was writing some SIMD code yesterday with a loop somewhat like this (simplified):
let mut v = vec![1.0, 2.0, 3.0, 4.0, 5.0, 6.0];
let mut s: &mut [f32] = &mut v;
while s.len() >= 2 {
println!("{:?}", s);
s = &mut s[2..];
}
But in all seriousness, unsafe in container code is somewhat expected, because at some point you need to wrap raw memory (as e.g. in Vec). I think quite a bit of, but probably not all, unsafe use in containers such as Vec comes from that.
> However, the mental cost a Rust programmer pays for that trade (due to borrow checker) seems too high for me if you can get the same thing simply by shifting your programming patterns slightly.
i remember reading about the stylo (multithreaded css styling) engine, something that has been tried times and times again to implement it in C++ without success: too complicated. sooner or later you'll get too-hard concurrency bugs and the project would be abandoned. and it wasn't just mozilla with firefox, google also failed to implement concurrent styling with chrome.
the blog post stated that this time they were successful _only because of rust_. the mental overhead of managing the borrow checker is far lower than the mental overhead of dealing with concurrency issues in a complex high performance project.
I think you misunderstood Rust. There’s no additional mental cost compared to C++. In C++ you must do thing the same way (keep track of references, variable lifetimes) in order to write any sane program. Rust just help you to declare those things and ask the compiler to enforce it. And no, message passing is not the savior to parallelisation.
There is definitely some mental cost. By making lifetimes explicit there are additional barriers in the way of doing certain things that are safe but involve invariants that are too high-level for rust's semantics. For example, taking mutable references to multiple elements of a container simultaneously.
.split_at_mut(), there you go. But I understand that there are cases when Rust rejects correct program, that is where we need unsafe to build safe abstraction.
Come back to additional mental cost, it’s just the same debate between static & dynamic typed language. Either you write more, specify constraints, and have the usage checked or you write less, imply your constraints and must satisfy the constraints of usage by yourself (or with helper tools/tests). To me that’s just “when” I have to spend the mental resource, and the total cost is roughly the same. But then, knowing the separation helps to choose which tool for which type of project.
I don't think this is just about being more or less verbose. It's about the awkward ways it contorts the ways you write programs. For example, in C++ you might write an asynchronous job scheduler with an API like the following:
The consumer of this API in C++ has an easy job. They can create an array of results and iteratively call schedule(jobs[i], &results[i]) for each job, and then call wait(). A similar API in Rust would be a giant pain in the neck to use, because there's no simple way to collect the results. You'd likely want to scrap this API entirely and have Scheduler expose some kind of queue or something.
My point is that these extra constraints are not just extra typing locally to specify how you're using lifetimes. You'll likely have to rethink the way you write APIs and whole programs in Rust because of the ways it restricts mutability. Which is OK, it's a trade-off, all I'm saying is the safety doesn't come for free.
While there might be all kinds of troubles _implementing_ that API, I don't think it wouldn't be that hard to use. You would need to have a valid "not ready" state for the results, because Rust requires that they are in some state at the moment they are constructed. From the lifetime perspective, the data structure where the results live should outlive the schedule calls – but that isn't a problem if you construct the array first. There will be a lifetime requirement but that's it.
The mutability thing isn't a problem. You can allocate a vector of results and jobs, and then have a mutable iterator over them, zip them together and then iterate over that. There's no problem passing mutable references to each of the elements.
The problem is that schedule() needs to mutate result asynchronously at some point in the future. It can't do this if result is a mutable reference, since it needs to return immediately so borrowing a reference doesn't work. It also can't take ownership of result, since it has no way to give it back (short of changing the API, for example by making wait() return a collection of results).
The right way to implement the API above is for schedule() to take Arc<Mutex<Result>>, and explicitly share mutable ownership over each individual result between some unspecified asynchronous execution mechanism and the caller of schedule(). This makes the calling code significantly more complicated as it needs to incur the overhead of atomic reference counting and a mutex per result, and must try to lock each result before using it, even if it knows there are no other owners of the result due to wait() returning.
You're exclusively borrowing res_a, res_b and res_c for the lifetime of the scheduler. Which works fine since res_a, res_b and res_c are hard-coded variables with separate lifetimes. What rust doesn't know how to do is borrow mutable references to elements of a container without borrowing the entire container.
Rust also offers some other methods to allow multiple mutable references to containers: .split_mut() comes to mind – that splits a mutable slice into two mutable slices that don't overlap. In the future I'd like to see in the stdlib "container adapters" that take in a normal container and provide an interface to it that allows taking multiple mutable references while ensuring the soundness using dynamic checks (it can keep an array of the pointers or indexes that are borrowed out, and check against that). I created such an interface as an experiment last year: https://github.com/golddranks/multi_mut
The sibling comment shows how to do that in safe Rust code. But then, even when you have much more complex control that Rust compiler can not prove safety, you can still resort to unsafe, build your algo, and wrap it into a safe API, just like `split_at_mut`
This cuts both ways. Because I can offload checking from my brain to the compiler, I feel personally that the mental cost is actually reduced overall. YMMV.
IMO the cost is worth it for the documentation alone. What lifetime a callee expects me to uphold for the arguments I pass is very often a mystery on which the docs are absolutely silent in C/C++. In Rust, it's part of the type signature and enforced by the compiler, so I can always read it off at a glance. This is a huge win in reading a new API.
> However, the mental cost a Rust programmer pays for that trade (due to borrow checker) seems too high for me
In C++ land, you'd have to use TSan thread sanitizer and a lot of other tooling (ASan, Valgrind, etc) to achieve even close to the same level of confidence. Not only is this a runtime check but all that tooling has a fairly heavy mental cost too.
What I find an even more interesting is that every story about rust have posts whining about hn posts about rust. this is such a boring, ever-present “question” that i don’t actually believe you asked it honestly.
Seems to me that Rust's current point on the popularity curve suggests that the vast majority of programmers probably don't know much at all about the lang, or even that it exists. Busy people can note that there's plenty of HN threads about a thing without investigating further. Until they finally do.
Better to be charitable, I think. Doing so costs us nothing.
Have you tried it? If you have - why do you think it shouldn't be as popular as it is? If not, perhaps in using it you will discover why it is popular.
Maybe I'm getting old and cynical, but the fact that something is both new and popular makes me even more reluctant to try it. The commons seem to be really good at picking mediocre-at-best winners, and fad chasing is really common.
Yes, but "mediocre" is not measurable on an absolute scale. If you're trying to obsolete 50-year old technology, today's mediocre technology is probably good enough.
Rust is a fairly conservative language. It doesn't really do anything novel, it mostly just synthesizes patterns that have existed for years. Hopefully Rust truly is mediocre (at least in hindsight). That would mean the things it is attempting to replace are seen for how primitive they truly are.
These are all tiny, targetted microoptimizations worth a percent or three of benefit in specific tests. They're worth doing (or at least evaluating) in any mature product and I have no complaint.
Nonetheless rustc remains really slow relative to other similar technologies, including C++ compilers. Is there any consensus as to why?
I mean, with C++, the answer is something to the effect of "template expansion happens syntactically and generally has to be expressed in headers, leading to many megabytes of code that has to be compiled repeatedly with every translation unit". And that isn't really amenable to microoptimization. We all agree that it sucks, and probably can't be fixed with the language as it's specified, and chalk it up to a design flaw.
What's the equivalent quip with rustc? I mean... is it going to get faster (in a real sense, not micro), or is it not? Is this fixable or not, and if not why?