If you're fighting, you've lost. The way to convert everyone to Rust you need to be better the the competition. Not just better as in "look at my features that will make your code safer". People may see the value but think "I get on just fine without the borrow checker so it isn't too important". You need to be far better then the replacement by providing the following:
* Great Tooling ( IDEs )
* Great Libraries ( Everything and a kitchen sink )
* Better Documentation
Anything that can be done easily in C or C++ will need to be easier in Rust for everyone to move. No amount of language features will pull people who are doing well at their job, currently building everything they need to, and who maintain low level systems. You have to be able to entierly replace the old systems in a completely feature-complete way that's also easy to migrate to.
Blog posts wont pull me away from C, tooling and docs will.
I don't really get this attitude, it means that nothing new will ever overcome anything that's been established.
> Anything that can be done easily in C or C++ will need to be easier in Rust for everyone to move
You don't even need folks to move from C. Rust has had lots of success when it comes to folks completely new to systems programming learning it through Rust. Predominantly python/ruby/whatever shops are using Rust because they need a fast language, but don't want to deal with safety issues.
> Blog posts wont pull me away from C, tooling and docs will.
Then you are not the right audience for this blog post :) I find that such blog posts are extremely helpful in convincing people who have a choice between starting to use C and starting to use Rust, not folks who are already invested in C or C++. But I have seen such overviews to have impact on invested C/C++ programmers too; everyone is different!
> I don't really get this attitude, it means that nothing new will ever overcome anything that's been established.
I think he's trying to quote Dan Saks' "extern c: Talking to C Programmers about C++" when he said "If you're arguing, you've lost." Saks, at least, meant that if you're striving with someone, you've already lost because your partner is already in a "frame" of mind set against your arguments that is unbreakably strong, even by logic. The talk is really good just for that aspect.
> You don't even need folks to move from C. Rust has had lots of success when it comes to folks completely new to systems programming learning it through Rust. Predominantly python/ruby/whatever shops are using Rust because they need a fast language, but don't want to deal with safety issues.
This resonates well with me. I came from PHP and I decided to learn C in order to contribute to an existing PHP extension. I was doing well following a C tutorial. Until I arrived at string manipulation at which point I said, f* this sh.
So looked for an alternative systems programming language to learn, and found Rust. And its a pleasant language to work on. Its a low level language that feels like a high level language.
My only fear is that, if Rust becomes a popular language, the community will end up as toxic as the JS and PHP community. Right now, the Rust community is filled with the smartest and friendliest devs I know of among open source communities, and I kinda like it to stay that way.
>Right now, the Rust community is filled with the smartest and friendliest devs I know of among open source communities, and I kinda like it to stay that way.
I have yet to find a community that has gotten popular and hasn't turned into a sewage plant. I think it's just a reflection on what the general population is like... :(
The Rust community is as great as it is because of very deliberate decisions and investment in engineering structures to maintain a positive community at scale, so I'm pretty optimistic that it will stay really good. Arguably this work is as innovative as the language itself.
The rust community is very small so it's easy to 'engineer it' at this phase. Once it gets so large that you have an eternal September it's effectively a complex system that will be very difficult to reason about with any changes resulting in lots of gnarly side effects.
Name a large programming community that is pleasant to interact with for newcomers and insiders alike.
One of the distinctive things about Rust community management is how much is literally engineered, and done with software and processes so that it can scale. All of the touch points are designed: the rustup tool manages the installation experience, Cargo the build process and 3rd party dependencies, the RFC process structures online discussion, the This Week in Rust newsletter provides a heartbeat for the community, bots like Bors handle routine interactions, all of the entry points for interacting with other community members display the Code of Conduct, and so on.
It's a level of purposeful design that only the .NET and Go ecosystems approach, and because it's unusual and largely unobtrusive, it's not obvious that this really good, highly productive community is not at all accidental.
I would say that most large programming communities split into multiple communities, so different kinds of folks don't interact as much: the people on the core developer mailing lists are mostly not the same people as on IRC, or on user mailing lists, or working on issues for 3rd party frameworks in GitHub. I can't think of a sub-community that I've had experience with that I would characterize as unpleasant.
"My only fear is..." - this is why I'm in the process of moving to Rust after 3 years of Go. I've come to the conclusion that a low barrier to entry leads to a bad state of affairs. Rust has a learning curve that I hope will help it avoid devolving into chaos.
I'm currently trying to learn Rust, but if my motivation were "I need a fast language and don't want to deal with safety issues", I'd use Java. It's plenty fast (especially compared to something like Ruby!), the ecosystem is huge and tooling is extremely mature.
> if my motivation were "I need a fast language and don't want to deal with safety issues", I'd use Java. It's plenty fast
Context is pretty important to the original comment:
> python/ruby/whatever shops are using Rust because they need a fast language, but don't want to deal with safety issues.
they're not looking for a fast language in isolation, they're looking for a fast language in the broader context of their existing technological stacks. Java doesn't help that much when you're looking into ways to speed up your Ruby application, at least not easily.
Furthermore, Java has the issues that it's only fast when memory is plentiful and it has a long startup/running time, it's "eventually fast" if you will.
Java's FFI story is not as good as rust's though. So if you're interacting with the OS / native libs a lot then it might not be a good choice.
Although there's JNR and JNA and it might become a standard feature at some point in the future[0]. But for a good FFI experience you also need arrays-of-structs, which means you need value types in java. So it's a long journey to get there.
Java is pretty popular for HFT applications. I think most programs are less latency sensitive than that. But of course, Java is not for you if you have hard real time constraints.
You can also use the Java Unsafe class to squeeze performance (every last bit) by recycling objects. I'm sad to say that I've had to do that at least once to make some vector math work.
> I think most programs are less latency sensitive than that.
Hardly, compared to anything that takes user input and aims at 60/120 FPS (where you inevitably end up bypassing GC if you can't force it to run when you want to).
> One thing about HFT is that the speed of the language oddly doesn't matter much. You are talking about latencies of 20 milliseconds, and any language can do 20 milliseconds since CPU times are measured in nano-seconds rather than milisecond.
and
> If you are playing in the sub 10 microsecond range doing everything in C++ helps a lot. [..] A lot of high frequency strategies run in the 20-30 microsecond time frame. I think this is where a lot of diversity in terms of setup occurs.
and
> To lower execution costs, we need a low latency system that has good determinism in response times. You cant pause for hash table rebucketing or garbage collectors or the OS taking away your time slice or moving you to a diferent processor or whatever. There are guys who try this in Java, and spend most of their time trying to make Java work deterministically. Good luck
and
> If you are in ultra HFT then C++ is the only answer (besides custom hardware - FPGA & ASIC). You just can't afford garbage collector.
Java's bad safety reputation comes mostly from the Java plugin, where many inventive ways to bypass its sandbox have been discovered. When used as a normal programming language, without depending on it for sandboxing (that is, when all code running in the JVM is trusted), its safety is quite good.
(Java also has a not so great reputation about speed. That one also comes from the Java plugin, which often took minutes to start up, while keeping the Netscape browser UI locked up.)
... and just to add to that, if you look at the other popular sandboxed client-side runtimes, i.e., Adobe Flash and various JavaScript-enabled browsers (all of them implemented in C++), you will find a similar number of critical vulnerabilities.
No sum types (the Java alternative to Rust enums is the horribly verbose visitor pattern, so a lot of the time people don't bother). null. A verbose and unabstractable "checked exception" system (that the standard library uses poorly in places) that leaves many users resorting to unchecked exceptions (panics) instead. Uninitialized field problems because methods are virtual even when called from superclass constructors. Excessive overhead to custom types that leads to using weakly typed Strings etc. rather than domain types. Poor abstraction facilities that lead to a culture of "magic" annotations and runtime reflection (or worse, proxies and bytecode manipulation) e.g. @Transactional, @Inject.
Java is safer than C, but it's a long way behind a modern functional language like Rust.
You can't say NPEs are an example that Java is unsafe. The propegation is Nulls isn't Java's fault and Java's evolving currently to make use of Optionals as we speak to attempt to avoid this.
Do people choose languages based on blog posts? I've never done that, usually it was a combination of literature and mailing lists that convinced me. I also ignore conferences entirely.
I would love examples of writing about rust that didn't mention or assume any C/C++ knowledge before. So much of what I've run into assumes that knowledge.
I'd love to read tutorials or books that assume Rust is someone's first programming language, period. What would that look like?
I mean by that logic, no progress is possible at all in programming languages, because at some point every language had worse tooling than the competition.
I'd say the greatest part about Rust is the community, which realizes all of the issues you've listed, and especially the really tough learning curve for beginners.
Generally, use the right tool for the job. I don't think I'd really want to use Rust in production yet, but it's really great for side-projects and other more experimental things.
> I don't think I'd really want to use Rust in production yet...
Why not out of curiosity? I've already been showing off to people at work how much shorter Rust code is than our corresponding Java, how much easier it is to build and deploy, and how much more stable it is.
Granted I've been using it at small micro service scale at the moment, but I see no reason not to go into production at the moment (well, except for the fact that I'm the only one that will be called at night if it fails... luckily it hasn't, and I'm not worried).
Well, I can't speak for tooling or productivity comparison to C/C++, but Rust has one of the best programming language documentations I have ever had the pleasure of reading [1].
I'm enjoying the new write-up on ownership/references/borrowing! It reads more like an fascinating reading on PL concept than a language-specific manual. Great work!
Count me excited. I've always liked the book, but I've also found it fairly lacking in ways I'm not sure I completely understand, I can likely lend part of that to my relative inexperience with Rust and systems programming in general.
One I can note is that I really like the new sections. I've never quite liked that the current book is just a bunch of chapters all in a row. Not that that's necessarily bad in and of itself, but I do think the new way is better.
Both would be ideal, if you have the time. The new book is far enough along that you can learn the most basic stuff from it, and the intermediate bits are coming along. The more advanced stuff is only
an outline. So if you start with new, and then switch to old, you'll get the best of both.
I believe I reported this once, but it's still there: You're hijacking alt-cmd-left/right, which I use for switching tabs in Safari. Your arrow-key event handler needs to have a guard:
Yeah I'm not sure mdbook has patched it yet. I'll make sure a ticket is filed tomorrow, I thought we did, but am not sure. Even if we did, well, that's open source.
Rust has by far the best tooling and documentation support out of all languages I've seen. Library support is great too, at least in that Cargo is an amazing platform and it's too easy to import C libraries.
What do you feel is actually missing from Rust? Nothing has stopped me from replacing C entirely on the low end, and even the high end for application software development.
> Rust has by far the best tooling and documentation support out of all languages I've seen.
IDE support is easily better for java. IDEs have total type knowledge since they directly integrate with the compiler, including in long chains of type-inferred code, usually even while you're in the middle of typing and your code is in a syntactically invalid intermediate state. They also automatically manage imports where needed.
Similarly, IDE-integrated debuggers can do a edit-code, recompile, hot patch into running application while you're stepping through the code.
CPU Profilers seem about en par, but memory profilers in java are again better because they offer a combination of taking heap dumps, allocation recording and diffing dumps at the same time. While also offering good GUIs for exploring object graphs.
Like you I need something like NumPy and SciPy for my work. I'd also like an IDE. An IDE goes a long way to helping me feel comfortable to use a language.
Rust's had great IDE support for more than a year now.
- There's Atom with Tokamak, which you must also install racer and clippy via cargo to get rapid in-line linting.
- Then there's Visual Studio Code with RustyCode which you can also integrate with racer and clippy, that provides faster code completion and hovering over items will show a tooltip that documents that items.
- Some people like IntelliJ Rust, but I've not tried it myself.
As for installing racer and clippy, it's been made a lot easier recently via rustup:
rustup component add rust-src
rustup component add rust-docs
rustup toolchain install nightly
rustup run nightly cargo install clippy
rustup run nightly cargo install racer
Now you're good to go.
As for NumPy, there's official crates like the `num` family, which provides a plethora of useful numerics capabilities. It's not my forte though so others would know more about the best numerics crates outside of the `num` family. If you know what functionality you are looking for, it may already be created and is searchable at Crates.io.
I don't really consider this to be "great IDE support", I consider this to be a start at IDE support. I love Rust, and I'm productive even without racer or YCM -- but I think there's a large gap between what Rust has and what many IDEs get you.
Most Java/C# IDEs will get you:
- Type-based autocomplete (racer/YCM/etc have this for Rust)
- Jump-to-definition / documentation (YCM/etc has this)
- Autoimport
- Non-grep-based refactoring
- Auto-boilerplating: for example, type `impl Trait for MyType` and have it tab-complete to a skeleton trait impl
- Some error integration (not just fileline jump-to), with tooltips and stuff asking you how you want the IDE to auto-fix the error
- A bunch of other smaller useful things which I can't remember at the moment
Now, autocomplete is a major chunk of this, but nowhere near being all of it.
The way I see it there are two camps on this issue. There's the "text editor" camp who use vim/emacs/sublime and when they are asking for IDE support they just mean autocomplete and jump-to that works from their editor. Then there's the camp coming from Eclipse or VS who want the whole deal.
The Rust Language Server project (https://github.com/jonathandturner/rls) is working on exposing this info from the compiler in a structured form so that IDEs and "text editor"s alike can get IDE features.
I've been using this plugin but it doesn't seem to do auto-import. Does it definitely support that? Apart from that, as you say, type detection isn't always particularly useful. Apart from those two features, it's pretty much there, definitely usable.
Agree that Rust lacks something comparable to NumPy for numeric work.
Rust does have lots of numeric crates. Too many. I took a look at matrix multiply functions recently.(See [1], below "Here is what the Rust compiler actually does", for some notes on the effectiveness of Rust's subscript checking optimization.) "algebloat" wouldn't compile on stable. "matrixmultiply" is all unsafe code, with C-type raw pointers. "ndarray" has unsafe indexing. "scirust" has more raw pointer manipulation. "matrices" is an empty project.
In the "num" crate, there are bigints, along with rationals and complex numbers, but no matrices. "numeric" has matrices, but it's just a wrapper for some common C libraries. (Also, calling "panic" for a singular matrix in SVD is kind of drastic.[2])
Each of these matrix crates has its own representation for multidimensional arrays. You can't mix them easily.
Matrices in Rust really need some standardization.
We've had this discussion before. ndarray exposes both a safe and unsafe API for indexing, just like the stdlib vector (or core arrays). That is not problematic.
I agree that the lack of standardization is a problem, though. I think that mostly folks use ndarray or nalgebra (and num if they need bigints). Anyone can upload a crate, the question is if the crate is the main one used by the community. There are some efforts to make it easier to choose crates amongst alternatives on crates.io, however.
I suspect most of the issue here is just that no group (only individuals) is using Rust for scientific computing yet, so there's no large driving effort behind getting good libraries here. When I needed this in Rust I just picked the library that I thought would work best with very little thought, with no thoughts on writing my own or improving the lib because I didn't have time. When larger groups work on things generally attention gets paid to stuff like this and better solutions come out on top. Rust isn't quite there yet in adoption for this to happen, maybe soon :)
Is it really necessary to expose an unsafe API that bypasses subscript checking? Another Rust advocate was arguing, the last time this came up, that LLVM could optimize out most of the subscript checks. As I showed, it optimizes out about half of them for multidimensional arrays implemented with an explicit multiply. That may improve, especially if there's some standard idiom for declaring multidimensional arrays and the compiler handles that idiom well.
For 1D, though, the optimization of checking is pretty good. "get_unchecked" for Vec may be obsolete. There's an amusing Stack Overflow question [1] from someone who complains that he changed an access from "[]" to "get_unchecked()" and his program didn't get faster.
Maybe it's time to deprecate some of the legacy "unsafe" stuff. Preferably before the first CERT advisory involving a buffer overflow in a Rust program.
> Is it really necessary to expose an unsafe API that bypasses subscript checking?
Sometimes you don't want to rely on LLVM, and sometimes the size invariants aren't as easily available to LLVM.
I'm not even sure if this would be that necessary outside of the library itself; unchecked indexing would be useful to implement ops like matrix multiplication within the library, once, and probably never used after that.
The API exists mostly to mirror the stdlib one and its use cases. I don't think it's supposed to be the API you reach for usually. Just because the API exists doesn't mean it's the one you're supposed to reach for.
> That may improve, especially if there's some standard idiom for declaring multidimensional arrays and the compiler handles that idiom well.
I mean, multidimensional arrays do exist in the language. I'm not sure what you're getting at here.
These libraries provide support for treating multidimensional arrays as matrices, with the semantics of matrices (multiplication, addition, etc). You seem to be arguing for making this a first-class compiler feature? I'm not sure how that's very different from just implementing it with some unsafe code -- the bug-prone-ness of the unsafe code is about the same as the bug-prone-ness of a compiler optimization that replicates it. There is a case to be made for moving it into the stdlib itself to avoid the "10 ways to do it" issue, but Rust wants to keep the stdlib small so that's not going to happen.
This is the point of unsafe code. You use it to implement a few safe abstractions like matrix multiplication and iteration in libraries, verify that, and use them. There's a stigma attached to unsafe code; rightly so; because there be nasal demons. But we shouldn't take it to its extreme and conclude that all unsafe code is bad.
> "get_unchecked" for Vec may be obsolete.
Wouldn't count on it. A single datapoint doesn't really mean much here.
I mostly see get_unchecked being used in the context of other unsafe code. Rust's stdlib uses it for the internals of CString and for implementing iterators. It also uses it in an optimization for the rand crate; which does some trickyness with bitmask indexing that LLVM might optimize but I suspect the authors didn't want to rely on LLVM optimizing.
Servo doesn't use it at all, though. Going through my cargo cache dir the crates that use it are those which implement abstractions. Some, but not all, of the use cases there probably get optimized out anyway. The regex stuff also uses it but the invariants there are complicated enough that llvm won't optimize it (there are some decent comments listing out these invariants and explaining exactly how to use it safely, though).
That's mostly a library for small vectors for 3D graphics and such. (I once did one of those myself, for C++.[1]) There's some support for 2D matrices in "nalgebra", but with heavy use of "unsafe".[2]
Every matrix package I've seen so far in Rust turns off subscript checking with unsafe code.
Is there a problem with doing that with unsafe code? You have to remember that unsafe doesn't mean unsafe in Rust. If a developer is choosing to use the unsafe keyword, it's merely telling the compiler that they know what they are doing and what they are doing is safe. One shouldn't automatically infer that unsafe code is bad or negative. It doesn't deserve the negative stigmatism that it's given.
No, it means the developer thinks they know that they are doing. Often, they don't. Read some CERT advisories for buffer overflows. There are hundreds of them.
I've still not been able to get Rust to work with work in IntellJ or Eclipse. I've not tried Atom but that doesn't support "Projects" or "Solutions" in the way I'd like. From what I understand there is no "go" button.
Atom does support Projects. The left panel is where you add your projects, which is created via cargo and has integration within Atom. You can hide the projects panel by pressing Ctrl + '\'. There's an integrated terminal if you install the tokamak terminal. I'm not sure what you mean by a 'go' button though.
I don't care about the backend and how it's implemented. I don't write IDEs. I write software in IDEs. If the IDE is good, and by good I mean provide autocomplete and features on par with Eclipse for Java,and it needs to be fast and easy to use.
Fast and easy to use are UI tricks that can't be handled by a server.
Edit: I just realized that this may be read in a negative connotion and I didn't mean it like that. I just mean that I'm not the person who should be looking at those. I just know it doesn't exist yet and I'd like it to. Telling me of the Rust Server thing gives me no information as to how close the IDE is to being done.
The only thing the RLS is going to provide is just faster completion for the most part. All the IDE's in existence right now are making use of racer for completion and clippy for linting quite well without it, especially Atom.
If I look at everything I used to write in C, I'd say 80% is well suited for Go and the rest I would fallthrough to Rust for. For the stuff where having a GC and slightly less control is OK, I don't see why I would want to use Rust. Rust is just much more complex and I prefer to keep it simple stupid (KISS).
Basically, Go is good for 90% of what I used to use Java for and 80% of what I used to use C for... trying to understand where it makes sense for Rust to fit in.
I don't use Rust just because I want to avoid GC. I also use it for algebraic data types, compile time elimination of data races, sophisticated polymorphism, a clear and simple module system, excellent tooling in the form of Cargo and an unrelenting focus on providing abstractions with as little overhead as possible.
(I've used Go and Rust daily for the past few years. I love them both.)
Good point. I agree there are other reasons to use Go, I what mean is that unless I have a project that gets a lot of bang for the buck out of those, the KISSness of Go wins out over those nice features and their correlated complexity. GC was just at the forefront of that list of features.
Go is against the sort of programming that builds up conceptual dream worlds of gratuitous abstraction and needless complexity. When that's the only way you want to think, Go will indeed seem anti-intellectual.
A modern programming language without any whatsoever support for generics from my point of view is just extremely bad, for someone else can very well be proof of anti-intellectualism.
Hmm, not sure I understand? Re-reading, perhaps my phrasing wasn't clear. What I meant was that I use Rust, and it's not simply because it lacks GC. There are lots of other good reasons too.
To be even clearer: I don't think Rust's value proposition depends on whether you absolutely must avoid GC or not.
This is a misleading comment. A Rust program may use reference counting by explicitly wrapping some value in Rc<> or Arc<>; the standard library provides these generic types. There are no "constructs" built in to the language or standard library that require the use of reference counting.
I wrote it's like shared_ptr which is also construct in C++ standard library, what is misleading here? You don't need to use it also in C++ and I did not wrote that you need to use it in Rust also, you disagree with something that I didn't wrote.
There is only one implementation of Rust, and it does not have tracing GC. The language does not include semantics for one, so it would be an extension of the language.
For me this is pretty easy. It's about leadership.
The leaders of Go foster an attitude of exclusiveness (just like a month ago they wanted to get rid of the Go subreddit in favor of a solely Google owned option of Google groups).
The leaders of Rust are very receptive and helpful to new people. They are on IRC / Reddit and many other channels.
I'd much rather invest my time into a truly open language and to me, that is not Go.
> just like a month ago they wanted to get rid of the Go subreddit
To be fair, this was a proposal by a single person on the Go mailing list, and it was in reaction to Reddit's CEO publicly admitting to editing other users' comments. The person who proposed closing the Go subreddit was also under the mistaken impression that the subreddit was hosted by the Go team, which wasn't the case. In the end, there was a lot of discussion, and nothing was deleted. Tempest in a teapot, as usual.
Go has a serious culture problem, but that's not a good example of it.
I don't want to single out anyone or any specific projects, so this is going to be a generalization, but since you asked:
In my experience, there's a lot of hostility and arrogance. More than in any other community, I've seen Go developers shut down discussion by closing comment threads on Github; reject pull requests and refuse to debate the merits of the change; castigate people for "not following proper procedure"; and being dismissive or contemptuous instead of humble when they fail to understand a problem that is being discussed (on, say, the official Go Slack channel).
My very personal hypothesis is that this is a case of mirroring. The Go development team may be said to have a laconic, authoritarian style, which is, of course, their prerogative, and which befits their position; unfortunately, a lot of Go developers seem to be under the impression that they, too, can behave like demigods. In particular, Go developers, more than other cultures, seem to get an ego boost out of telling people "no".
But this is a generalization. There are lots of friendly Go devs around, to be sure (I hope I'm one of them). I'm particularly happy with the Kubernetes team. At the same time, I do think it's an issue and one that the community needs to be aware of.
This is very misleading. There is no official Go slack channel. The slack channel you are referring is just a slack channel and is by no means official.
> In particular, Go developers, more than other cultures, seem to get an ego boost out of telling people "no".
This is misleading as well. Go was designed from the start to exclude certain features e.g. inheritance and many others. There are usually very good reasons for those decisions that people who've been following Go from its creation know and understand very well.
Now what happens when someone who does not understand those design choices comes to Go? They want their favorite feature obviously! When Go members attempt to explain them why something like that does not exist in Go (and probably will never be included) it starts feeling like "no". And people do not take "no" well. They get emotional and fail to see reason. If they had bothered to explore the language or at least read the official documentation and FAQ, that state of mind might have been avoided.
Including everyone's favorite feature does not make a language better.
Go has created a community around a certain school of programming. Nobody claims that it is better than other schools but it is a fact that it exists. But who is the one that is close-minded in this case? The new person that doesn't bother with the teachings of the school or the school that dismisses the ideas of the new person? Who is really the one that says "no"?
From my experience, the Go developers always carefully consider every new idea that is brought to the table. But it is also a fact that after the 10th time you've seen the same idea, you are not going to sit down and spend time considering it. You are just gonna link a previous discussion and say "Sorry this has been brought up before, please check these". How does that feel to a new person? I bet it feels like "no" again.
I find it pretty terrible that Kubernetes develops on Slack/Github/Videochats. For being an Open Source project, it rejects open source tooling and makes it difficult for people with poor english skills to participate (text-based meetings are much better for inclusiveness). Even if you do listen in on the video meetings, they give you a feeling as an outsider that many of the decisions have already been made elsewhere (some obscure github issue conversation, a google doc comment on a 1 year old doc, etc). It seems the only way to meaningfully participate in the k8s community is to be a known insider... :(
Kubernetes is opensource-by-Google. It's suprisingly open, and still alive, which is better than almost any of the other Google known projects. (Chrome and Android come to mind, but Android is write-only, and Chrome isn't that great at listening to users either. Not that Mozilla is better about Firefox stuff, but the Rust team is wonderful.)
I only saw one Go team member speak in favor of deleting /r/golang, in order to dissociate the project from what he viewed as deplorable actions by reddit's CEO. I thought the proposal was rash but I don't believe the purpose was to herd people onto Google properties.
In Go I can only write libraries for Go programs. In Rust I can write libraries for any program.
i.e. Rust can easily produce static and dynamic libraries that are linkable with C programs and any language with a C FFI. I can write Rust code that works for programmers using C, C++, C#, D, Go, Swift, Python, PHP, Java, etc.
This is only because the Rust implementations are using particularly slow code paths, either because SIMD/AVX optimizations requires a nightly compiler, some optimizations would require unsafe code, or that other languages are using particularly hacky code that would never fly in real world software.
For example, many of the Java/C/C++ benchmarks are using custom optimizations that should be illegal for the benchmarking. Case in point, some are featuring custom hash maps that feature hashing algorithms that, while fast, would never be useful as they provide no protection against collisions. You'll see a hashing algorithm in a C preprocessor, for example, that just fakes having an actual algorithm whereas Rust examples are sticking to the tried and tested production-grade algorithms shipping in the standard library.
> Case in point, some are featuring custom hash maps that feature hashing algorithms that, while fast, would never be useful as they provide no protection against collisions.
They are useful in this case, aren't they? Custom hash map implementation can be also useful in other scenarios. This is feature of the language that allow you to do that so it's not "illegal" to use it. If Rust doesn't allow you to do something while other language does, it doesn't mean it should be illegal.
I don't see any tricks in Go and it's faster than Rust and more memory efficient in almost half of the benchmarks. So what's your excuse for those cases ?
I believe that what your OP mmstick might have been saying is that the hashing algorithm used in the C version of the algorithm might be very different from the hashing algorithm in the Rust version, and this difference might be significant.
I don't speak Rust, so I can't quite tell what's going on here:
Again, I don't speak Rust so hopefully someone can comment on this, but I don't see that same simple hash is being used in the Rust...I think it might be using a default hash function, which would probably be more collision resistant.
I am sure that the Rust could be made to do the same thing, but I am not sure that, as-is, this is an apples to apples comparison.
I have wondered the same for a while now, and I hope someone familiar with Rust can explain it: in many of the benchmarks Go is faster and/or uses less memory despite the GC. Like you, I don't see much trickiness in the Go code.
In many cases a garbage collected language will be faster when it comes to allocations than using naive allocation strategies (e.g. reallocating a hashmap many times as it grows instead of reserving memory upfront). This is because the garbage collector tends to have preallocated memory lying around, while the RAII needs to call malloc and free every time a heap object is created.
Another factor could also be that the GC in the Go examples just never runs since it doesn't allocate enough memory. It's hard to say exactly what's happening without tracing the runtime, which would also affect the benchmarks a bit.
It's important to note that while both languages are natively compiled (and I suspect Rust would inch out in that category due to LLVM's backing) most of the overhead would probably be from memory, whether allocations, or cache usage, which makes comparing them in microbenchmarks a little inaccurate.
> This is because the garbage collector tends to have preallocated memory lying around,
This is an allocator feature independent of garbage collection; jemalloc does this for example. GCd languages have the potential to do this better since they can do additional analysis whilst tracing for little additional cost, but non-GCd languages can still benefit from this.
Part of this is also that Go is really good at not allocating from the heap when it can allocate from the stack (escape analysis). Until Go 1.5 the Go garbage collector was pretty weak, but it didn't matter as much as it would have for a language with heavier heap allocation.
Wasn't there some issues around a "custom library" for this? I distinctly remember there being some kind of argument about what is legal and what isn't.
That is, I think I'm thinking of this:
> k-nucleotide will explicitly require built-in / library HashMap.
Since C doesn't have a standard library hashmap, you can write an entirely custom one just for the benchmark. But since Rust has a built-in hashmap in the standard library, you cannot. Even though you could write Rust code that's the same as the custom C hashmap.
But they say 'don't write a custom hash table', not 'don't write a custom hash function'. Maybe the problem is that the data in this benchmark is just not a good way to exercise the hash tables in a given language the way the benchmark intended. That probably means the benchmark should be modified; the complaint that some implementations use 'laughably bad' hash functions that seem to be measurably decent hash functions for the data at hand seems really strange.
The "library" distinction you're drawing here is arbitrary. Or at least, my understanding of it is.
Am I allowed to use a custom HashMap for the game, or am I required to use the one in std::collections? My understanding is that it's the latter, and so that penalizes Rust or any other language that includes a map in their standard library.
> If you want an additional Rust hashmap implementation then you need to persuade the Rust community to provide one in std::collections.
Again, this means that for Rust, it _has_ to be in the standard library. But C doesn't have one in the standard library. So C gets to write their own, and Rust doesn't.
If it compiles and run it is allowed. There is no rule to benchmark other than having the same end result with best performance.
I still can't get all the moral over this.
The best thing here for Rust would be to achieve the best performance on this game and nothing else.
To me this game is important and winning on it is more yet. So, if Rust don't have better result by "morals" this is so wrong that hurts. But each time I read these responses I ask myself if there is not a real problem there.
I like Rust, but performance is decisive. It is really sad to see that Rust keeps going down on this game and that makes me question myself when trying to invest time on Rust.
It isn't a "moral" issue. It is about the usefulness of the benchmarks. If you're using them to make a recommendation about which programming language to choose, it is important to have a sense for how well the results generalize to the kind of programs you will be writing. Using unrealistic hacks targeted specifically to each benchmark is contra that goal.
If, on the other hand, you're just treating the benchmarks as a fun competition akin to code golf, then by all means, hacks ahoy!
> The best thing here for Rust would be to achieve the best performance on this game and nothing else.
That's absurd. The benchmarks game doesn't measure real-world performance in any sense whatsoever. And many of the implementations are written in ways that you'd never ever put into production code (for example, the laughably bad hashing functions that are mentioned elsewhere in this thread). From what I understand, the Rust implementations tend to not get up to those shenanigans, which makes them appear worse than the implementations from other languages that do.
It's not the goal of a programming language to be the top on a silly benchmarks game. The goal is to be an excellent language for real-world use.
But people are criticizing C because the way it is implemented is not "realistic". Or that the "hash" is a hack that will never work in real life.
I don't know the background of these people, but they couldn't be more wrong. So that looks like "morals" or just they don't ever saw real C code out there.
You're making an assumption that the C code is allowed to have the same algorithm as the Rust code. This is not actually exactly true, based on the rules of the game.
Which ones do you have in mind? Preprocessor sounds a little cheaty but picking a hash function that's a better fit for the data is a pretty basic, practical sort of optimization.
If that's the one it's a custom hash function which is then inlined as a macro. That still seems like a pretty vanilla C optimization that one might write in real C code.
The problem isn't that it's a macro. It's that it's a horrifically bad hash function. It's blazing fast, but completely unsuitable for use in any real code.
If it's a hash function that only works for this exact data set, I can see the argument. If it's a hash function that works for this kind of data (these particularly formatted, particularly constrained strings), it's fair play. Which one is it? 'It's not a good general purpose hash function' alone doesn't seem like a valid criticism, especially along with 'the Rust version uses a general purpose hash function'. Nobody said you had to use a general purpose hash function.
Imagine you got a million strings of variable length such that the last 4 bytes are guaranteed to be unique value between 0 and 999999. Lopping off the last four bytes is a perfectly good hash function for that kind of data.
Sure, but it's not a good benchmark. If you can demonstrate that you have blazing fast performance, but only for the exact input data that the benchmark uses, then you haven't demonstrated anything at all beyond an ability to tailor your code to a very specific input. But in the real world, we don't get to write our code for a very specific input (after all, if we knew what the input was ahead of time, we could just pre-calculate the answers instead of publishing the program).
So yeah, if you can come up with a tailored hash algorithm that works correctly and is DoS-immune for all possible input for the program, go ahead and use that tailored hash algorithm. But if your hash algorithm is only correct for the specific input data the benchmark game uses, and would not be collision-resistant under real world conditions, then you really shouldn't use that hash as it does not demonstrate anything useful.
Well, here's the thing. I don't think it's for the exact input, it's for a type of inputs. Custom hash functions for specific types of data are a basic optimization technique and I find it odd you'd even suggest every hash function should be 'DoS-immune'. There's absolutely nothing 'real world' about this unless you think the world consists entirely of hostile inputs. In the real world, people absolutely optimize.
Your argument seems to be that that's not the intent of the benchmark which may be true but it's not clear from the rules provided at all. To me, it looks like the opposite is true - they talk about using a standard hash table and most of those allow user-specified hash functions.
Rust's default hashmap algorithm is DoS-immune for any kind of input, which is a perfectly logical default algorithm for a language intended to be used in security-critical areas like operating systems, system libraries, web browsers, etc. and promotes memory safety.
That's really great but I don't see how it's related unless your argument is truly 'custom hash functions are bad', in which case I don't really know what else to tell you beside 'that's completely wrong'.
yeah of course, doing simple programs and checking their time is a good benchmark... oh wait..
also real world performance in bigger programs is mostly different, especially when you deal with big heaps.
Btw. this site is extremly bad for benchmarks since it also measure's the startup time of the runtime in java/go/rust.
While you're checking out the performance of Rust and Go benchmark implementations, you can also check out some SaferCPlusPlus benchmarks[1] (and kind of compare them with other languages (transitively via the C++ benchmaks)). I've already suggested that these days a "memory safe" implementation category for the benchmarks would be of interest. But apparently not to the current maintainer of the benchmarks. Anyone else out there got nothing better to do than maintain a benchmark site for memory safe implementations? :)
The binary trees benchmark forks off a large number of threads. Those are OS threads in Rust, and that is probably an inefficient way to compute something if you have more threads than CPUs.
This site explicitly mentions that results of benchmarks mean almost nothing. Google published results that tell that golang is slower than java in real applications
The ownership system isn't strictly about memory management and can make it easier to catch yourself making larger architectural errors, and you can have more confidence in a refactor with Rust than Go. In fact, I'd argue Rust leads to simpler architectures that fit well into the ownership model as opposed the the "ad-hoc" architectures programs written in other languages seem to invariably turn into.
I've literally just started learning Rust after following it for a few years. I wanted a language that was type-safe and produced binaries to simplify deployment. I chose Rust over Go because I wanted a functional language with generics. Go's repetitiveness regarding error handling just put me off.
I've tried learning C/C++ at several times but I just don't have the inclination to have to bother about null-terminating strings, etc in 2016. I don't mind spending a little more time getting something to compile if it prevents silly mistakes.
Having said all that, it's obviously too early for me to say whether I like Rust. I'm picking it up pretty quickly since I know FP thanks to Scala, but I'll see how much time I spend fighting the borrow checker.
It's kind of a shame, because I don't really feel like the languages are used for similar things in practice, so the constant comparisons don't do either of them justice.
I was writing software in Go for a year before I switched to Rust. I've not felt a need to touch Go since. Basically, anything you can do in Go, you can also do in Rust, but Rust will let you do it with higher efficiency and with significantly less lines of code. In the end, it's just easier to write software with Rust than it is Go.
Feature-wise, Rust features generics and functional programming via higher-order functions and iterators, which is something that Go especially lacks in. Go doesn't have nice concepts like `Option`, `Result`, or `Iterator`. That's not something I'd personally want to live without today. The Go method is effectively writing boiler plate code everywhere, which leads to much room for error prone implementations that require more testing.
I haven't felt that Rust was more complex than Go, at least when you're actually writing software in Rust. Rust libraries feature semantic versioning and are automatically downloaded and verified at build time based on the contents of your `Cargo.toml` and `Cargo.lock` files. No importing of Git repositories directly required. Go does not provide an equal on that front.
There are a lot of great libraries out there to bring you extreme performance, simply, such as the bytecount crate, which is just a library that features a single function, a function that counts the occurrence of a specific byte, 32 bytes at a time with AVX, with additional SSE/SIMD implementations depending on what the processor supports.
All there is to truly know about Rust is the borrowing and ownership mechanism and how to implement a custom `Iterator`. If you have a solid understanding of both then you've pretty much mastered all you need to know about Rust.
The borrowing and ownership mechanism can be simplified down to:
- Passing a variable by value will move ownership, dropping the original variable from memory
- Passing a variable by mutable reference will keep the original variable, but allow you to modify the variable.
- You may only borrow a variable mutably once at a time, and you may not immutably borrow while mutably borrowing.
- You may have as many immutable borrows as you want, so long as you aren't modifying that value.
- You may mutably borrow a field in a struct, and then mutably borrow a different field in the same struct simultaneously, so long as you aren't also mutably borrowing the overall struct.
- You can use `Cell` and `RefCell` to allow for mutably modifying an immutable field in a struct.
- You may mutably borrow multiple slices from the same array simultaneously so long as there is no overlap.
- Safe memory practices means that instead of mutably borrowing the same variable in multiple places, you queue the changes to make in a separate location and apply them serially one after another.
Then for the `Iterator` trait, you would know that all traits have required methods, whereby as long as you implement the required methods for your type, you will automatically gain all of the other methods associated with the trait. For the `Iterator` type, you only need to implement the `next` method, and that looks something like so:
enum Token<'a> {
One(&'a [u8]),
Two(&'a [u8])
}
impl<'a> Iterator for DataIterator<'a> {
type Item = Token<'a>;
fn next(&mut self) -> Option<Token<'a>> {
let start = self.read;
for element in self.data.iter().skip(self.read) {
self.read += 1;
// if next value is found then return Some(&value[start..self.read])
}
None
}
}
```
Four of them are related to calling libc/kernel32 functions, which need unsafe to be called. One is due to using a memory map, which needs unsafe to be called. Only two are actual unsafe rust functions.
That's a "real-world" example though, you're asking for a more specific implementation. I can't compare because my Go would be very poor; if you posted a Go implementation, I'd be willing to give this a shot and write a Rust one.
(Mostly out of personal interest, I don't think it really proves anything larger about the two languages.)
None of the unsafe code that Steve linked to had anything to do with searching a file line by line. It has to do with other parts of ripgrep, like determining whether a tty is available or communicating with a Windows console to do coloring. (Hell, ripgrep doesn't even require memory maps, but they are faster in some cases.)
Your benchmark proposal is interesting on the surface, but if done correctly, its speed will come down to highly optimized SIMD routines. For example, in Go, the code to search for a single byte on amd64 is written in Assembly (as it is in glibc too). This means it's probably not a good indicator of language performance overall.
The thing is I don't really care if Go implementation is highly optimized for amd64 like memchr is in C which is also written in assembler and optimized for different platforms. What I care is that simple code written by me is faster without going into C/unsafe code myself. So it's correct, fast, simple and I do not pay with my time to figure out how to make it as fast in Rust. This is the point I am making. Of course this is only one example, but still a proof that what OP wrote is not valid in all cases.
ripgrep is your proof. Go read the blog post Steve linked. If you still don't believe it, read the code. There's not that much of it.
Then compare it with similar tools written in Go, like sift and the platinum searcher.
The problem is, searching a file quickly is not as simple as you want to believe. If you show me a naive line by line approah, I'll show you an approach that is much faster but more complex. Top speed requires work, sometimes algorithmic work, regardless of the language you choose.
> If you show me a naive line by line approah, I'll show you an approach that is much faster but more complex.
Of course, I did it myself many times. But this is NOT the point, I've already wrote it. The point is that I wrote naive approach in both languages and it's a lot faster in Go. Which is a reply to what OP wrote (don't forget where this discussion started). In this case this is the fact and I don't see any reason to fight with facts if we get bias out of the equation. In other cases? I don't know. What I would expect is that naive searching for one string in haystack to be faster than Go in a language that is performance/zero-cost abstractions oriented like Rust but its false in this case. But it doesn't mean it's false in every case. And to be honest writing that "but they have faster implementation in assembler" is not an excuse at all, you can also write your own, especially if it works so well for those languages that have custom asm for specific platforms. In the end average Joe will not care if it's hand written assembler, he will care that his naive solution using standard library without any magic is just faster.
> The point is that I wrote naive approach in both languages and it's a lot faster in Go.
I tried your challenge, and the first data point I uncovered contradicts this. Here is the source code of both programs: https://gist.github.com/anonymous/f01fc324ba8cccd690551caa43... --- The Rust program doesn't use unsafe, doesn't explicitly use C code, is shorter than the Go program, faster in terms of CPU time and uses less memory. I ran the following:
$ /usr/bin/time -v ./lossolo-go /tmp/OpenSubtitles2016.raw.sample.en the
$ /usr/bin/time -v ./target/release/lossolo-rust /tmp/OpenSubtitles2016.raw.sample.en the
Both runs report 6,123,710 matching lines (out of 32,722,372 total lines). The corpus is ~1GB and can be downloaded here (266 MB compressed): http://burntsushi.net/stuff/OpenSubtitles2016.raw.sample.en.... --- My /tmp is a ramdisk, so the file is in cache and I'm therefore not benchmarking disk reads. My CPU is an Intel i7-6900K.
The Go program takes ~6.5 seconds and has a maximum heap usage of 7.7 MB. The Rust program takes ~4.2 seconds and has a maximum heap usage of 6 MB. (As measured by GNU time using `time -v`.)
---
IMO, both programs reflect "naive" solutions. The point of me doing this exercise is to show just how silly this is, because now we're going to optimize these programs, but we'll limit ourselves to smallish perturbations in order to put a reasonable bound on the task.
If I run the Go program through `perf record`, the top hotspot is runtime.mallocgc. Now, I happen to know from experience that Scanner.Text is going to allocate a new string while Scanner.Bytes will not. I also happen to know that the Go standard library `bytes` package recently got a nice optimization that makes bytes.Contains as fast as strings.Contains: https://github.com/golang/go/commit/44f1854c9dc82d8dba415ef1... --- Since reading into a Go `string` doesn't actually do any UTF-8 validation, we don't lose anything by switching to using raw bytes.
Now let's see if we can tweak Rust, which is now twice as slow as the Go program. Running perf, it looks like there's an even split between allocation, searching and UTF-8 validation, with a bit more towards searching. Like the Go program, let's attack allocation. In this case, I happen to know that the `lines` method returns an iterator that yields `String` values, which implies that it's allocating a fresh `String` for every line, just like our Go program was. Can we get rid of that? The BufReader API provides a `read_line` method, which permits the caller to control the `String` allocation. If we use that, our Rust program is tweaked to this: https://gist.github.com/anonymous/a6cf1aa51bf8e26e9dda4c50b0... --- It's not quite as symmetrical as a change as we made to the Go program, but it's pretty straight-forward IMO. Running the same command as above, we now get a time of ~3.3 seconds and a maximum heap usage of 6 MB.
OK, so we're still slower than the Go program. Looking at the profile again, the time now seems split completely between searching and UTF-8 validation. The allocation doesn't show up at all any more.
Is this where you got stuck? The next step from here isn't straight-forward because getting rid of the UTF-8 validation isn't possible to do safely while still using the String/&str search APIs. Notably, Rust's standard library doesn't provide a way to search an `&[u8]` directly using optimized substring search routines. Even if you knew your input was valid UTF-8 before hand, there's no obvious place to insert an unsafe `from_utf8_unchecked` because the BufReader itself is in control of producing the string contents. (You could do this by switching to using `BufReader.read_until` and then transmuting the result into an &str, but that would require unsafe.)
Let's take a leap. Rust's regex library has a little known feature that it can actually search the contents of an &[u8]. Rust's regex library isn't part of the standard library, but it is maintained as an official crate by the Rust project. If you know all of this, then it's possible to tweak the Rust program just a bit more to regain the speed lost by UTF-8 checking: https://gist.github.com/anonymous/bfa42d4f86e03695f3c880aace... --- Running the same command as above once again, we now get a time of ~2.1 seconds and a maximum heap usage of 6.5 MB.
In sum, we've beaten Go in CPU time, but lost the Battle for Memory and the battle for obviousness. Beating Go required noticing the `read_until` API of BufReader and knowing that 1) Rust's regexes are fast and 2) they can search &[u8] directly. It's not entirely unreasonable, but to be fair, I've done this without explicitly using any unsafe or any C code.
None of this process was rocket science. Both the Go and Rust programs were initially significantly sub-optimal because of allocation, but after some light profiling, it was possible to speed up both programs quite a bit.
---
Compared to the naive solution, some of our search tools can be a lot faster. Performing the same query on the same corpus:
The differences between real search tools and our naive solution actually aren't that big here. The reason why is because of your initial requirement that the query match lots of lines. Lots of matches results in a lot of overhead. If we change the query to a more common type of search that produces very few matches (e.g., `Sherlock Holmes`), then our best naive programs drop down to about ~1.4 seconds, but ripgrep drops to about 200 milliseconds.
From here, the next step would be stop parsing lines and start searching the entire buffer directly. (I hope to make even this task very easy by moving some of the searching code inside of ripgrep to an easy to use library.)
---
In sum, your litmus test essentially comes down to these trade offs:
- Rust provides a rich API for its String/&str types, which are guaranteed to be valid UTF-8.
- Rust lacks a rich substring search API in the standard library for Vec<u8>/&[u8] types. Because of this, efficient substring search using only the standard library has an unavoidable UTF-8 validation cost in safe code.
- Go doesn't do any kind of UTF-8 checking and provides mirrored substring search APIs between its `bytes` and `strings` packages.
- The actual performance of searching in both programs probably boils down to optimized SIMD algorithms. Therefore, once you get past the ability to search each line of a file with minimal allocation, you've basically hit a wall that's probably the same in most mainstream languages.
In my opinion, these trade offs strike me as something terribly specific, and it's probably not something that is usefully generalizable. More than that, in the naive case, Rust is doing you a good service by checking that your input is valid UTF-8, which is something that Go doesn't do. I think this could go either way, but I think it's uncontroversial that guaranteeing valid UTF-8 up front like this probably eliminates a few possibly subtle bugs. (I will say that my experience with text encoding in Go has been stellar though.)
Most importantly, both languages at least have a path to writing a very fast program, which is often what most folks end up caring about at the end of the day.
Do you think you could refactor out bytestring-based string manipulation into its own library? Even better would be something that worked for all encodings (using https://github.com/servo/tendril or something)
> Do you think you could refactor out bytestring-based string manipulation into its own library?
IIRC, someone was working on making the Pattern trait work on &[u8], but I'm guessing that work is stalled.
To factor it out into a separate crate means copying the &str substring routines, since there's no way to safely use them on an &[u8] from the standard library. (bluss did that in the `twoway` crate, so we could just use that.)
It does seem like a plausible thing to do, at least until std gets better &[u8] support.
> Even better would be something that worked for all encodings
I suspect the standard practice here is something like "transcode to UTF-8 and then search the UTF-8." (This is what I hope to do with ripgrep.)
woah, amazing comment. I usually just a silent reader on hacker news, but this comment urge me to create an account. I think lossolo just want a flamewar. He already has opinion which you cannot easly change. So any futher discussion after this comment will be pointless.
EDIT: and how can you have time to write this? I just usually close the browser tab when this situation occurs...
> I think lossolo just want a flamewar. He already has opinion which you cannot easly change. So any futher discussion after this comment will be pointless.
My opinion in this case is empirically checked. I am not saying this to start flame war, I am just showing my observations in particular example. I can also say that Rust regex implementation eats Go regex implementation by magnitudes (performance wise) and it will also be true and I am not looking for any flame war in this case also. I am only sharing my experience that I've backed up with proofs (code + perf results) for that particular use case. This is a factual discussion, I don't agree it's pointless.
Your Rust program corresponds to my second Rust program.
Your Go program is not what I would expect. A bufio.Scanner is the idiomatic (and naive) way to read lines in Go. But this is immaterial. Your results are consistent with mine. They aren't different. I just included more analysis and more programs to provide proper context to this challenge of yours.
Seems I can't reply to your other comment, so I'll reply here. How can you say that my naive implementation is not naive? What is not naive about it? It's very naive. It's basically the same naive code that you were writing, but in actual idiomatic Rust with the linting issues fixed.
Using a `lines()` approach is naive because that allocates owned heap-allocated strings. An optimal, non-naive solution would not use heap allocation for buffering lines but use a stack-allocated array. That alone would bring significant speedup versus the line approach.
As for ripgrep, it's a pretty comprehensive application that makes use of SIMD/AVX so it's only natural that it's fast.
Your solution is not naive in my opinion because you set the size of the buffer and use map/filter but ok... Let's check. Your solution is the slowest from all the solutions.
It took 4.6s which only confirms what I wrote on beginning when we started this discussion. Perf counters for your solution in here:
Your Rust example is kind of odd. Did you not see the linting errors in your editor or from the compiler? You can basically boil it down to just two lines of Rust.
As for timing, I'm doubtful that Go is any faster than Rust. I don't have this www.js file so I can't test it on my laptop, but I'm pretty sure you didn't even attempt to do things like enabling LTO, setting O3, disabling jemalloc, and using the musl target. All these things can make significant differences to the performance of the binary that's produced.
I don't know if you noticed but we are discussing naive solutions which I've mentioned couple of times in previous posts. Code you linked is not naive solution. Things you proposed to do are not naive either.
Here's the thing; it is a naive solution. You may not want to accept it because that contradicts your claim of the Go solution being faster, but at that point it becomes a he-said/she-said kind of scenario because I can claim that your Go solution isn't naive as well and have exactly the same "validity" for such a claim as you do.
Anyway this really doesn't matter as his solution is the slowest of all which confirms what I wrote and contradicts what he wrote.
For me it's not naive solution. Do you have any proof it is? Can you mathematically prove that it's naive solution? I know I can't. For everyone naive solution is something different, what I saw in burntsushi reply and what I wrote myself is the closest to what I think are naive solutions.
> like enabling LTO, setting O3, disabling jemalloc, and using the musl target.
And this is for sure not part of naive solution either.
> You may not want to accept it because that contradicts your claim of the Go solution being faster
This is not true. You can find perf numbers of his solution in my second reply to him. Or you can compare those solutions yourself.
> but still a proof that what OP wrote is not valid in all cases.
You are not proving anything until you post your Go code, which you don't seem to have done (please correct me if I'm wrong, I am legitimately curious to see for myself how Go and Rust stack up against each other for this problem). Until then, all you're doing is making vague claims backed up by precisely zero evidence. Why should anyone take you seriously?
I've seen several of your comments indicate "requiring naive implementation". This seems strange to me. Why require a naive implementation and then be concerned over some slight differences in performance?
Yes, and I was saying "even in production-grade, world-class grep implementation, there is barely any unsafe." I also acknowledged that that was different than what you were asking about.
> I told you NO unsafe code. Use only standard library, no third party tools.
I'm not sure what such an example would prove, other than maybe "Go is better than a subset of Rust excluding some of the core features of the language and its greater ecosystem".
This is a tricky requirement. The standard libraries of both languages use a crap ton of unsafe code. You might end up just asking for a comparison of which standard library is bigger.
As someone who writes Go full time, once I'm done with my current big Go project I will be taking a break to investigate alternatives. Both Rust and Swift are at the top of my list.
Go is good, even great, at many things. But it's a language largely defined by its limitations, usually intentionally. It's an engineering language, not made for big abstractions. For me, the largest frustration is that the language gets in the way, and the pain increases with the scale of the problem. Which is to say: I think Go scales to large projects just fine, but there are problems where you'd like to build big building blocks on top of smaller blocks on top of smaller blocks, and Go doesn't lend itself to certain kinds of big, composable, data-oriented abstractions. It's small building blocks all the way.
I've bumped into several very real problems recently where Go's coarse, not-very-data-oriented imperative approach has revealed itself as a liability, and where I found myself fantasizing how I could have done it in just a few elegant lines in Haskell. Sometimes they're about expressing things simply in a composable manner, and sometimes these problems simply manifest themselves in immense blobs of boilerplate/repetition (for example, because you have to implement the same method a few dozen times on different data structures, which in a different language could be solved with a generic implementation), where Go's solution is to either eschew type safety, use slow reflection APIs, or programmatically generate the Go code as part of the build process.
Go's is also frustrating in its selective pragmatism. Where Go has chosen to automate and sugarcoat some complicated things (memory management, goroutines), it's stubbornly unpragmatic about other things (error handling, working with polymorphic data, memory safety). Go has been ridiculed for its simplistic error system, but I'm not an extremist here; I'm all for errors being values, and not a fan of exceptions. But if you look at actual Go code, a huge amount of code has to interact with errors. When nearly every function is riddled with "if err != nil", you should know that your language is crying out for just a little syntactic sugar. Or a solid type-system solution for that matter. Enums (Rust-style) and pattern matching wouldn't go against Go's grain at all, but since Go is "done", we're stuck with how it is.
I think Go's focus on simplicity is very important (I'm a big fan of the Wirth school of languages), and my worry about Rust and Swift is that they never learned this lesson. To me, both Rust and Swift looked more promising early in the design process than in their current state; Swift looks increasingly like Scala every time I visit it, whereas Rust often feels lost in a sea of punctuation. That said, my annoyance with Go is acute enough that I'm willing to deal with a few downsides if I can get a language that better matches the kinds of projects that I build.
> my worry about Rust and Swift is that they never learned this lesson
FWIW the Rust team does consider simplicity to be important. However, it's not the only goal, which means that sometimes it has to be sacrificed to be able to make something possible. But most new language features do get discussed in the context of a "complexity budget".
So Rust tries pretty hard to ensure it doesn't get more complex than it has to; but it doesn't put simplicity as the overriding goal and hang all else to get it.
> Enums and pattern matching wouldn't go against Go's grain at all
I've said this many times before -- I don't miss generics from Go; I understand why they're not in the language. I miss ADTs and pattern matching.
I'm learning Rust. I considered Go but feature wise compared to Rust it seems really boring and plain. IMHO modern language has to have functional flavor.
Go's GC isn't a big deal for most use cases. However, the loss of static guarantees regarding thread-safe manipulation of arbitrarily complex data structures is a big deal.
> * Great Libraries ( Everything and a kitchen sink )
This is a must for me. I often give up on compiled languages because libfoobar isn't yet available on the language and I have no interest in writing (and maintaining) language bindings for third party libraries I use.
My dream is that one day I'll be able to do something like:
Foobar = link_cimport({"headers": ["foobar.h"], "packages": [ "foobar" ], "prefix": "foobar_"});
Foobar.do_something(); // would call foobar_do_something();
Foobar.hidden_function(); // would call hidden_function();
and the compiler will give me an executable that was linked against `foobar` instead of dlopen()ing it at runtime. Something like:
Foobar = loadable_cimport({"headers": ["foobar.h"]});
foo = Foobar.load("/path/to/foo.so");
bar = Foobar.load("/path/to/bar.so");
would also be nice because I often have to load libraries that implement the same API/ABI.
It converts C headers to a Rust module containing the relevant type/function/etc. definitions. On stable you need to either pregenerate the module (not too bad if you're pinning particular versions of libraries anyway) or add a build script to autogenerate it. On nightly, there's also a compiler plugin that does all the work for you, and looks not-too-dissimilar to what you wanted. The only thing it lacks is the prefix removal stuff.
Which is undersold in the propaganda! Safety, ehhh I'll take it but I'm not jonesing for it. A package manager that works and has adoption? Thank you! I've wasted far too much time wrangling C++ dependencies by hand...
Because Rust is IMHO the first viable replacement for C. Well-written C programs translate almost 1:1 to Rust.
It can do (roughly) everything C can, i.e. it's natively compiled, gives control of memory layout, and doesn't depend on a GC runtime.
Previous C killers were either dependent on a fat runtime (which isn't a big problem in general, but is a problem for some of niches dominated by C), or didn't offer meaningful improvement in safety/concurrency/expressiveness.
* "First viable replacement for C" - C++ was invented decades before Rust.
* "Well-written C programs translate almost 1:1 to Rust" - C programs can be compiled with minimal changes as C++.
* "or didn't offer meaningful improvement in safety/concurrency/expressiveness." - C++ offers major improvements in all of those areas.
A lot of C programmers don't want to switch to C++ and they won't want to switch to Rust either, because they favor simplicity and neither Rust nor C++ are simple. They'll probably switch to golang for some stuff if anything and just stick to C for the rest.
C++ is complex because it's a mashup of overlapping features from different eras.
I don't write C++, because whatever I write will crash for subtle reasons and I'll be told it's my fault, because of course you don't do this when you use that feature.
I can write Rust with all its complexity, and as long as I get it to compile, I'm confident it works more reliably than any C++ that I could have written.
On my system (2015 MBP running Ubuntu 16.04) Rust's hello world compiles to a nearly 3.5 MB executable. Compiling the equivalent C program with clang results in an executable under 9 KB. Rust may not have a GC, but its executables are certainly fat.
Note: Rust executables can be trimmed down by stripping them (not done by default, even for release builds) and using libc malloc instead of jemalloc. But even then, C wins by a lot.
C links dynamically to its libraries. Rust does not (by default).
This overhead is a constant overhead so it rarely matters. When it does, you can strip it down further and get it to the same level as C.
This is not evidence of a fat runtime, just a different default compilation strategy that prodces larger binary executables. jemalloc is (an optional) part of the runtime, but the rest isn't.
Sure. It's not a fair comparison. But dynamically linking rust's std isn't an option. If you are optimizing for executable size, c wins because dynamically linking libc is possible. This is even true controlling for glibc, since most rust code also dynamically links glibc too (including hello world). In fact, rust's hello world has quite a few more shared dependencies than the equivalent c anyway: it uses libdl, librt, libpthread, and libgcc_s as well.
Dynamically linking to rust std (with a specially compiled libstd dylib) is an option iirc, just that nobody does it. Lack of stable ABI makes it less useful, but if you really need to do it there's nothing stopping you.
I suspect rust hello world using those libraries is due to a lack of LTO? Hello world doesn't need any of those.
Not a particularly fair comparison though, as that C binary is loading dynamic libraries that are already in memory. Case in point: that C program needs the C standard library, but by default that library (glibc) is dynamically loaded. Rust is not offering the Rust standard library as a dynamic library, hence the larger size. Not to mention, interfacing with C functions in the kernel.
You'll get different results if you compile them with musl for truly static and fair binary size comparisons.
I am no expert, but you should compile barebone executable for comparing executable size! not "hello world", because hello world does use shitload of library (for printing and loading and stuff), and those ones in C is compiled in dynamic, but in rust those compiled inot executable itself.
if I remember correctly you should compile with --nostd flag and provide _start function for loader to load your executable!
you can do same with rust too (But I think you should provide a special flag for compiler). You can find all of it on documentation.
To me it's almost the opposite. I like C a lot, but I ended up compromising on C++11 because in some programs I need more features to keep the implementation clean. I pay the cost of a messy language (C++) when implementing my libraries in order to have simpler applications that use those libraries.
I've written C-like programs in Rust, and that goes very well. But I really like function overloading, generic operators that I can overload from the left and the right, integer parameters for my templates/generics, copy semantics as the default (opt-in for moves), placement new, explicit destructors, and maybe a few other niceties. Rust does none of these things the way I want, and so I'm stuck with C++.
There are a few other things where I think Rust just made the wrong choices and didn't improve over C++. I've always hated the dichotomy in C++ strings between const char* pointers and an actual string class. Rust had the chance to make strings feel as comfortable as integers, but instead they introduced their own dichotomy with String and &str.
Speaking of integers, Rust's choice of 32 bit integers as the default for literals is painful given that every machine most of us will ever care about is now 64 bit. I routinely deal with arrays and files that are too large for a 32 bit integer. This means I have to remember to suffix all of my integers in for loops etc... C++ has this wrong too (backwards compatibility), but Rust had a chance to make a clean start and did not.
> Rust had the chance to make strings feel as comfortable as integers, but instead they introduced their own dichotomy with String and &str.
It makes perfect sense when you understand the differences and reasoning behind it. A `String` is a heap-allocated string that can grow in size. On the other hand, a `str` is basically a fixed-size string array, but you'll never interact directly with this type because there's no point in it. It's tucked away inside the `String` that created it.
Meanwhile, an `&str` is a slice of that fixed-size `str` array that's hidden within the `String`. You can think of a `str` as an `[char]` with it's own special `str` methods, and an `&str` as an `&[char]` which has access to all of the `str` methods as well.
> Speaking of integers, Rust's choice of 32 bit integers as the default for literals is painful given that every machine most of us will ever care about is now 64 bit. I routinely deal with arrays and files that are too large for a 32 bit integer.
I'm pretty sure the default integer is a `usize`, which is 32-bit on 32-bit systems and 64-bit on 64-bit systems.
No, the default integer is u32. If type inference can't provide a better type, Rust will pick u32. If type inference can pick a better type it will use it. So if you create an unsuffixed int literal and use it for indexing, that literal will be a usize.
Rust will type error if you try to use u32s with an array so it's all good though. Just means that you need to specifically `: usize` things.
If you need an integer type for counting stuff, u32 is fine. If you actually need to talk about memory, use usize. The compiler will force you to do it.
I believe it used to be uint (usize) back in pre-1.0, and IIRC inference didn't work well or wasn't supposed to work at all. I recall needing explicit prefixes on my int literals back then. No longer the case.
How would you have gone about making Strings be "as comfortable as integers"?
Arrays are indexed by usize, so if you're on a 64-bit machine, then you shouldn't need a cast. It's _unconstrained_ numbers that default to i32, not anything without a suffix.
> How would you have gone about making Strings be "as comfortable as integers"?
I would prefer a str be a str be a str, regardless of how you got it. Lowercase type-name and fundamental like an integer.
I'm fairly certain I understand why Rust made the choice they did. I've read the forum threads at HN, Reddit, and users.rust-lang, and I've seen previous replies by you and other Rusties, so I hope you won't try to educate me about the performance advantages of having slices as references and another string type as an ownership class or why we need OsStr and friends.
If strings are the fundamental processing concern in your application, then I think you should be able to opt-in to that kind of micro-optimization and complexity, but it would've been better to spare the rest of us who have different concerns. I don't want to become a string expert to build a filename, and the default implementation could (at least conceptually) be always on the heap for all I care. Go one step further and implement the "small string optimization" (Alexandrescu's fbstring), and you'd probably get back most of the performance without nearly the complexity.
> Arrays are indexed by usize
I've spent the last half hour trying to troubleshoot why this line hangs:
let aa = vec![1u8; 10e9 as usize];
However, I'm at home using the Ubuntu under Windows thing, so maybe there's some bad mojo between Rust and my less than usual setup. (If you're interested, I have 32 Gigs of memory, and the equivalent C malloc and memset code runs just fine, so I don't think Rust is doing the right thing here...).
Anyways, I wanted to test the commented out line, but I'm stuck for now. If that line would work, I'll admit I was wrong, but I think the uncommented line shows a similar complaint.
for ss in 0..63 {
//print!("{}\n", aa[1<<ss]);
print!("{}\n", 1<<ss);
}
> It's _unconstrained_ numbers that default to i32, not anything without a suffix.
Looking at the present and the future, why is that a sensible default? Both x64 and ARM are going to use a 64 bit integer register for the operations, and many of those operations are going to be 1-clock throughput. You can probably find a counter example, but 32 bit integers aren't generally faster than 64 bit ones.
> I would prefer a str be a str be a str, regardless of how you got it. Lowercase type-name and fundamental like an integer.
The lowercase type names are reserved to primitives. The String type is not a primitive but a comprehensive data structure, hence the capital S. The String type contains an `str` primitive though, along with size information.
> I don't want to become a string expert to build a filename, and the default implementation could (at least conceptually) be always on the heap for all I care.
Is it that hard to understand that when you create a string, you will create it as either a `String` or `PathBuf`? File methods are designed to automatically convert input parameters into a `&Path` so it doesn't matter what string structure you provide.
There is also no way (currently) to create a stack-allocated string with the standard library out of the box. You can do this with crates like `arrayvec` though. It's very much opt-in for that performance.
let path = String::from("/tmp/file");
let mut file = File::open(&path).unwrap();
> Looking at the present and the future, why is that a sensible default? Both x64 and ARM are going to use a 64 bit integer register for the operations, and many of those operations are going to be 1-clock throughput. You can probably find a counter example, but 32 bit integers aren't generally faster than 64 bit ones.
No need to use a 64-bit integer when you only need a 32-bit integer. You can fit two 32-bit integers into a single 64-bit integer and perform a calculation on both simultaneously with a single cycle, versus spending two cycles to calculate two 64-bit integers. There's also no need to pay that memory cost either.
No, I'm interested in what tradeoffs you would have made differently. I now understand. Thanks! (I disagree, but at least I understand.)
> why this line hangs:
It compiles and runs effectively instantaneously for me on Ubuntu under Windows as well, so that's very strange. Maybe file a bug?
> I think the uncommented line shows a similar complaint.
Yes, there's no constraint on that literal, so it's going to be an i32. When we made this decision, we did some analysis, basically no numbers in real world programs weren't constrained, it was often tests, toy programs, and documentation. It should be a rare thing. YMMV.
> why is that a sensible default?
Your assertion about the speed was the opposite of what was asserted while we had the discussion, basically. And not everybody is running on 64-bit hardware, so it's a broader default.
> It compiles and runs effectively instantaneously for me on Ubuntu under Windows as well, so that's very strange. Maybe file a bug?
Follow-up: I tried it with -O (don't know why I didn't think of that earlier), and it runs fine. So maybe the debug version is just generating terrible code, initializing by iterating through 10 billion bounds checks or something?
Anyways, more importantly, it works as I would like and does not behave as a 32 bit integer. I think I understand what you mean by "constrained" now. And clearly, I was wrong.
However, if most un-suffixed integers in real-world programs will become constrained (as you claimed), this further confuses me why i32 is the unconstrained choice. It doesn't seem like something so rare could be enough of a performance problem to justify being anything but the largest supported size.
I remember installing it by cut and pasting one of the "curl ... | sh" commands there.
> Nothing about that code should be doing bounds checks, as it's just allocating an array.
I didn't dive into the macro definition for vec!, but I assume there is a loop in there to Copy the initialization element 10 billion times. I think you guys do bounds checking on the lower level reference to a slice that Vec uses. But I really don't know. If it's not that, then it was hanging or spinning doing something else. (Debug version of your memory allocator?)
> basically no numbers in real world programs weren't constrained, it was often tests, toy programs
The fact that C and C++ compilers generally chose to leave int at 32 bit on 64 bit platforms, combined with the standards requiring "usual promotions" for smaller types to go to int bites me all the time. I'm very happy that Rust dodges the promotions problem altogether, and I'm sorry if I'm wrong about the array subscripting thing (does the snippet I provided panic at 1<<32 or 1<<34?).
> And not everybody is running on 64-bit hardware, so it's a broader default.
That argument could be used to justify 8 or 16 bit integers... :-)
> (does the snippet I provided panic at 1<<32 or 1<<34?).
Overflow is a "program error", and in debug builds, is required to panic. In other builds, if it does not panic, it's required to two's compliment overflow. Rustc currently just overflows, but in the future, we'll see.
That's true except our 16 bit support is nonexistant at the moment :)
> But I really like function overloading, generic operators that I can overload from the left and the right, integer parameters for my templates/generics, copy semantics as the default [etc.]
I do too, and I'd like to think that rust needs all those things to be a replacement, but realistically I think that the only killer feature that rust is still missing is reasonable interoperability with C++ (which admittedly might require implementing a few of those features).
Disclaimer: I have been following rust since Graydon initial announcement, but I have yet to write a single line of code in it.
What do you mean by "reasonable?" There are several crates that provide inline C++ macros on both nightly (rustcxx) and stable (rust-cpp). I haven't used the latter but the former works really well (albeit using a gcc specific feature). I usually break out the C++ code into a wrapper (unless its a few lines) to make it more idiomatic Rust but at the end of the day, there aren't that many hoops to jump through.
You use a lot of typedefs, so I couldn't tell for sure, but I think your operator[] returns a C++ reference right?
The problem here is there is only one operator[] for both reading and writing. This is a simple contrived example, and taking a reference like that looks artificial, but there are a lot of other ways in real programs to stumble on to this. (I don't think it's as bad as the Rusties do, but I stumble into this bug once or twice a year...)
The Rust folks seem to believe you need a borrow checker to solve this problem, but I think that a different container library in C++ could do the trick. For instance, favoring value copies instead of references, and returning a proxy object from operator[] instead of a reference.
Native C++ references are technically unsafe, so code that uses them would not qualify as "strict" SaferCPlusPlus code. In the case of your example, the "double&" is technically not kosher. The easiest way to make it safe would probably be to use a (safe) iterator instead of a native reference. So instead of
double& dangling = data[0];
you could make it
auto not_dangling_iter = data.begin();
// not_dangling_iter += 0;
C++ references are the one unsafe element that does not have a "compatible" safe replacement. Unfortunately, you have to convert your references to pointers (or iterators). I don't think there is a way to create a "safe" reference with an interface compatible with native references. Apparently C++ will at some point add the ability to overload the dot operator, but I'm not sure that will be enough to be able to emulate C++ references.
And while I can overload the & (address of) operator to "prevent" you from getting a native pointer to a "safe" object, I don't know if there's a way to prevent you from getting a native reference. If you wanted to somehow enforce a prohibition on the use of unsafe C++ elements (like references), that would probably require some sort of static tool that is not yet available. But should be fairly straightforward to implement, I think.
But if you just want some confidence in the safety of the code you write, it doesn't take much effort to reliably avoid using C++'s unsafe elements.
> returning a proxy object from operator[] instead of a reference.
Oh yeah, maybe. But that would still require the ability to overload the dot operator, wouldn't it? And how would you know when to deallocate the proxy object? And presumably there would be some run-time overhead. Hmm, I don't know if it wouldn't be more practical to create a static tool (or "precompiler") to automatically convert references to (safe) pointers (or iterators).
Yes, the dot operator is a headache. As soon as you march down the road of a precompiler, you're off to building a new language. I think C++'s grammar is too much of a mess to just tweak the parse tree reliably. I suspect there really isn't a way to win at this - every workaround is partial and involves compromise.
You know, while C++ references are technically unsafe, there is TRegisteredRefWrapper<> [1]. It's a safe version of std::reference_wrapper. Which kind of acts like a reference. So, if you don't mind me using std::strings instead of doubles, your example could be rewritten like
Does that work for you? I'm not an expert on std::reference_wrapper, so I'm not sure when it can and cannot substitute for a reference. (Btw, if it's a little verbose for you, there are shorter aliases available. Just search for "shorter aliases" in the header files.)
DICE and others have been investigating and using it to create tools for developing games. There is interest in using it within game engines, but there's just the issue of Rust support for major consoles. There is great interest in the PC gaming world though that's not constrained by console support.
There's been some interest, but I think the likes of jblows Jai language that emphasize rapid development and programmer convenience over correctness will likely catch on faster.
I appreciate rust making a break from C++ and cleaning up some of the warts, immutability by default, having real modules etc. But I am not really impressed with its memory management thing. It's a bit tiresome to check stuff with valgrind, sure, but I don't have to worry about making cyclic data structures satisfy the borrow checker. I don't see the trade off as worth it.
C has also been relatively stable for the last 25 years or so. Rust changes every 6 weeks.
Yes, the language is supposedly not undergoing breaking changes without strict deprecation and feature-gating. However, as new features are added to the language, libraries are usually updated to leverage them, and suddenly you are also forced to keep up with that rapid release cadence to ensure your project still builds.
Meanwhile, C code from the mid-80s still compiles with modern compilers, often without any changes whatsoever, and without any feature gates.
Don't get me wrong. I like Rust and what the project is trying to do. But I see two major problems that are preventing me from using it for anything major at the moment: 1) the extremely rapid rate of change, and 2) difficulty of implementation (both of which, taken together, lead me to expect no serious alternative implementations any time soon).
Before I will be willing to adopt Rust as a C replacement in full, there needs to be a solid, stable language definition that compiler writers can target and programmers can rely on for a good chunk of time. I have a feeling we'll get there eventually once the number of new, useful features that are missing from the language starts to dwindle and it begins to stabilize naturally.
In the meantime, I'd settle for an "LTS" release or something. That could work.
I don't understand. Rust 1.0 code will compile on Rust 1.14, in the same way that C code from the 80s will compile on modern C compilers. That's what "backwards compatible" means!
Your dependencies don't upgrade themselves. Its true that if you upgrade your dependencies, they may depend on a more recent verison of rustc, in which case you need to upgrade rustc as a part of upgrading that dependency. But if you are using rustup this is not harder than upgrading those dependencies - probably easier in fact.
In exchange for downloading a tarball at most every six weeks, you get a language that is developing new features.
The problem with your concept is that C does not offer an official standard library and compiler suite, so you're trying to compare the C spec to the Rust standard library + compiler.
The actual Rust API has changed very little since the 1.0 release. All the changes that have occurred have been mere additions to the language -- nothing breaking. Hence why the 1.0 release was termed as a 1.0 release. There won't be anything happening that causes breakage until a 2.0 release.
You'll find that C compiler and library development is also rapid like Rust. There's new versions of LLVM/Clang and GCC being released on a regular basis. There are constant updates to musl and glibc. These changes tend to break far more often than Rust, which is developed in Rust and can be guaranteed to be much safer.
That Rust is being openly developed on GitHub with a significant number of developers, with an official compiler + library + doc + comprehensive suite of resources collectively being maintained, that's a significant incentive to work with Rust. There's a solid RFC process to prevent anything silly from entering the language. No random decisions by random people -- it takes a lot of convincing to add a new feature to Rust.
C does have a standard library. It's described in full in the ANSI/ISO/IEC standard for the C programming language, and it's on your system, likely called libc or something along those lines.
Anyway, I don't think that's a problem with my concept at all. What I want for Rust, and what I think will eventually be feasible for Rust as the language matures, is a standard that can be shared by compiler writers and Rust programmers. A standard is worthless if it's inadequate (see ISO Pascal or ISO Basic, for example), so it's a good thing that right now Rust is iterating quickly and improving rapidly. But eventually, there's going to need to be a stationary target for compiler writers to aim for, and for Rust programmers to be confident that the code they write today won't be littered with deprecated features, anti-patterns, and worst practices tomorrow.
Frankly, I don't see wide adoption happening in the corporate world until there's an LTS compiler at the very least.
> C does have a standard library. It's described in full in the ANSI/ISO/IEC standard for the C programming language, and it's on your system, likely called libc or something along those lines.
What you're describing is the spec and not the implementation. An implementation would be glibc (GNU Libc with open source proprietary extensions) and musl (MIT implementation that strictly adheres to spec). These libraries are updated often.
Meanwhile, Rust offers more than just a specification. It offers an implementation. This is libcore-rust + libstd-rust. This is what receives updates each 6 weeks. It also offers a compiler that comes with it, rustc, which is also what receives updates each 6 weeks.
Rust strictly adheres to semantic versioning, so changes to the API are not allowed without a major version bump. Code that was written for Rust 1.0 will still compile on Rust 1.15. Code written for Rust 1.15 may not compile for a Rust 1.0 compiler though because bumping the second version implies adding new features. Rust doesn't yet utilize a patch version though. There's never been any critical issues discovered in each stable release.
> Frankly, I don't see wide adoption happening in the corporate world until there's an LTS compiler at the very least.
Can't say that I understand why a LTS compiler would be required for adoption. Many in the corporate world are already using Rust, either publicly as an official friend of Rust or privately in smaller projects. There's nothing a LTS compiler would provide that would be beneficial to the corporate world. Semantic versioning and the ease of managing Rust toolchains with rustup has everything pretty well covered.
I can only attest to my own experiences at various corporations, and those corporations want to thoroughly vet any software platform before targeting it and supporting it for customers.
As to my own concerns (rather than corporate concerns), here's where I'm coming from:
When I wind up choosing C for a particular project, I don't use it because I like it (I don't particularly like C). Typically, I use it because there are three factors I've considered: 1) This particular project needs the low-level access or performance characteristics provided by a systems language, and 2) C is such a language, 3) C is ubiquitous.[1]
The thing is, every platform for the last 25 years or so has had a reliable ANSI C compiler. If the goal of my project is to be cross-platform, then a carefully-written application in standard C is guaranteed to be able to reach the widest number of users, many of whom won't have to do or install anything to build my application, since many environments already include a C compiler without any additional installation necessary (think Unix, BSD, a shrinking number of Linux distros, ...). Using C puts my application on the path of least resistance for potential users.
Similarly, if I'm writing a library, every language with an FFI can trivially access a library written in C. Even without an FFI, most languages can interface with C code through some kind of bindings (like the Lua-C API, for example). Again, choosing C is a path to wide adoption.
Again, I don't particularly like C as a language. However, right now, it has an undeniable advantage as a platform. I do like Rust, however, and I would love to see Rust give C the boot. In order to do that, it needs to eventually (one day when the rate of additions/changes to the language has slowed and it has grown more stable/mature) offer a platform that can be relied on to "just work" on any given platform with minimal fuss and without any sudden changes. As such, both the "end-programmer" and the compiler-writer in me would like to see some version of Rust (again, eventually) that is "set in stone" and widely available, which really would solidify that notion of stability/maturity.
I'm not advocating for Rust to stop innovating. For all I care, the "standard" can be a simple snapshot of some state of the language, the way the Haskell Reports seem to be nowadays: GHC keeps on iterating and innovating the language, yet there's still a standard updated every x years that provides a static target. But for that designated snapshot, I want to see a compiler (or two, or several) and a bunch of libraries that are supported for the long haul.
It's simply a matter of "C has this, it's an important property thereof, and any replacement will need it, too".
[1]: Some other projects demand C because they constitute contributions or extensions of other projects that are already written in C; e.g., kernel code for an existing OS that is written in C.
When Rust (or Swift or golang) gets ISO standardized we'll know that it's mature and left the "move fast and break things" phase. This won't happen, because it's still growing now, so it doesn't make sense to standardize. They might even always want to be able to move fast. I don't see golang ever being standardized.
The fact that someone says on HN "we're backwards compatible" doesn't quite offer the same level of trust as an ISO standard. Especially when there are some conflicting reports out there i.e. there are many super-awesome libraries that only work with nightly (I think it's called), because many of those maintainers are early adopters that love to try out new features.
Programming languages and libraries are building blocks, I don't want them to change under me in the middle of a project. This is why I dropped Swift, the 1.0 -> 2.0 migration was a very painful, and from what I hear recent major version changes aren't running smooth either.
> Especially when there are some conflicting reports out there i.e. there are many super-awesome libraries that only work with nightly (I think it's called), because many of those maintainers are early adopters that love to try out new features.
This is totally orthogonal to whether or not Rust stable is backward compatible, which it is. People put a lot of work into making sure it is, and this sort of FUD is very frustrating to read.
> The fact that someone says on HN "we're backwards compatible" doesn't quite offer the same level of trust as an ISO standard.
Then you shouldn't take the word of a random person but from the Semantic Versioning guidelines. If you're not following the semantic versioning guidelines then you're doing it wrong.
> Especially when there are some conflicting reports out there i.e. there are many super-awesome libraries that only work with nightly (I think it's called), because many of those maintainers are early adopters that love to try out new features.
Totally irrelevant. Very few libraries, if any, require a nightly compiler. The libraries that do offer features that require a nightly compiler have those features as optional. Anyway, once the new Macros update releases, the need for that nightly compiler will go away. These libraries are just taking advantage of compiler plugins.
I could definitely live with such a solution, but even six months might be too quick. So, as long as the plan is to lengthen the LTS window, I think that could work.
The company that employs me just upgraded to a C++11-capable version of GCC three months ago. Even C++'s three year cadence seems like it might be too fast! Thing is, the company wants to vet our entire platform, which itself only changes yearly. They want to know they can get bug fixes, and vetting/upgrading the compiler is a lot of work regardless, so they tend to do one and stick with it for a while. OpenBSD does the same thing for security reasons. If Rust wants to replace C and/or C++, these are things that will need consideration eventually. I don't see my company allowing Rust any time soon, unfortunately.
As a compiler-writer, I'm more concerned that Rust is a quickly-moving target at the moment. I think that as Rust becomes more mature, it will naturally begin to stabilize feature-wise, in which case something akin to the Haskell model would work well, with innovation happening in the flagship compiler and "snapshots" of the language being taken as standards periodically. That's just a matter of time, though.
Completely agree, C++'s three year cadence is probably the fastest more conservative industries are willing to go.
And why should one invest time for such a low-level component in their toolkit every 6 months?! I want to build software and solve problems, not spend my time keeping up with programming languages and library updates.
I have upgraded Rust every six weeks. I run `rustup update stable` and wait a minute. Its not a time investment! Rust puts so much work into guaranteeing backwards compatibility so that updating every 6 weeks is an effortless process.
You seem to come from an experience where software updates are kinda sorta mostly backwards compatible. Rust works very hard to actually be backwards compatible. There is infrastructure to check for regressions across the entire package ecosystem, which is run regularly on HEAD and on any commit we think might potentially be breaking before we even land it.
If the software does what you want, being able to run on 30 years worth of hardware with just a recompile is a good thing. At the end of the day, it's what the software does that's important.
> If the software does what you want, being able to run on 30 years worth of hardware with just a recompile is a good thing
That's not a responsible metric. 30 year old software was written against 30 year old APIs vulnerable to known attack vectors, and with 30 year old notions about security. The belief that this software is suitable for modern use is absurd IMO.
And if the types of programs you're talking about are small UNIX-like utilities, well then they're small and easily updated aren't they?
More often than not, brand-spanking-new software is written against 30-year-old APIs.
Lots of software we use day-to-day is old. For example, a couple years ago I fixed a 20-year-old bug in netcat, which itself hadn't been updated since the mid 90s. After I fixed the bug, I didn't need to fish out an old compiler or any compatibility libraries; I just compiled it with the latest GCC and glibc, and carried on with my day.
Don't get me wrong, I like Rust a lot and I'm excited that there's finally a real contender for the systems programming throne after all these years. I'm just less enthused about the prospect of refactoring my codebase once every few months to stay current. This is something that I think will level out with time, though. C itself was almost 20 years old before it was standardized, after all. I don't think it will take Rust that long, and I look forward to the day my copy of the Rust standard arrives on my doorstep.
> More often than not, brand-spanking-new software is written against 30-year-old APIs.
Let's assume this is true, brand-spanking new software will be written against 30 year old APIs that have survived 30 years of attacks and audits, but will avoid the ones that have proven vulnerable.
> I'm just less enthused about the prospect of refactoring my codebase once every few months to stay current.
The perception that you need to do this is false. You might feel pressured to refactor simply because of new syntactic improvements that make code clearer (like switching try!() macro to infix ?), but that pressure is a fiction. There's no real need to do it.
> After I fixed the bug, I didn't need to fish out an old compiler or any compatibility libraries; I just compiled it with the latest GCC and glibc, and carried on with my day.
Could you show me an example where post-1.0 you wouldn't be able to do this with Rust?
If the software does what you want includes all of those criteria. If you find that there are deficiencies in any of those areas, that would qualify as a reason to make changes, weighed against the cost of doing so.
And the plan is that you'll be able to do the same with Rust; the more rapid release strategy doesn't affect that at all.
What the rapid releases do affect is the rate at which new releases of libraries stop working on older compilers, which absolutely has to do with ignoring progress.
This is my same problem with JavaScript. Yes it's wonderful that changes are backwards compatible. But if the entire community is updating libraries you rely on to use new language features then you have no choice but to use those features too.
Or stop using other peoples' code and just use core language features and build everything from scratch. Which is what I end up doing, and it has some benefits. But it definitely slows you down.
Not everyone uses C only for the cases when they need a "small and portable language". C is used quite a bit in places where Rust or C++ would do fine too, for various reasons. That's where you want to compare them.
Maybe Rust folks should make their own C backend for better integration with C compilers and have that portability, since LLVM folks decided to abandon theirs.
We've slowly been shifting how we talk about Rust as it evolves. I agree with you that these things matter, and they're all things we're actively working on.
I think they are well aware of the problem at this point. Their best bet is to focus on high-performance networking software and networking ecosystem, where alternatives are still weak.
> It's very rare that any programmer switches computer languages.
Or is it?
I regularly use six different languages of three paradigms and various
abstraction levels, and this doesn't count DSLs nor Turing-complete tools like
make. Of my colleagues from current job, half of the more experienced set
write in at least three languages as well. Of the team from my previous job,
all of them are polyglots. And of course none of us started with all the
languages, we picked them up as needed and wanted.
I'm 19, I started out with Java, I switched to JavaScript for some time, I move on to Python for which most of my projects exist in and sometimes I use C when the job counts for it. I'm also a pretty big fan of PHP and a few other technologies that are very nice.
To recap this has been my life as a programmer: Java -> JavaScript -> PHP -> Python/C/Assembly
This dream that most people have made up about programmers not switching languages is a dream. I switch whenever one language can do a task better then another. I don't get caught up in "best language" fights.
Yeah but you're just getting started. Talk to me after you've been paid to be a python developer for 20 years or whatever and tell me how interested you are in learning a new language.
I play with a lot of languages but I've invested a lot of time into learning python and getting a nice python workflow going. It would take a lot for me to switch to another primary first language.
I work at a tech company that's been around since the 90s and the guys that were writing perl and java in the 90s are still writing perl and java. You'd have to drag them kicking and screaming into using a different language.
Yeah I can use several languages (C,C++,python,go,rust,x64,perl,scala,etc...) But I always envision starting a project in C or C-like C++ because I learned it first and after decades I know that language inside and out and my brain's "workflow" and my actual tool setup and personal libraries means I can crank working solutions out extremely quickly.
I pick the best tool for the job. I'm using pything because right now I'm in a python shop. I'd like to go back to Java though as pythons asyncio is insane. The only reason I'm using pyhton is because the people inheriting my code will only know python.
What may look like "bah humbug I'll use X because I like it" may just be picking a language on concerns not limited to features. Looking at social concerns is important as well. Who is maintaining your code after you're gone? Company tooling for certain languages? Will I get fired for using it (AI winter)?
That's definitely not true. I've had the pleasure of working with many colleagues that have, and (since good Scala engineers are thin on the ground) my company will often hire good engineers who're interested in working with Scala. Heck - I myself have picked up Scala and Rust in the past couple of years.
> “Safe” code is guaranteed to be 100% safe. Not statistically safe. Not safe when the compiler feels like it. As long as your code compiles, it will be safe in terms of memory safety and data-race freedom.
As far as I am aware, the Rust compiler has not been proved correct. So whether or not your code is correct still depends on the correctness of the compiler. Of course this is probably correct, but still not 100% guaranteed.
The kind of safety guarantees Rust provides are, in my opinion, insufficient justification for experienced developers to move from C or C++. Rust has other features that make it generally superior in certain (many) contexts. The safety is a nice "add-on" effect, I suppose, but my view is that constantly hyping safety as the biggest selling point is missing a mark.
I've seen fatal flaws from supposed experienced C and C++ developers in a large number of high profile projects, that would otherwise be nuked from existence had they used Rust.
I've also encountered severe flaws in libraries written by experienced programmers working for companies like Google and Red Hat that the Rust compiler caught during translation to Rust.
That's not counting the immense amount of tooling and features that make managing large projects a breeze in Rust. Managing a 100K Rust project with a large team is easier than managing a 100K C++ project with a large team. You can't guarantee that no one will ever make mistakes.
I find segmentation faults to be common place in C/C++ projects from experienced developers, and it's quite difficult to debug them at times.
> The kind of safety guarantees Rust provides are, in my opinion, insufficient justification for experienced developers to move from C or C++.
Avoiding security related issues based on a whole class of problems prevented in Rust by design is a huge benefit. It's better for compiler to prevent that, than to rely on levels of experience, while anyone can make a mistake in reality.
Rust only deals with half of the story at the compile time because it can't deal with stuff like cyclical refs, etc., and even without it it also makes a lot of valid code invalid because of it's borrowing rules.
That's not saying that borrow checker is worthless but it does add overhead and frankly in a lot of scenarios that's just not necessary, even with all that fancy compile time checking you still need to do testing and there are many applications where "working for the stuff tested" is enough. Obviously when you're writing a browser that people will try to exploit and you need to execute third party code and load content from entrusted sources things will be different.
So I get where OP is coming from, Rust safety is oversold IMO - it's nice but it's not the only thing that's interesting about Rust - just the fact that it has modules, an official build system and a package manager that's used by everyone is enough for me to consider it over C++. It's lacking IDE like tooling and meta-programming capabilities (in stable at least)
It adds runtime overhead by forcing the programmer to work around the single-ownership rules, via runtime checking (Arc, etc) and more expensive interfaces.
For example, consider a simple thread-local counter. In C this would be e.g. an object in the tdata section, accessible via thread-local addressing. But in Rust, you have to use something like RefCell, which requires runtime tracking of mutable borrows. So Rust's borrow checker inflicts runtime overhead in this case.
That's not actually true though. You can use an atomic with a relaxed ordering. For example: https://is.gd/M1Q3Bu
Now, to be fair, this is only one example and I'm sure you could come up with a better one. But this isn't especially controversial. Rust's entire standard library is living proof that you need unsafe to do some things efficiently. The value proposition is that such things can be bundled up behind an abstraction barrier.
In sum, I think "the borrow checker forces runtime overhead" is an incomplete picture. I think a better picture is, "if you completely ban explicit use of unsafe from your code, then you may miss out on some optimization opportunities." i.e., When the borrow checker fails you, you're given a choice: 1) accept safety and the possible performance loss or 2) accept the onus of proving safety and do the fastest thing possible.
I think the fact that those choices are available is an excellent thing. I choose (1) all the time because not every line of code is relevant to performance. But sometimes I get to choose (2), and I'm thankful that such a choice is always very explicit.
For good measure, here's the unsafe version without atomics: https://is.gd/V1K6nD
Edit: Zero overhead as compared to C, that is. Cell has a slight overhead in Rust because it prevents certain optimizations that aren't the default in C.
Because an atomic provides interior mutability for `usize`.
(It's a hack that plays on the GP's specific example of a counter. It doesn't generalize arbitrarily to removing use of RefCell for thread locals in safe code.)
You only need a Cell for a counter (not even an atomic). Cell has no overhead over C -- Cell turns off some optimizations that exist in Rust to give you a C-like type.
I've been programming in Rust and C++ codebases for a while now. In general I find that Rust lets you dance close to the line of unsafety without worrying, resorting to Arc/RefCell only when you have to. In contrast, most C++ codebases I have worked with make use of shared_ptr (or its equivalent) quite often, because the language doesn't give you the tools to thread scoped borrows safely through a chain of functions and have it stay safe in the face of future code changes.
Sorry I meant overhead in terms of code size/use case complexity - not runtime performance (although like someone below mentioned you can often do "unsafe" stuff that's faster but can't be proved by Rust)
> If only these mythical experienced developers that never shoot themselves in the foot actually existed.
Rust is a great language, but people have to stop "marketing" it by saying things like: no matter how great you are with C, you're going to fuck up at some point, so that's why you should be using Rust. Like Go or whatever other language, the main grab needs to be a positive: you know what you're doing, and here's how (a, b, c... etc.) Rust will make you even better at what you do.
Another point, it's safe to say C isn't going away anytime soon, it sits nicely at the next approximately natural level (in abstraction) above assembly. That was a design point in the language. Yes it can be a rough tool to use, but that's a reality when you get closer to the metal. You get tremendous flexibility and that comes at a price, as all things do.
I don't see it as a negative, really, just realistic. It is a fact that everyone makes mistakes, and with the inherent complexity and features that are demanded of modern software, it is harder and harder to maintain a comprehensive understanding of a codebase so that you can detect these mistakes before they cause damage. This means that even if you are a perfect developer, you need to expend a lot of energy to maintain that perfection if you're using tools that don't help you.
To me, modern tools like Rust are supposed to relieve part of the burden. In the common case, if I can be reasonably certain that I can't make certain classes of errors, my brainpower is freed to focus on other things and I can work with significatly less stress.
Rust is interesting because it allows you worry less without actually compromising much in capability. In the cases where I would need to be able to bypass the tooling, it is explicitly possible.
Infallible developers don't exist, but use cases where the particular class of errors eliminated by Rust's safety guarantees is insignificant certainly do.
I'm terribly sorry you've never encountered an experienced developer who uses C or C++ before, or think we're non-existent, or that using extremes like "never" is a reasonable position instead of an incredibly terrible hasty generalization.
If there were as many blown off feet as comments on HN suggested technology even as it currently is simply wouldn't function.
Do you even understand how much mission and safety critical software is written in C? I think if you did you'd either have a constant panic attack (given your apparent belief that it's impossible to write "safe" C) or else have to adjust your world view a little bit.
No one has ever argued that it's impossible to write c code, but just because you haven't found those kinds of bugs in your code, doesn't mean that it isn't there. We're still finding 10+ year old bugs in Linux.
It's been around for roughly six or eight already, depending on how you count. Of course, pre-1.0 was a different thing, but still. Ten years is not that long a time.
(I still agree that results at that point will be more interesting then speculating today.)
> It's been around for roughly six or eight already, depending on how you count.
I don't think you get to play on both sides of that fence. Some of your team has been working on Rust for that long, but I doubt any code with sigils compiles, and I doubt there were many large projects using it then. I started my stopwatch at May 2015.
> Ten years is not that long a time.
Totally agree.
> I still agree that results at that point will be more interesting then speculating today.
Yeah that's why I said it depends. :) periodization is tough. 18 months, four years, six years, and 9ish years are all valid, depending on how. What I mean to say is that it's already been quite a while, and now that it's making its way into distros and required for building Firefox and all that, I think it has even more of a chance of sticking around for a long time, given that it was around for quite a while when it wasn't even a viable "real" language. I don't mean to insinuate that today's Rust is mega mature because those old Rusts exist.
Who claimed I haven't found these kinds of bugs in my code?
My response was to a commenter who, as far as I can tell, represents the mainstream Rust community's view: that it's impossible to write safe C or, worse, that infallibility is a reasonable standard to apply. The latter is especially vex-some from my point of view, because Rust is not infallible, I happen to find it (so far) much more pleasurable to use than either C or C++, and I would very much like to see it have broader adoption.
Sure, all things are possible. It is _possible_ to write perfect code, it's just not very likely. However, under commercial conditions it is so unlikely as to be implausible; you just won't have the time or budget for it. NASA can do it, most of the time, but the difference is that they take the time to find all of the bugs, not that they can write the code perfectly the first time through.
It's important for a lot of people and that's why it's marketed like that. It's also one of the most unique aspects of the language, and the primary goal of the entire project.
I completely agree. The biggest reason I haven't seriously tried Rust yet is because, despite all the vocal pro-Rust opinions we've all been inundated lately, I struggle to name a single interesting thing about the language apart from "safety". I wish the Rust evangelists would come to terms with the fact that to many developers memory safety is not a particularly important concern (for many different and often very good reasons) and talk more about other aspects of their language.
The tooling seems nice, but far from unique (D, Go and other languages are quite competitive there). It would be far more interesting to hear more about, say, traits (can it compete with C++ in compile time polymorphism?) and other technical aspects that make Rust stand apart from its competitors.
Compared to a lot of C++ and other compile time polymorphism, the error messages are consistently excellent, if sometimes long, and some of this is due to the simplicity of traits.
I am also extremely fond of pattern matching and algebraic data types (first class tagged unions). It is a style of data modeling borrowed from ML/Haskell and is quite distinct from OOP. The key distinction is that by making a datatype with multiple variants closed to extension, you can clearly see and handle all possible cases for a particular purpose (with checking from the compiler) which is often more natural then having fragments of implementation across many child classes.
I've not seen it mentioned here (unless included under generic "safety"), but prevention of data races in concurrent code seems like a big thing for Rust. Most examples of Rust being "safe" compared to C are usually about buffer overflows use-after-free, but elimination of data races to me seems like a huge win in its own right. I'm primarily a Java developer and a lot of the practices I'm learning from Rust apply to how I think about implementations in regular work, such as aliasing and mutability.
You're only going to anger people by referring to another group as 'evangelists' when you yourself have already made note that you don't know enough about Rust to see why they are promoting it. There's much more to Rust than safety. You should actually spend a few months with it, and then give your opinion.
I've used rust extensively and my opinion mirrors the parent comment precisely.
Safety is a cool feature, but Rust is more than just safety, and pitching it as 'rust is safe' when it's not (and, often, it's not...) and when people dont care about that, doesn't convince people.
There are a lot of people who blog about rust, and almost invariably the 'killer feature' they come up with is safety.
It's not the killer feature; it's the side effect of strong memory management without a GC on an excellent general purpose programming language.
We could certainly do with more posts about how great and practical rust is, what the state of rust tooling is, and what great rust libraries there are with no mention at all of safety.
(... because safety is a cool technical feature; but the others; tooling, libraries and productivity are active barriers to adoption)
> ...you don't know enough about Rust to see why they are promoting it.
If you have to know a lot about Rust to find out why you're using it in the first place, you certainly do have a chicken-or-the-egg problem. And, if the promotion doesn't leave you with a clear idea of why you might want to use Rust, that promotion couldn't have been very successful.
The same is true of any technology. One cannot give an informed opinion without first being informed about the technology in the first place. Technology moves forward because there are many whom are willing to leave the comfort of their boxes into the unknown to try things without being required to be convinced.
I certainly didn't mean to offend through my choice of words (many people seem to like the term "evangelist" nowadays), but it's not particularly realistic in an age of so many new languages to expect everyone to spend a few months on every single language just to form a basic opinion (that alone would be more than a full time job). Even the most polyglottic programmer has to pick and choose at least to some extent, and that's where the public perception of the language comes into play. I'm just pointing out that as long as Rust is all about memory/thread safety in the public mind, it will be ignored by large swathes of developers for whom that argument is simply orthogonal to their interests and everyday concerns.
A quick look at Rust's website and history should be more than enough to convince one to try it out versus other random programming languages with questionable backgrounds and ambitions. You have a large following that's backed by strong ambitions and companies like Mozilla and Samsung. It's always important to gain new perspectives and learn new concepts from languages that come by, as the ideas they teach are useful everywhere -- not just in the language of origin.
In any case, it's true that one shouldn't attempt to make an opinion about a technology without first being well informed with that technology, and that requires spending time with it to determine the validity of one's own hypothesis about it. Speaking down about it without having tried it is rather offensive to those who have spent the time to learn it.
Rust is strongly about memory and thread safety, which is a major plus. I don't see how that would cause it to be ignored by large swathes of developers when, in fact, that's what is causing large swathes of developers to make the transition. Even long time C houses like GNOME and Red Hat are calling out to stop writing future software in C, and to instead use Rust. GNOME's put their money where their mouth is and they are beginning to migrate their C codebases to Rust. librsvg, for example, is now written fully in Rust.
Rust's safeness allows me to feel comfortable working with really heavy optimizations that make extensive use of fat pointers everywhere to mitigate memory costs and keep everything on the stack. I can feel confident that if my solution compiles, it works. That Rust features test-driven development built into the language by merely adding a #[test] line above a test function to ensure that my logic is correct is even more comforting.
Yet there are many more arguments to make for Rust besides the memory safety. The memory safety is a side effect of the type system. The type system allows for more kinds of safety than just memory safety alone. It allows for compile-time checking of state machines, eliminating a large number of runtime checks at compile-time when you know how to take advantage of it.
Graphics APIs like OpenGL and Vulkan, for example, exhibit much cleaner APIs without the need for performing runtime checks to ensure that instructions are called in the right order, enabling faster execution out of the box.
Basically, taking a state by self will drop the original state, whereby you can return a new state in its place. Code completion will additionally aid the developer in only displaying methods that the current state supports. You can find examples in the hyper and vulkano crates.
In addition, Rust is heavy on the concept of data-oriented programming via protocol-oriented programming: traits featuring ad-hoc polymorphism akin to the likes of Haskell. This encourages more efficient programming practices than the object-oriented approach found in higher level languages like C++. Traits allow for a powerful form of generics whereby you can specify a range of input/output type parameters for all types that support the included traits.
The Iterator trait is by far one of my favorite features of Rust, and it is available without the standard library where it is absolutely vital. Basically, by implementing the Iterator trait for a type, which entails merely implementing the Iterator's next method, you gain access to all of the Iterator's adapters, which opens the door to all of Haskell's best features -- lazy functional programming, but without requiring a garbage collector and without using the heap. It boils down to very efficient machine code compared to if you had written it using a loop.
The sum types and pattern matching is also one of my favorite features of Rust. It's also key in the creation of powerful custom Iterators that may return multiple possible outputs, and it's one of the best parts of Rust's error handling strategy.
If a function may or may not return a value, then the return type is an Option which may either by Some(value) or None. Iterators return Options. If a function has a possibility of an error, then the return parameter is a Result which defines either Ok(value) or Err(error).
Enums may have their own parameters too, so pattern checking can become quite comprehensive to cover every possible result, which may have completely differing input parameters. One enum field could return an &str, another could return a usize, another could return an &str and a usize, and they can contain other enums with their own fields as well.
This then allows concepts like CoW smart pointers to be represented as an Cow::{Borrowed, Owned} enum in the standard library.
Anyway, language features aside, of which there's many more important features that Rust has to offer that'd take too long to explain, there's also the tooling that makes Rust powerful.
The rustup toolchain is one of my favorite things about installing and managing Rust on a system. It's trivial to install, and yet makes installation and updating stupid simple, regardless of your platform. I can easily tell others how to compile my software without them having to know anything about Rust or finding packages in a software repository.
rustup docs (opens offline version of Rust documentation website)
All from the same box. Cross-platform development made easy.
Then there's the powerful capabilities that cargo provides itself. It has a plethora of built-in subcommands, and exports a public API so that you may create and distribute additional subcommands. It automatically creates project hierarchies for you, even initializing git for you.
The Cargo.toml file allows you to define conditional compilation features for your project and libraries that you are importing. It allows you to define what dependencies that you want to pull and from where you want to pull them. Merely specifying a dependency name with a version number will have Cargo search Crates.io for the corresponding library and compile it when you build your project.
For binary projects, a Cargo.lock is provided which notes the exact version and hash of each library that you built to ensure that others that build your software will build it with the exact version libraries that you did when you released it. You only need concern yourself with dependencies on user systems when importing C libraries. If you build on Linux with the MUSL target, you can even build fully static binaries with zero dependencies, so long as you either don't use any C libraries, or build those libraries with MUSL.
Subcommands may even use their own tables in the Cargo.toml file to specify extra behavior, such as my cargo deb subcommand, which requires a few more fields to package Rust projects into Debian binary archives. Cargo provides many interesting subcommands, such as:
cargo build
cargo install
cargo run
cargo check
cargo update
cargo publish
cargo search
cargo doc
cargo edit
cargo test
cargo bench
cargo flame
cargo profile
cargo watch
cargo deb
cargo rpm
and probably more that I'm not yet aware of..
There's much to be said of Rust's community and documentation as well. Never before has a language had such comprehensive and ambitious effort as Rust into establishing an official community with a wide range of community resources. There's an official Rust Docs team that's continuing to add more documentation and examples for Rust and top crates in the Rust ecosystem; then there's a team dedicated to authoring official markdown-based books to teach Rust as the go-to free, printable book for Rust; there's an official Reddit thread, an internals discourse forum, a users forum, multiple IRC channels, and apparently quite a number of key developers browse through and comment on this domain as well.
> I wish the Rust evangelists would come to terms with the fact that to many developers memory safety is not a particularly important concern (for many different and often very good reasons) and talk more about other aspects of their language.
Seriously. However, I've seen a lot of comments to that effect from C and C++ programmers who don't want memory safety.
I come at it from a different angle: I'm a Java developer, so I already have memory safety. Pointer arithmetic, dangling pointers, and buffer overflows are not allowed, there is a garbage collector, and so on. In fact, most of the currently popular languages other than C and C++ are memory safe in similar ways. So, in terms of memory safety, Rust does not give me anything critical that I don't already have.
When choosing among the memory-safe languages for a real-world project, I'm going to need a pretty compelling reason to choose Rust instead of Java, Scala, Go, Ruby, Python, etc. -- especially when it is so much easier to hire developers for more popular languages.
Rust did a nice job with algebraic data types. Haxe has a very similar approach, and I've loved it since I first saw it. I think this is a very elegant way to define data structures. I listed plenty of complaints about Rust in my other comment, but this one stands out as a "single interesting thing" worthy of praise.
I see a lot of folks talking about this, but I live in a Rust bubble. From my POV a lot of the folks using Rust use it for primary reasons other than memory safety; many could afford to use python or something and get away with it. I suspect there are fewer blog posts about this, however.
I'll take a shot at some of the advantages:
Algebraic datatypes: Seriously. These are amazing. C++ has them with Boost's variant (and now std::variant), but they're awkward to use, making them somewhat a niche instead of the very central place they take in Rust programming. Till now ADTs were mostly the domain of functional languages, so I'm very happy that Swift and Rust giving them first class support and making them essential.
ADTs in Rust work with the enum keyword:
enum Shape {
Rectangle(Point, Point),
Circle(Point, u32)
}
let some_shape = Circle(Point::new(1, 2), 5);
match some_shape { // like switch, but with pattern matching
Rectangle(p1, p2) => println!("Rect from {} to {}", p1, p2),
Circle(p, radius) => println!("Circle at {} with radius {}", p, r)
}
An enum is essentially an "or" type; i.e. "a Shape is a Rectangle (X)OR a Circle". You are forced to handle this fact when you access this data -- if you try to access the stuff inside some_shape, you must match (or use a method that does this for you), and the match must be exhaustive (cover all cases). This is how null works in Rust -- if you want to say something is nullable, you use `Option<T>` -- `Some(T)` for when it's there and `None` for when it's null. You're forced to check for the none case if you want to be able to get to the inner data. You can call `.unwrap()`, but internally that's a method that just does a match and panics in the error case.
Traits: I mention this elsewhere in the thread, but Rust metaprogramming via generics cannot and will never beat C++ TMP's level of expressivity. Rust is going to get procedural macros which make it easier to fill in that gap of metaprogramming in a less hacky way, but if you just compare generics and templates then templates can do a lot more. However, that's not necessarily a good thing. The way templates are structured; they basically get "monomorphized" at something akin to parse time. This means that the error messages can be atrocious, and an API can never be self-documenting since you have to explain what kinds of types can be passed in. On the other hand, in Rust, if you get a function from a library you can pretend that it's a black box. The error messages will never mention the contents of the box. You can adequately figure out how to not make the compiler error out by looking at the type signature. The way Rust traits work is that you first define them:
Now, you can write functions (or types, or methods, or whatever) which accept these:
fn frob_something_10x<T: Frob>(frobber: T, frobbee: &str) {
for _ in 1..10 {
frobber.frob(frobbee)
}
}
If I did not have the `T: Frob` bound, I would not be allowed to write this method; the compiler would say that type `T` doesn't have a frob method, and I'd be forced to add a bound that gives it one. If a library user passed the wrong type to the function, the compiler will tell them that their type doesn't implement Frob, at which point they can add an implementation or use a different type or whatever. It's a very nice, clear API separation which leads to clear error messages and makes it easier to figure out what kinds of types to use. It also means that in the autogenerated docs I can just click on the trait to find out what types I can feed to the function -- in C++ many codebases have their own bespoke "trait" system using template specialization but it won't work with the docs and still lends itself to rabbit-hole-y error messages.
Moving: I personally find the lack of copy/move constructors and default construction to be very nice. In Rust, uninitialized types aren't a thing; initialization is always explicit. Copies are just copies, moves are also just copies, and moving is the default (except for POD types). There's no unknown overhead to deal with when I push to a vector, for example. Move-by-default and affine types IMO make it very easy to think about the program.
Thank you, this is exactly the sort of comment I was hoping to inspire, and it gave me some food for thought.
On first glance, I can see some utility in the ADT matching, but not necessarily something I miss in C++ (std::variant + templates gets me most of the way, although there are some issues with that approach). On the other hand, the clean separation of concerns and clear syntax of the traits example looks very useful indeed, and quite tempting.
Yeah, my point about std::variant is that it doesn't get used pervasively. With a few standardized macros for pattern matching we could get someplace, but right now, like with most new C++ features, codebases and libraries are very selective about the ones they use. This means that your natural C++ "style" can't rely on them, and not all libraries are written with the pattern in mind. OTOH in Rust everything is designed using enums, and any Rust code I write will be free to use enums. It works out quite nicely. In Rust it's not a tool you sometimes might reach for, it's a tool you reach for a lot, because most data is an "or" type.
Except not everyone is writing software remotely relevant to security. If the goal is to market Rust as a niche language for the security-conscious, that's fine, but there seems to be a drive to show that it's a superior general programming language. The safety argument doesn't do that for most people.
We are having new languages every year. Instead of debating which language is the best, why can't we invent a way to let components implemented in different languages talk with each other easily? We have pipes, sockets and message queues, but it's never simple enough to glue everything together.
The point of languages is not what you can do, but what you cannot do---making mistakes impossible so you can trust your own, and especially other's work (libraries).
Combining languages is likely to circumnavigate those prohibitions, defeating the purposes of the thrown-together languages. Your polyglot tower of babel is likely to fail.
Now I'd love to see a deep Rust<->Haskell bridge. But getting this right by not breaking the interesting invariants of either langauge is research-level work.
This is an interesting idea that has been explored before in VMS, an old operating system I believe competed with UNIX.
VMS had a feature called CLE (Common Language Enviornment) [1] which defined calling conventions for computing primitives (functions, registers, stacks...you get it) independent of any language. You could call bits of code from all sorts of languages like COBAL, FORTRAN, C, and some others I'm not really familiar with. Because the calling conventions were specifically designed for language interopperability in mind, VMS was implemented in several different languages. Different components were coded in whatever language best expressed them. This directly contrasts with Unix, which we all know champions C.
I'm not too familiar with Unix calling convention specifics, but as I understand, it revolves around C and its memory model. I believe this is what gives some languages difficulty "talking" with each other; if a language doesn't have a memory or execution model close to C's, it needs to translate through a FFI (Foreign Function Interface) [2] before exchanging execution routines efficiently.
I think Unix's pipes are better examples that how components could communicate. It's a pity that due to terminal limits, the best we can do about connecting components in Unix is to pipe things through.
Microsoft has sort of been working on this for decades. It started as Dynamic Data Exchange in Windows 3.x, then evolved into Object Linking and Embedding and Component Object Model, which in turn became the basis for ActiveX, Distributed COM, and COM+.
Reactions vary.
I believe GNOME was originally envisioned as providing a GNU framework for this kind of functionality (whence the "Object Model Environment" in the original acronym expansion), but I think those particular ambitions have mostly been abandoned.
The problem isn't that communication is hard, it's that semantics matter, we want different semantics between languages, and there is always difficulty bridging those gaps.
As a simple example, consider the JSON string
{"a": 36893488147419103232}
That's 2 to the power of 65. Let's decode that in Python. What do I get?
json: cannot unmarshal number 36893488147419103232 into Go value of type int64
What we're seeing here is a fundamental semantic difference between the languages and how they represent numbers. Go has a machine-level type representation that focuses on the memory being allocated. Python has a fundamental representation that implements arbitrary-precision numbers and can decode that without any fuss.
For bonus points, it's worth pointing out that while the JSON specification itself provides no limits on the size of numbers, it is generally unwise to use JSON numbers that can't be represented as IEEE 64-bit floats because there's a lot of languages that will get that wrong. I can also cause issues by sending large ints out of Python or Go into a language that expects floats to actually be floats, and then lossily decodes them in the process. The JSON spec theoretically doesn't have a problem here but the "real world" JSON spec is quite messy in this area.
My point here is not that these problems can't be solved. They all can be solved, at least on a case-by-case basis (two specific programs communicating with JSON). My point here is A: these problems exist B: these problems generally don't admit of practical generalized solutions (for instance, you'll find "programs must use BigInt libraries and programmers must understand all implications of that to use any JSON parser" isn't going to fly, and that would still be ignoring issues I could go on about for some paragraphs) and C: these issues are belligerent and numerous.
This is one minor issue in a corner of the JSON spec between two languages I happened to pick. This is not the exhaustive listing of such issues, this is merely one of thousands of examples you could construct between Python and Go alone. (In fact, it isn't even necessarily a mismatch between "the two languages" so much as "the two languages and the particular JSON parsing library", which means it's even worse than it sounds; I could rotate JSON libraries in either language and potentially get other issues!) Here's another thing that isn't really an issue so much as a meta-issue between all sorts of language pairings: How do you pass a value across different memory recovery types? That is, how do you pass a value from a GC'ed language and back to a non-GC-ed language? Bearing in mind that "GC'ed language" and "non-GC'ed" language are both themselves categories, and the details of both of those things matter a lot. Python and Go are both garbage collected, but you still can't pass values back and forth between them even if you jam them both into the same OS process!
You die the death of a thousand cuts trying to fix these issues between even two languages, then it gets worse if you try to pull more into the fold.
So what you end up with in practice is a protocol that is set at the OS level, writes into stone a whole lot of semantics that deeply, deeply affect the sort of code that can use them, and then those semantics bend the design of every language written on top of them. Right now, on Linux that language is C, Windows has C++ and a .Net runtime, and other people can pipe up with the base language of other OSes. So Linux has a ton of languages that, for all their glory and features and libraries, are ultimately just C with really, really pretty wrappers: Python, Perl, Lua, PHP, etc. Then these languages can communicate on the "C bus". The biggest exception I know of is Java, which is biggest runtime that become its own ecosystem without having an OS to go with it, so you get languages that are ultimately just Java (or JVM if you prefer) with really, really pretty wrappers: Scala, Groovy, Clojure, etc. This is the only solution that I'd say has every achieved any sort of scale, but you still get islands between the languages, the .Net island, the C island, the Java island, a lot of little islands from languages that have their own runtimes which are really cool but can't be expressed on "the C bus" very well, etc.
Personally I think one of UNIX's major problems right now is managing to escape from the "C bus". Back on the original topic, Rust is one of the most interesting stories I've seen in a while there; it can operate on that bus and even provide functionality, while using its type system to escape from the fundamental weaknesses that being on the "C bus" usually entails with memory unsafety, use of dangerous pointer semantics, etc. I think there's a distinct possibility Rust may be able to "bootstrap" us out of there in a way no other language has yet managed to.
As I said, all the problems can be solved on a case-by-case basis. But you can't just replace all numbers in JSON with big.Int, because you'll miss floats. You can't just use big.Float for all JSON decoding, because you'll trash performance, something that Go users care about more than Python users (because in Python you've already accepted that your code is going to be slow relative to C). You can solve each problem on a case-by-case basis, but to provide a general solution that makes everybody happy is impossible, which is what you want for this big ol' happy "let's just let everybody call any function they want" plan to work.
Rust seems to be more lightweight and portable in general. I think the compiler uses far less memory (and IIRC has better cross-compilation support too). The binaries are much smaller and the runtime is simpler. Cargo is amazing and I have much more confidence in it to not give me build trouble. In particular, you get a test framework including doc tests for free; I know Haskell tools offer similar functionality but in practice the setup cost is far higher. Haddock is not bad, but it uses its own weird syntax; Rust's documentation system uses Markdown IIRC.
Also, some things are just easier to write in an imperative language. I expect that the performance and memory use of Rust programs is also easier to predict and understand, though generally I think Haskell gets a lot of unfair criticism in that department.
I haven't used Haskell in a while so some of the things I mentioned may have been improved upon since then. In particular, Stack may have grown up a bit.
Probably not. There's been a lot of work put into Haskell design, and a lot of work put into Rust design.
A stand-out difference is that Rust put a lot of work into the borrow checker. But if a borrow checker is irrelevant (i.e. GC is fast/small enough), there isn't a huge reason.
(To be clear, there are a number of differences, e.g. strict vs. lazy evaluation, but whether one is better than another is debatable.)
The syntax for accessing records (dot notation) is a lot nicer in Rust in my opinion. This makes a huge difference in practice, since I don't have to have ridiculously long accessor functions. Though I haven't learned how to use lenses in Haskell yet, they're supposed to alleviate some of that pain.
I know you left out performance, but it's very nice to have code that can be easily profiled.
Using GSL gives adds some safety to pointers and memory allocation -- while providing the bare-metal performance that C is known for. (It still feels very low-level.)
To play the devil's advocate a bit, most of these are features you already get with C++, especially if you turn on all relevant warnings and treat them as errors. I can see the advantage of having things (sorta, given "unsafe") statically guaranteed for a shared codebase, but what are some compelling reasons to switch for personal projects?
What are you talking about? I don't get people making these posts without any detail whatsoever as to what they're talking about.
I have a fully static binary that includes futures, nix, tokio, uuid, and more. Even statically compiled with Muslims it's less than three megs.
And if I didn't use the nix package for a setsockopt call, it would be perfectly capable of running on Windows (and I could easily make it cross platform if I wanted to invest the effort in adding Windows support). Further, there are an abundance of great resources out there for extremely painless cross compilation.
So far I've not heard of problems with multiplat or bloat so I'm hoping you'll elaborate.
There are more systems than POSIX and Windows. Also, even in popular Linux systems Rust is not even installable in a straightforward way (e.g. Ubuntu 15.10, my system), so go figure. Regarding Rust bloat, I don't know if has got any better recently, but it was crazy bloated in comparison to C.
Rust statically links to libstd by default because it's not there as a dynamic lib on most platforms. On top of that, it links in jemalloc. This makes small Rust binaries look larger than they need to be. The overhead dwindles as you start looking at larger Rust programs. If you actually need to get rid of that overhead, the option is there, it's just not default.
Addition to popular package managers is being worked on.
That counts as a personal attack, and those are not allowed on HN—especially not against brand new users, which most new accounts belong to. Please don't do this again.
This comment breaks the HN guidelines by being uncivil. Please express your point substantively. If someone else is wrong, show them (and the rest of us readers) how. Then we all learn something. Also, please don't use uppercase for emphasis—that's in the site guidelines too.
Blog posts wont pull me away from C, tooling and docs will.