The reason I was moved to comment is that I think it's actually not that large a space (the one that Rust plays in). Instead, I think it's just that the applications in that space loom large in our consciousness, because we all use and pay attention to kernels and browsers. And I expect the space of applications "languages like Python" play in to grow over time, and Rust's to shrink, not just because computers will get faster but because the runtimes for those languages will get cleverer (see, for instance, what's happening with Javascript).
I would actually like to make the exact opposite prediction.
A generation of developers grew up on Python, Ruby, etc., rather than C, and they were all brought up on this notion that performance doesn't matter much anymore because computers keep getting faster. But the problems we're solving with computers keep getting harder (in terms of the amount of computation required).
These days, I can comfortably state that I prefer Rust over Python, even for simple programs. I prefer it because the type system is much more useful than Python 3 with mypy, the performance is extraordinary out of the box, the language is often just as expressive, and I rarely have any actual issues with the borrow checker. I also prefer it because I've had far too many experiences with Python where supposedly performance didn't matter... until it did and Python was no longer appropriate.
I think we're entering into a generation of languages where things like Rust are going to start really seriously displacing C++, and as more people learn them they're even going to start displacing supposedly easier languages. Right now, few folks are going to learn Rust as an alternative to Python. But once they're exposed to it for other work and realize how productive they are, there's just going to be less of a need to reach for Python over time.
What is the thing happening in computing making the largest demands on compute performance? Data science and ML. And what's the lingua franca of data science and ML? Not even Java or Go, the managed compiled languages; no, it's Python.†
You can see the phenomenon I'm talking about happen right now, in every HN thread where we push against the ocean trying to keep Electron from being a thing. What's going to happen first: Electron's runtime will become clever enough that its performance is tolerable even to nerds, or most Electron applications being rewritten in a language that forces programmers to manually manage memory?
It's interesting, because Rust will probably be a part of the story that makes things like Electron tolerable! But the point isn't that Rust is a doomed language (it's not, it's going to be durably important), but rather that it's not going to be the language most programmers work in.
† It's not pure Python, of course, but that's part of my point.
> What is the thing happening in computing making the largest demands on compute performance? Data science and ML. And what's the lingua franca of data science and ML? Not Java or Go, the managed compiled languages; no, it's Python.
While ML might be the most demanding area in terms of computation, it's also a very small part of what developers in general do. My claim/prediction is more based on the notion that everything is becoming more compute-intensive.
Where I work, we've probably incurred many, many millions of dollars of expenses (primarily hardware) due to the choice to write a lot of software in scripting languages over the years. We made that choice because C++ was so unwieldy in comparison that we figured we'd end up spending the same amount on C++ but in different ways. If we were starting from scratch today, I doubt we'd use scripting languages like Python for hardly any of our software, because there are choices available now that just didn't exist in the past.
I'm not saying there's no place for something like Python, nor am I saying existing Python stuff should be rewritten. I think Python is going to continue being an extremely popular and important language. My claim/prediction is more that in the future we'll increasingly see Python chosen less for completely greenfield projects because there are now options that are high-level, well-designed, and extremely fast, and at least in my experience that has a lot of business value that is going to be harder and harder to ignore over time as it becomes more obvious.
Candidly, there once was a vocal subculture of developers that would aver as to how much more comfortable they were in C than in Python (or Tcl or Perl), and, I'm told, in the long long ago there were them that said the same thing about assembly language. As an engineering manager, your job, if done well, would be to mollify those developers without conceding any ground to them, because the higher-level language was practically always the better option.
People will burn money on compute no matter what language things are built in; they'll ship on Fargate instead of EC2, on-demand instead of reserved instances, RDS instead of running their own Postgres, Postgres instead of just using Sleepycat, anything at all instead of running their own iron, and on and on down the list.
(All of this is out the window if you're shipping a browser! Or a hypervisor! Or an OS kernel! Or a new router! Or a AAA game!)
Most of the time, the most expensive thing you can burn is an hour of programmer time. And, with your fluency in Rust and other languages, you must see immediately why someone would believe Rust is more time-consuming than, say, Java, let alone Python.
Paul Graham got this part, at least, right in "Beating The Averages", when he noted how much more worrisome a competitor would be if their job reqs were for Perl programmer than if they were for C developers. The Perl company is actually getting lots of stuff done.
You have two premises here, one explicit, one implicit:
(1) [explicit]: a higher level language is practically always the better choice.
(2) [implicit]: Rust is not high level (or at least not compared to Python).
Our biggest disagreement is with (2). Rust is actually higher level and more expressive than Python in many ways. Of course, it also exposes some lower level concepts like lifetimes and ownership that Python hides.
Your comparison doesn’t apply because there isn’t a simple hierarchy into which Rust and Python fall. Each language is both higher- and lower-level than the other in a number of ways.
I've written a lot of Python (thankfully not so much anymore), and Rust is part of my day job. I am struggling to think of a circumstance in which something would be easier to write in Rust than in Python, factoring out "performance" --- just comparing the languages themselves. Maybe, you can provide examples, and we can see if we're working from the same premises.
> I am struggling to think of a circumstance in which something would be easier to write in Rust than in Python, factoring out "performance" --- just comparing the languages themselves.
Most Python backend projects I've worked on would have been better off written in Rust (had it existed) the same way most of the Javascript projects I've worked on would have been better off written in Typescript.
Maybe it's just the projects I've worked on (or their size) but maintenance and refactoring to deal with growth of or changes in business requirements were by far more time consuming than actually writing code. The amount of time lost on onboarding and writing trivial tests that could have been handled by the Rust compiler dwarfs true R&D time, let alone time spent debugging bugs that could have been prevented by encoding business logic into the type system.
At my current job I often still pull out Python to prototype logic or Typescript to spin up a quick and dirty intranet app but for the stuff that needs to last, just work, or be maintained into the future, I often find myself yearning to use Rust. The last obstacles are organization talent pool and Rust's ecosystem.
As much as I'd love to use more performant APIs and programs that crash less on my daily basis, I don't see Rust even close to overtaking Python.
For starters, memory management in Python is trivial while in Rust the developer has to learn a whole new set of skills in the form of borrowing, lifetimes and ownership.
And for large projects where Python's limitations hamper team coordination, GC'd languages such as C# and Java offer much softer learning curve and a mature ecosystem. Even when performance is critical, projects get by just fine with using Java like in High Frequency Trading shops.
Perhaps we shouldn't settle for Rust but refine the idea in a new, more ergonomic, language with focus on achieving the same but with lower cognitive load. Faster compilation times wouldn't hurt too.
I rarely use Rust for the performance or memory safety - it's a nice perk that allows for data structures and algorithms I wouldn't otherwise use but it's not the main reason I reach for it. I use Rust for its type system including traits, phantom marker types, affine types, sum types, exhaustive pattern matching, and so on. Once you get used to traits and composition over inheritance, it allows writing very reusable code that enforces a significant fraction of logic at the compiler level and scales much better with code and team size (modulo dependency hell, which I think is language agnostic but a work in progress in Rust). Rust's model of wrapping unsafe code in safe types also makes working with the world of C/C++ libraries much more pleasant once you get the hang of FFI.
Plenty of other languages have these features but not quite in the right combination or their communities are too small and they haven't caught on yet. Rust is, in my opinion, at an organizational and technical sweet spot for large projects.
I do not believe this is a statement that I can reply to in a meaningful way. You definitely believe Rust is much harder or more complicated than I think it is, and that’s basically where we disagree.
There’s not any particular way I can respond to this, because there really isn’t any substance except to attempt to link me to something you find distasteful/annoying. I use both languages professionally on a daily basis and it is on that basis that I make my claim.
Here’s one way I could respond. There are numerous things in Python that I find more difficult than in Rust. For instance, I think async/.await in Rust is generally easier than asyncio is in Python, and in the former case the compiler is quite good at detecting various concurrency bugs whereas you’re entirely on your own in Python.
It’s far easier to write multicore programs in Rust than Python thanks to the lack of a global interpreter lock.
Python makes extensive use of runtime exceptions for basic error handling, which I find very exhausting to reason about and account for during development; tools like mypy are also unable to help there at all, and I must rely on the questionable quality of documentation or just scavenging through code.
Pythonic things like None-equivalent values often make it impractical to use Optional values as return types in idiomatic code, thus pressuring you to use more exceptions in places where one would just use Option or Result in Rust.
Python has poor support at best for sum types, which I tend to use extensively for almost every problem space.
Python does not support pattern matching, though some support for this has been proposed for a future version of the language.
Handling dependencies generally is much easier in Rust as well thanks to cargo.
Perhaps you think these are all just the concerns of “FP nerds,” but for me these differences often mean I can write correct, high quality backend services in Rust faster than I can in Python — and the result is often 10x-100x faster with no optimization work whatsoever. That seems worth the relatively small amount of time I spend worrying about lifetimes or ownership or the fact that I use map/filter/iterators instead of list or dictionary comprehensions.
> You can see the phenomenon I'm talking about happen right now, in every HN thread where we push against the ocean trying to keep Electron from being a thing. What's going to happen first: Electron's runtime will become clever enough that its performance is tolerable even to nerds, or most Electron applications being rewritten in a language that forces programmers to manually manage memory?
Browsers already have an immense amount of effort poured into making them fast. They will get faster, but not by much.
But then, JavaScript was never the problem with Electron in the first place. Or at least, not the fact that UI code is written in JavaScript. After all, a lot of the people complaining about Electron (me included) are Mac users pining specifically for native Cocoa apps. Which, up until recently, were all written in Objective-C – a language which is much slower than JavaScript! Even Swift is not necessarily faster.
Rather, from a performance perspective, I think the biggest problems are (1) the DOM being a terrible abstraction to design user interfaces on top of, and (2) the web forcing you to use JavaScript for everything rather than letting you drop down to native code for performance-critical bits (other than what is implemented by the browser itself).
"I can comfortably state that I prefer Rust over Python, even for simple programs"
That's fine but that will never be generally true because Python is a high level language that will always have 10x more developers using it, including your Data Science prof, the high school kids learning 'basic programming', and all the AI researchers.
And the tons of devs doing stuff for which Python just makes more sense.
I don't think it's appropriate to call a language with an advanced and useful type system, expressive syntax out of box, and reasonable macro support a "low level" language.
Python, in comparison, lacks a lot. But it has a very low barrier of entry: the syntax is English-like, you can start doing something with very little knowledge of advanced features, and everything is easy to inspect and play with in a REPL.
Respectfully, I feel like you're dancing around what Python has that Rust does not. It's not the "English-like syntax" (with its invisible syntax errors). It's that Python, like most modern languages, manages memory automatically, and Rust does not.
Rust doesn't for good reason! Not just zero-cost abstraction, but also, because for the applications it's designed for, it makes controlling memory much easier. There are ways in which writing a Rust program is probably easier than writing a Java program, in Rust's stomping grounds. It's just that it doesn't make things easier anywhere else.
I don't disagree with your premise, but I disagree with the claim that Rust doesn't automatically manage memory. Rust absolutely does automatic memory management - that's one of the main motivations behind the ownership/lifetime system. For instance, when you create a vector, the standard library will automatically allocate, re-allocate, clone and/or deallocate the buffer when necessary.
What Rust doesn't do is to decide for you how to pass or share that memory between different parts of your program - it just provides you with options. One could argue that this should also be under the automatic memory management -umbrella, and I wouldn't necessarily disagree. But I don't think it's fair to say that Rust doesn't have automatic memory management.
I would also argue that for many simpler programs you can get away with basically paying no attention to memory management. If your program is a simple pipeline which reads some input, applies some processing and produces an output, your code might not look substantially different from any GC-based language. That doesn't mean Rust is an easy language or even a good idea to use for these kinds of programs, but I would argue that a developer proficient in both Rust and Python could write the same (simple, pipeline-like) program in either language with comparable time and effort.
Rust is a high enough level programming language to have a GC as a library. You can go with Rc and Arc all over the place, and have a Python-style ref-counting GC.
OTOH Go is a lower-level, less expressive language with a built-in GC memory management.
To me, the level of a language is the level of abstraction one can comfortably reach with the language, and not being close to the metal or not.
