These threads always devolve into "rust is too slow" written by developers (or enthusiasts) that have never written no_std code in production. I've written and shipped firmware for embedded devices written in rust, yes, still using cargo and external crates, and had zero issues with compile time because the nature of the dependencies in the deps tree is different and very carefully curated.
Anyway, I really just wanted to point out that from the mailing list we have Linus and Greg endorsing this experiment/effort from the Linux side and a commitment from Josh on behalf of the rust team to grow the language itself with the needs of the kernel in mind. That's quite impressive and more than I could have hoped for.
I've actually played with writing kernel code in rust - for Windows/NT, however - and it's quite weird to be able to use such high-level type constructs in code where you typically manually chases pointers and wouldn't be surprised to see statically allocated global variables used to monitor reference counts.
Linus has commented on Rust twice (that I'm aware of).
First, back in 2016:
Q: What do you think of the projects currently underway to develop OS kernels in languages like Rust (touted for having built-in safeties that C does not)?
A: That's not a new phenomenon at all. We've had the system people who used Modula-2 or Ada, and I have to say Rust looks a lot better than either of those two disasters.
I'm not convinced about Rust for an OS kernel (there's a lot more to system programming than the kernel, though), but at the same time there is no question that C has a lot of limitations.
People have been looking at that for years now. I’m convinced it’s going to happen one day. It might not be Rust, but it’s going to happen that we will have different models for writing these kinds of things.” He acknowledges that right now it’s C or assembly, “but things are afoot.” Though he also adds a word of caution. “These things take a long, long time. The kind of infrastructure you need to start integrating other languages into a kernel, and making people trust these other languages — that’s a big step.
I was once in the same room as him at a Linux Foundation event, and really wanted to ask him about it, but also didn't want to be a bother.
Note that the C++ opinion everyone cites is from 2007. I don't know how he feels about C++ today. It seems he's using it for some things at least. I know I've changed a lot since 2007, and so has C++.
I don't know Linus's reasons specifically, but our presentation at Linux Security Summit last year laid out why we think that Linus's past objections to C++ don't apply to Rust. See slides 19-21 of https://ldpreload.com/p/kernel-modules-in-rust-lssna2019.pdf .
His previous objections were:
In fact, in Linux we did try C++ once already, back in 1992.
It sucks. Trust me - writing kernel code in C++ is a BLOODY STUPID IDEA.
The fact is, C++ compilers are not trustworthy. They were even worse in
1992, but some fundamental facts haven't changed:
- the whole C++ exception handling thing is fundamentally broken. It's
_especially_ broken for kernels.
- any compiler or language that likes to hide things like memory
allocations behind your back just isn't a good choice for a kernel.
- you can write object-oriented code (useful for filesystems etc) in C,
_without_ the crap that is C++.
In brief, Rust does not rely on C++-style exception handling/unwinding, it does not do memory allocations behind your back, and its OO model is closer to the existing kernel OO implementation in C than it is to C++'s model. (There are other good safe languages besides Rust that I personally like in general but do not satisfy these constraints for this particular use case.)
Is the following code from page/slide 64 supposed to be a talking point?
FFI: calling C from Rust
extern {
fn readlink(path: *const u8, buf: *const u8, bufsize: usize) -> i64;
}
fn rs_readlink(path: &str) -> Result<String, ...> {
let mut r = vec![0u8; 100];
if unsafe { readlink(path.as_ptr(), r.as_mut_ptr(), 100) }
...
The example doesn't bother using r.capacity(), which is exactly the kind of poor code hygiene leading to overflows you would see in bad C code--i.e. rather than using the sizeof operator, manually passing a literal integer, simple macro, or other ancillary variable that hopefully was actually used in the object's declaration.
Also, notably, the presentation says one of the desirable characteristics for a kernel language is "Don't 'hide things like memory allocations behind your back'". I'm not very well versed in Rust, but when I write a small test program using vec![0u8; 100], the buffer is heap allocated. Which would be a hidden allocation in my book. I realize these are just introductions, but if you're pitching Rust code for the kernel, it seems disingenuous not to show the style of Rust code, presumably much more verbose, which would actually be used in the kernel. Constructs like boxes can't really be used, at least not easily. C++ was disliked because the prospect of fiddling and debugging hidden allocators is nightmarish. And to the extent Rust's type system relies on boxes and similar implicit allocation for ergonomics, some of the type safety dividends might be lost. How much would be lost is hard to judge without showing the kind of actual code that would be required when used in Linux. Toy Rust OS experiments don't count, because they can just panic on OOM and other hard conditions, and don't need to wrestle with difficult constructs like RCU.
`Vec` is always heap-allocated; it's not hidden if you know that. It's intentional that you have to write `vec![0u8; 100]` to get this, rather than just, say, `[0u8; 100]` (which will give you a stack allocation).
It's true that the example should use capacity() rather than hardcoding 100.
If you want a stack allocation, here's an example:
It's about the same length as the original, but it's hard to make a direct comparison. On one hand, the original snippet is totally wrong: it doesn't properly handle the nul terminator of either the input or the output. I fixed that, but at the cost of making the code a bit more verbose. On the other hand, the original snippet does try to parse the result as UTF-8, which is not what you want, so I removed it (it would add 2 lines or so).
That said… it's not true that avoiding heap allocation is what would actually be done in the kernel. Not for file paths, which is what readlink typically returns. Since kernel stack sizes are so small, paths have to be allocated on the heap. It's only in userland that you can plop PATH_MAX sized buffers on the stack.
On the other hand, if you were really doing heap allocation in the kernel you would need to handle out-of-memory conditions. Rust's standard library doesn't support recovering from OOM at all, but it's not like you'd be using the standard library in the kernel anyway. In contrast, Rust the language is actually quite good at this. A fallible allocation function would return something like Result<String, OOMError>; the use of Result forces you to handle the error case somehow (unlike C, you can't just assume it succeeded), but all you need to do to handle it is use the `?` operator to propagate errors to the caller.
I realize Rust-the-language can handle OOM, and that with Result types it could prove cleaner and safer than in C. But it's rare to see examples of this, and the wg-allocator working group still seems to have alot of unfinished business.
> In brief, Rust does not rely on C++-style exception handling/unwinding, it does not do memory allocations behind your back
It kind of does a little bit. Panicking is implemented via the same machinery. But of course code is not expected to panic unless things are terribly wrong (ie. kind of like the existing kernel panic thing). It also does sometimes allocate things, but it is true that Rust is a lot more explicit about this - ie. you have to call clone() or Box::new() etc.
`no_std` environments are different. They don't allow allocations unless you explicitly add an allocator. You also have to define the panic handler yourself.
Generally `no_std` libraries will not allocate or panic themselves. There is still the possibility of panicking (e.g. out of bounds array access) but there are alternatives that don't panic if that's a concern even with a custom handler.
Yeah, and it shouldn't be too hard to hook up existing nightly Rust to use the Kernel's panic functionality; it already supports user-overridable panics for no_std, iirc
#include <iostream>
void p(const std::string &x) {
std::cout << x << std::endl;
}
int main() {
p("hello");
}
Neither p, which takes an already-allocated std::string, nor main, which uses a string literal, explicitly/obviously requests dynamic memory allocation. Yet it happens.
In userspace, that's not a huge problem - even if allocation requires asking for more memory from the kernel, which requires paging out some dirty files to disk, which blocks for a few seconds on NFS, that's fine - the kernel will just suspend the userspace process for a few seconds while all that happens. Inside the pager or NFS implementation, though, an unintentional memory allocation can deadlock the whole system.
In Rust you'd have to write .to_owned() to do the equivalent, and it would be obvious that doing it in code that might be responsible for freeing up memory is a mistake.
Something as innocuous as `T x;` or `T x = y;` will allocate in C++. In Rust, the equivalents are `let x = T::new();` and `let x = y.clone();` which are much more obvious as potential allocations.
> Something as innocuous as `T x;` or `T x = y;` will allocate in C++
I think saying these will allocate is wrong. They can allocate, but only if T allocates and manages memory, which depends on what T does.
I am skeptical of the more general claim. Anything but a truly cursory skim of the code is going to tell you that these lines are constructing a new object, regardless of the syntax, and to actually know whether it allocates memory, you need to know more about T, which is true in either Rust or C++.
Unless someone implements Copy on a type that allocates in Rust (which you shouldn't do; Copy is specifically for types that are cheap to copy), you really won't get implicit copies, though.
That means in any reasonable code, `let x = y` won't allocate in Rust while `T x = y` could in C++. `f(x)` in Rust won't either for the same reason. And that's on top of how C++ will implicitly take a reference, so even if you know x is a heavy object, you don't know if `f(x)` will be expensive (or if `f(move(x))` would be helpful).
It's just not exactly equivalent. C++ is more implicit and has less baked-in good practices like `Copy` vs `Clone`. The indirection is often shallower (less often I have to inspect a function prototype or type I'm using for specifics), and I find that very useful.
Minor clarification: afaik it's impossible to implement Copy such that it would allocate. Unlike Clone, the Copy trait is only a marker trait which doesn't have methods, so you cannot customize its behavior. It will always be a bitwise stack copy.
This is not minor. In Rust "x = y" specifically only copies or moves bytes, no magic involved, no code is run except something like memcpy.
- If the type isn't marked Copy, it's a bytes move (which will very probably be optimized out).
- If the type is marked Copy (which is possible only if all its subtypes are also marked Copy, which allocating types are not) then it's a bytes-for-bytes copy.
As soon as you need something more involved, then you have to implement (or derive for simple types) Clone.
The easiest to understand example (for me) is that you can manually implement polymorphism support using function pointers that take in a struct instance as an argument. Polymorphism is fundamentally just a method call that's dependent on which target object it's being called on, so this pattern allows you to pass in the target object (as a struct since it's C) explicitly.
From the article:
> Some simple examples of this in the Linux kernel are the file_lock_operations structure which contains two function pointers each of which take a pointer to a struct file_lock, and the seq_operations vtable which contains four function pointers which each operate on a struct seq_file.
The huge and important difference is when you use this pattern in C the compiler has no idea what you are doing and can't check anything, nor can it reduce the cost of this dispatch mechanism. A compiler for a language where this is a feature can check what you are doing and de-virtualize virtual dispatch. In C you pay the highest possible cost for polymorphism while enjoying the least benefits.
> The huge and important difference is when you use this pattern in C the compiler has no idea what you are doing and can't check anything
Function pointers exist in C and are type-checked by the compiler just fine.
> A compiler for a language where this is a feature can check what you are doing and de-virtualize virtual dispatch.
Virtual dispatch is pretty much only used in the kernel in places where virtualization is necessary. Devirtualization is pretty rare in C++ in practice (I don't believe I've ever seen a "virtual final" method, and my day job is on a large C/C++ codebase).
> Function pointers exist in C and are type-checked by the compiler just fine.
This amount of type safety is almost useless. Many of the function prototypes, for example in inode and dentry ops, have the exact same signature. If you accidentally swap inode.rmdir with inode.unlink, compiler isn't going to say anything. And it won't be caught in code review either, to the extent that Linux even has code review culture.
> I don't believe I've ever seen a "virtual final" method, and my day job is on a large C/C++ codebase
It's common in my experience, for performance-sensitive code.
I don't think that kind of optim is usually important in a kernel, especially one architected a lot with dynamical modules. Plus, on the convenience side, you can also use other models (e.g. have some function pointers at instance level) than the one of C++ in ways that do not look like completely different when reading the source.
Devirtualization is mainly useful to cope with some C++ self inflicted wounds. Linux obviously never had them the first place.
I'm really not a compiler expert, but is this optimization common in C++ for example? I thought all virtual methods incurred the cost of a vtable lookup
Just slapping the virtual keyword on some function doesn't do anything. Overrides may or may not happen via vtable. Often compilers can figure out how to dispatch at compile time and don't need the vtable.
In Linux yes, however Arduino, MBed, Android, iOS/macOS and Windows drivers tend to be written in a C++ subset, and Swift, Java and .NET are also an option for userspace drivers.
I got your point, and naturally we are in the context of Linux kernel, however I like to make a point that not every OS constrains themselves to C and having been embracing more productive options for a while now.
Anyway, I really just wanted to point out that from the mailing list we have Linus and Greg endorsing this experiment/effort from the Linux side and a commitment from Josh on behalf of the rust team to grow the language itself with the needs of the kernel in mind. That's quite impressive and more than I could have hoped for.
I've actually played with writing kernel code in rust - for Windows/NT, however - and it's quite weird to be able to use such high-level type constructs in code where you typically manually chases pointers and wouldn't be surprised to see statically allocated global variables used to monitor reference counts.