Hacker News new | past | comments | ask | show | jobs | submit login

I'm the author of https://github.com/dropbox/rust-brotli and I certainly ran into the issues mentioned in the article, and the initial version of my brotli decoder and encoder were each almost 10x slower. I also worked around each and every one of them, and the result is something that performs at 80-90% of the speed of the original on average, and some files compress faster with rust than with the original brotli codec.

allocator: I ran into the same situation and abstracted it with a generic allocator: https://github.com/dropbox/rust-alloc-no-stdlib it's an ugly solution, but it works and can allow for significant perf improvements down the line by bundling all same types together

benchmarks: I simply made a test that printed the time

primitive types: it's easy to write generic functions that do this

macro system: I found to be amazing, but I never used it to compact data types

borrow checker: split_at_mut was super helpful...also core::cmp::replace was useful to taking away pieces of a structure and manipulating them, then putting them back

Though it wasn't all roses: to try to get SSE vectorization I had to convert this 6 line short string matching function https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d...

into this multi hundred line monster with macros and so forth https://github.com/dropbox/rust-brotli/blob/238c9c539b446d7d...

but in the end it was as fast as hand-tuned intrinsics in C

I also found myself scared by dependencies, including onto the std library, so I tried to get everything to remain within the core library. This should allow something as low-level as a codec to be used in kernel space or in another place where a custom allocator is needed.

I also found that "rewriting in safe rust from C" was made easy by corrode https://github.com/jameysharp/corrode since C and rust can be interface compatible, it's easy to go one file at a time and turn it into working rust, then safe rust.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: