Our zlib-rs project implements a memory-safe and performant drop-in replacement for zlib, a widely-used data compression library.
@folkertdev shares the status quo of zlib-rs, including the good news that performance for the highest compression level is on par with the zlib-ng fork of zlib.
Read the blog for all the details:
https://tweedegolf.nl/en/blog/134/current-zlib-rs-performance
We already use runtime CPU feature detection, e.g. here:
The advantage of `target-cpu=native` is that those branches are compiled away, because it is statically known what features are available and hence which path will be taken.
Runtime CPU feature detection has a performance cost, and we're still looking for the most performant way to do it, but it totally works.
Cargo subcommand to build multiple versions of the same binary, each with a different CPU features set, merged into a single portable optimized binary - ronnychevalier/cargo-multivers
@tweedegolf @folkertdev @trifectatech
Folkert, thanks for a really informative and well-written post. it looks like `zlib-rs` is coming along fantastically well!
The point you make about "bounds checks are just a bunch of correctly predicted branches that don't cost much wall clock time" deserves a followup post, maybe? I've heard it before and believed it, but it's good to emphasize to those less familiar and afraid of the costs of checks: having hard evidence really carries the point home.
@tweedegolf @folkertdev @trifectatech
Further, you might try turning on overflow checks and running the benchmarks again? Everybody is afraid of turning on overflow checks in release mode, but my suspicion is that the situation is really similar: I'd like to see these checks be on by default in release mode someday, although that ship may have already sailed.
Looking forward to watching the ongoing `zlib-rs` work. Thank you and your co-devs and sponsors for this!
@po8 @tweedegolf @trifectatech
yes, I ran some further benchmarks.
First, we can disable bounds checks in the rust compiler (see below for how) and see whether that gives any speedup. It turns out that there is no significant effect, but the other performance counts are interesting.
Next, overflow checks do have a cost, about 3% in my measurements. That's too much for our library, but totally worth it for most production applications (that do less number crunching).
https://gist.github.com/folkertdev/23a92e853eb78e9a66829b71a276bd4b
@tweedegolf @folkertdev
There is a typo you may want to fix in the intro ("Trifecata").
It's funny to note that the typo would change the Latin meaning to something like "three-times shitted", while later in the post you show a benchmarking tool called "poop". 💩😀
Tweede golf
Fabio Valentini
Folkert de Vries
flamion 
Bart Massey
bondolo
Erik Jonkers
Luca Bruno