The State of SSL Stacks

The SSL landscape has shifted dramatically. In this paper, we examine OpenSSL 3.x, BoringSSL, LibreSSL, WolfSSL, and AWS-LC with HAProxy.

HAProxy Technologies

as the site is asleep, I'll offer the tldr: aws-lc is the fastest, wolfssl is number two, openssl is not very fast

but read the post later, it is super long and detailed

"in a world where everyone is striving to reduce their energy footprint, sticking to a library that operates at only a quarter of its predecessor's efficiency, and six to nine times slower than the competition, contradicts global sustainability efforts"

@bagder yeah, everything you can possibly imagine to do wrong, they did wrong. https://github.com/openssl/openssl/issues/18317

It's difficult to show all the degradations in a reproducible single-threaded benchmark, so I never updated that ticket. But in a multithreaded workload, their heavy reliance on malloc and all the locking it requires interacts badly with every other need for synchronization in a process.

@bagder not only did they kill their multithreaded performance by making every crypto mechanism dependent on malloc, they also made it impossible to safely free all of the library's allocated memory. https://github.com/openssl/openssl/issues/25294#issuecomment-2312841205

It's like the entire OpenSSL dev team got replaced by a bunch of recent CS grad students.

Race condition on error strings cleanup? · Issue #25294 · openssl/openssl

This report is inspired by latchset/pkcs11-provider#429 (comment) and latchset/pkcs11-provider#429 (comment). This report is based on my understanding of the issue and may not be correct, we didn't...

GitHub
@bagder I am not sure going PQ hybrid is an energy saver 😜
@letoams @bagder it's not the cryptography costs that are the burden in OpenSSL 3. It is the overhead that they introduced.
@bagder did we hug it too hard?
@pearofdoom @bagder that would very much be funny, seeing that the ha in haproxy is for "high availability" 😳

@bagder

> but read the post later, it is super long and detailed

Open site: « 53 minutes read ». « Super long » is almost an euphemism at this point!

@bagder thanks for the tl;dr! If this was an academic paper, it would be desk rejected...
@bagder TBF, a big part of the article is a deep dive into the finding of where the performance hit lives, which I happily skipped over. That cut the reading time in two. Talk about performance :)
@bagder The website seems down. Hope no fedi-ddos happened
@Jbb @bagder which would be a bit ironic for the website of a highly scalable load balancer software :)
@bagder grml… it mentions LibreSSL only at the beginning but doesn’t consider it later or elaborate more on it (it’s my current goal for lower- to low-end scenarios)
@mirabilos We consider that LibreSSL is a really good implementation for small use-cases, but it wasn't performant enough nor complete enough to be considered as a replacement of OpenSSL for most haproxy users. However we will still support this library and will be happy to improve its support within haproxy.
@sekh ah ok, fair; thanks
@bagder while I understand the reasoning, the position that modern libraries should support tls 1.0 doesn’t sit right with me. Especially with no caveats about limited use cases.
@kalfeher
Hopefully TLS 1.0 will disappear, but there are still cases where people want to use a modern HAProxy stack in front of a proprietary software which only handles TLS 1.0 for example. HAProxy is a swiss army knife which handles that kind of cases, and make people life easy in hostile environment 🙂

@sekh ppl are free to design software as they wish. But requiring support for something deliberately removed from the protocol 4yrs ago is something that may end up hurting consumers of the service.

Their choice, but it’s a short sighted choice imho

@kalfeher obsolete protocols and softwares are part of a lot of ecosystem. Unfortunately deprecating something doesn't make it disappear, but rest assured, TLSv1.0 and 1.1 are disabled by default. Like we said in the article, that's to "accommodate diverse client requirements while encouraging migration to more secure protocols.". Most openssl drop-in replacement still implements TLS1.0, so that's not really an important point. It's not about designing a software, switching this on is basically 2 lines of code.
@bagder omg this is so verbose... does it say anything that wasn't said by https://github.com/haproxy/wiki/wiki/SSL-Libraries-Support-Status ?
SSL Libraries Support Status

HAProxy's Wiki. Contribute to haproxy/wiki development by creating an account on GitHub.

GitHub
When some of the project members considered a 32% performance regression “pretty near” the original performance
Damn that sounds bad, lemme click on the link
....
Oh they had 1146% regression before...
well, ok, it is "pretty near" by that scale. Not near enough, but pretty near
3.0 performance degraded due to locking · Issue #20286 · openssl/openssl

Using some internal QUIC tests, we see this timing in 1.1: $ USE_GQUIC_VERSIONS=1 perf record -F 999 -g ./quic_lib_tests --gtest_repeat=100 --gtest_filter=*ZeroRttDisabled* $ perf report + 31.73% 3...

GitHub
@bagder Not performance but security related; here's a CVE from 2019 that remains unfixed. OpenSSL finally acked it in 2024 as "feature request", with no plans to fix: https://github.com/openssl/openssl/issues/24528 It affects Windows. Forks fixed it throughout the years.
Windows, world-writable config path (re: CVE-2019-5443) · Issue #24528 · openssl/openssl

This is a long time problem, and opening a new Issue to give it some visibility and place to discuss. OpenSSL loads its config (openssl.cnf) from a disk location baked into the binary at build time...

GitHub

@bagder "In SSL, performance is the most critical aspect."

Surely it's the *second* most critical aspect?