Rustls Outperforms OpenSSL and BoringSSL
memorysafety.org
memorysafety.org
Testing on an intel processor, with frequency scaling disabled, which will adversely affect non AVX-512 more than AVX-512 stuff due to the limited boost available when using this. I'm pretty sure this is a not totally fair comparison, and tuning the box to give your solution an advantage rather than tuning it for each solution to give optimal performance would be more realistic.
However, i'm not knocking it, sounds like a great achievement, and it'll spur the other solutions on to improve their implementations which is a win all round.
As a side-note, I believe the CPU we tested this on does not suffer from the AVX-512 power limits reported with earlier AVX-512 parts. https://travisdowns.github.io/blog/2020/08/19/icl-avx512-fre... seems to confirm that.
But it just makes me sad.~
Edit: Based of this comment [0] and replies, it appears I've misunderstood what 'license' means. My apologies
Would be nice to see the non-AVX512 results.
Only if you want good performance. If you're doing a lot of networking, you want your userland socket servicing pinned to the same CPU that the kernel is using for that socket. Which is easiest to achieve if you cpu pin nic queues and server threads. (If you want really good performance, you might want to skip userland with sendfile + kTLS or nic TLS, or maybe skip the kernel with userland networking)
For a benchmark, cpu pinning and tight control of the system is a clear prerequisite; you want to maximize repeatability, and user threads bouncing around cpu threads leads to less repeatable results.
The project is the best one for use on the internet with modern SSL standards, however.
I'm no longer blocked on this particular issue that I filed on behalf of my work at Deno, but they aren't interested in adding less-secure suites that may be required by certain server configurations, but still appropriate for traffic that isn't general web-use.
https://github.com/rustls/rustls/issues/1607
At some point I had a list of suites required to connect to some older versions of MySQL/Microsoft SQL Server, but again, no longer blocked.
For server-to-server use where I don't control one end of the equation, I stick with the OpenSSL crate. If there's potentially older servers in the mix, I'm OK with using rustls as a backend for things like reqwest, but it'll be openssl for servers for now.
I understand the philosophy, but rustls is never going to be an OpenSSL drop-in until this approach changes.
Semi-related, I now avoid native-tls because MacOS + gatekeeper + weird JAMF configuration makes that library completely unreliable in the wild.
I suspect most of the team would tell anyone "We have to write this in Assembly and C, but you don't have to! Rust is what we prefer to see at the application layer."
However, TLS still involves a lot of code code that isn't pure low-level cryptography, like numerous protocol and certificate parsers, CA store interface and chain validation, networking, protocol state handling, etc.
If the other commenter was right, then what they’re saying is that people seeing a Rust TLS stack outperform non-Rust stacks might assume critical operations were written in memory-safe Rust. Then, that the post was implying memory-safe Rust is fast even with low-level operations. That maybe they could use Rust to replace C/C++ in other low-level, performance-critical routines. Then, they find out the heavy-lifting was memory-unsafe code called by Rust.
It does feel misleading if a reader thought Rust was replacing ASM/C/C++ in the low-level parts. I mean, even the AI people are getting high performance wrapping unsafe accelerator code in Python. So, what’s that prove?
In these situations, I might advertise that the protocol engine is in memory-safe code while the primitives are still unsafe. Something like that.
The detractors are talking about how they’re marketing or describing it. They want the memory safe and Rust labels to only be used for memory safe and purely-Rust programs. That’s fair.
Outside the marketing, the stack is good work. I’m grateful for all the TLS teams do to keep us safer.
I've been arguing this for years. It makes the area you need to review more tightly much smaller. Making it way easier to find bugs in the first place. I'm sometimes wondering if unsafe was the right choice of keyword. Because to people that don't understand the language, it conveys the sense that Rust doesn't help with memory safety at all.
I've written a bunch of Rust, and rarely needed to use unsafe. I'd say less than 0.1% of the lines written.
Aside from that unsafe Rust still has a lot more safety precautions than standard C++. It doesn't even deactivate the borrow checker. [1]
Note: I don’t know enough about Rust to tell you how they should label it.
Another thing you might find interesting is external verification of unsafe modules. What you do is build static analyzers, verifiers, etc that can prove the absence of entire categories of bugs, esp memory safety. It’s usually for small code. You run that on the code that doesn’t use memory safety.
Another technique is making a verified, reference implementation that’s used to confirm the high-performance implementation. Their interfaces and structure are designed to match. Then, automated methods for equivalence checking verify the unsafe code matches the safe code in all observed cases. The equivalence might be formal and/or test generators.
You can also wrap the unsafe code in safe interfaces that force it to be used correctly. I imagine the Rust TLS does this to some degree. Projects like miTLS go further to enforce specific, security properties during interactions between verified and unsafe code.
The last thing to consider are abstraction gap attacks. If mixing languages or models, then the behavior of one can make the other unsafe just because they work differently. Especially in how the compiler structures or links them. This led to real vulnerabilities in Ada code that used C just due to interactions, not the C code. Although previously checked by eye, there’s a new field called secure compilation or abstract compilation trying to eliminate the integration vulnerabilities.
Lastly, if not too bad for performance, some sandboxed the unsafe code with the interfaces checking communication both ways. Techniques used include processes (seL4), segments (GEMSOS), capabilities (CHERI), and physical (FPGA coprocessors). It’s usually performance-prohibitive to separate crypto primitives like this. Whereas, coprocessors can have verified crypto and be faster, though. (See Cryptol-generated VHDL.)
I also promoted techniques to verify the “unsafe” portions by using different, verification methods with some kind of secure linking to avoid abstraction gap attacks.
The detractors were complaining about changing the definition of memory-safe code. It was code in a language that was immune to classes of memory safety errors. If the code compiles, the errors probably can’t occur. A guarantee.
The new definition they’re using for this project includes core blocks written in a memory unsafe language that might also nullify the safety guarantees in the other code. When compiled, you don’t know if it will have memory errors or not. That contradicts what’s expected out of memory-safe code.
So, people were objecting to it being described as memory safe Rust if it included code blocks of memory-unsafe, not-Rust code. There’s projects that write the core, performance-critical blocks in safe languages. There’s also those doing making crypto safer, like Galois’ Cryptol or SPARK Skein. So, using the right terminology helps users know what they’re getting and reviewers do apples to apples comparisons.
For this one, they might say it’s “mostly safe Rust with performance blocks written in unsafe assembler for speed.” (Or whatever else is in there.) The high-security community has often talked like that. Instead of hurting perception, it makes suppliers more trustworthy with our users more educated on well-balanced security.
You are correct.
Although, communication has two parts: sending and receiving.
Application named “rustFoo”, automatically is an advertising for rust, and title “RustFoo is faster than Foo” for many implies “rust is faster than <probablyC>”.
The question is: are they cheating?
Could it possibly be that that they have (somewhat suicidally) chosen to force the AVX512 execution path, when more reasonable implementations have decided that it's not really worth risking halving the performance of EVERY OTHER TASK ON THE ENTIRE COMPUTER in order to use AVX512 for a performance gain that isn't going to matter except in the very tiniest slice of use cases -- big iron running on the edge with dozens (hundreds?) of gazillo-bit/s network adapters, doing nothing but streaming TLS connections. Plus the fact that you'd have to lock your TLS encryption code to a particular CPU core on previous-generation CPUS, which is also a Really Bad Thing To Do for a TLS transfer.
I rather suspect it's entirely that.
Even on latest generation intel CPUs it's not clear whether using AVX512 for TLS is a sensible choice. AVX52 still drops the processor frequency by 10% on latest-gen CPUs. So every core on the entire CPU would have to be spending 80% (60%?) of their time running TLS crypto code in order to realize actual benefit from using AVX-512 crypto code.
That's what and.
However, there's more to TLS than just the lowest-level primitives. There's parsing, packet construction, certificate handling, protocol logic, buffer management, connection management, etc. These things can be written in safe Rust.
It’s also the same strategy I would have used except maybe attempting extra verification of the assembly. It’s one of the best choices with today’s tools. There’s work on constant-time compilation and certification but I don’t know its maturity.
Seems like a cheap shot, considering Rustls's default cryptography is implemented using a fork of OpenSSL's libcrypto.
Of course, there's nothing wrong with writing memory-safe TLS atop C and assembly primitives. But to say that OpenSSL causes memory safety vulnerabilities without being clear that aws-lc-rs uses FFI to call down into AWS-LC, which is based on libcrypto from OpenSSL and BoringSSL seems disingenuous.
Ah, so if it's just a question of identifying and using the "good" C code, it really makes me wonder what Rust is actually adding here.
The protocol code, and the x.509 code from OpenSSL hasn't always been great. Rust providing memory protection on that is a nice thing.
There's certainly a question of how that makes for significantly more performance; handshake performance is usually dominated by crypto performance, and the same for data transfer... so if the crypto is coming from the same library, it's unexpected to see such a big change (10%+ in most graphs by my eye?)
Seems like it would be interesting to take one of these and really dig into it. That said, I know OpenSSL 3 was having some performance issues in some applications because of new locking behaviors; I don't know if BoringSSL took those or not.
That said, the performance speaks for itself here.
Comparing it to OpenSSL is a bit much.
This Problem exists all the way down in Rust‘s crypto libraries. OpenSSL just uses bindings, Ring uses BoringSSL which is just again C under the hood.
The only real Rust-only crypto project is RustCrypto, but they got a bit too clever with traits and generics. Also, the project is pretty undocumented.
They are doing good solid work, but I would not expect any dramatic improvements in security and seeing as the openbsd project values correctness over speed, very likely a small hit in the speed department.
It's probably worth benching just to see the impact of these changes - the fork happened around the same time as BoringSSL, which as these graphs show has quite different perf characteristics.