OpenSSL bug exposed up to 255 bytes of client heap and existed since 2011
jbp.io
jbp.io
When SPDY, the precursor of HTTP/2, got introduced, it needed a mechanism to signal that within TLS, a different protocol (SPDY instead of HTTP/1.1) was spoken. That mechanism was originally NPN. I don't know the exact details and motivation, but eventually people seem to have decided that NPN wasn't exactly what they wanted, and they invented a new mechanism called ALPN.
Now, that was a decade ago (the ALPN RFC is from 2014), so the question is: why do we still have NPN code in OpenSSL? I don't think anyone uses it any more. Shouldn't it have been removed long ago?
To put this in a larger context: it appears to me that OpenSSL still has a strong tendency to bloat. Heartbleed was essentially a "we added this feature, although noone knows why we need it" kind of bug, but it doesn't look to me they've changed. I still get the feeling that OpenSSL adds many features that they probably just should ignore (e.g. obscure "not invented here"-type algorithms), and they don't remove features that are obsolete.
I have to maintain a few VM’s and statically linked tools specifically for interacting with old network appliances that modern Linux hosts won’t “talk to” without crippling their SSL/TLS configuration or doing other horrendous workarounds like specific OpenSSL/OpenSSH config files for those things.
It’s actually an interesting problem when performing security assessments - some tools for scanning will have false negatives because they can’t “talk to” old shit.
I am of course well aware of backwards compatibility issues. But I don't see how they'd impact NPN.
You might have to update some code that hard-codes NPN functionality, but well, OpenSSL has done API changes in the past, quite significantly so.
You have to understand that outside of silicon valley and the hobbyist realm there are tons and tons of systems out there where the security posture is "don't allow anyone to have physical or network access to these systems who doesn't belong here" because the companies that made them deprecated them a decade ago or they were built before the widespread practice of supporting the Linux BSP for more than a couple of months.
These systems still work and you won't convince very many users of these systems to spend a bunch of money upgrading them for some notional security risk. You may also be working for a company that is contractually obligated to support these systems in some way. So what do you do?
In many cases, you do the best you can to mitigate the risks you know are there with tools like firewalls, port knocking, non-standard ports, and so on. And you hope that the 2-3% of attackers that actually know what they're doing never try to crack these things.
Look, I get it. Deprecating things has tradeoffs. But NPN looks like an incredibly safe thing to deprecate. It has only been used for a very short timeframe. Its only use case (SPDY) has a fallback (HTTP/1.1) making sure things still work if it's not supported. Your story about OpenSSH ciphers has nothing to do with it.
I noticed also that some of the security scanning tools we use “silent fail” on some old appliances because they can no longer negotiate the appropriate SSH connection due to library updates :)
I wish I could replace these old devices, and I recommend to do so, but usually replacing them won’t happen until they physically break.
Think: Industrial IoT devices at client sites and other such security horrors.
On that point: why is the previously allowed crypto settings now considered as good as nothing? I have to force SECLEVEL=0 for openvpn/openssl to allow connecting to my company's vpn. From my reading that would allow any old cipher or hash rather than the previous minimum. Why is the previous level not kept as M and the library bumps the default to N? I know that a theoretical weakness means M is vulnerable to breaking but you force people to make it even worse.
I think my boss hasn't updated his openvpn so he doesn't have new openssl so he never saw the problem. His company, his problem.
What they're going to do if given this is call your hypothetical "SECLEVEL=M" feature secure and then be outraged when it isn't.
If nobody is attacking you, no security will work fine. If you are being attacked, obsolete security likely won't help anyway.
I guess it could be that it's not possible.
I mean, in the intervening period there was OpenSSL 3 which was a large backwards-incompatible (API and ABI) release, with a huge amount of effort on the part of OpenSSL developers and users to follow along. It was the ideal opportunity to drop this sort of old stuff.
Looking at the spec for NPN and knowing about ALPN from implementing just enough of if to enable TLS false start way back when, ALPN is better because it's less steps.
With ALPN, the client includes its supported protocols in the Client Hello, and the server picks one or discards the extension in the Server Hello. With NPN, the client indicates it supports NPN in the Client Hello, the server sends back a list of supported protocols in the Server Hello, and the client sends an extra message to indicate its selection between change cipher spec and finished.
There's perhaps more client profiling information on the wire with ALPN, but it's not very interesting when 99% of connections just advertise h2,http/1.1. During the development of http/2, you could probably have a pretty tight range on client versions depending on what pre-standard versions were declared.
So it's useless? Yes! So why do you want it? Because it's rare!
The fact that lots of real world code uses OpenSSL is one of those things it's going to be hard for our descendants to understand. Like explaining that people used to Smoke cigarettes on aeroplanes. But why though, that's crazy. Yes, yes it is.
I would have been bragging too, were I removing ifdefs by the shovelload every day for weeks.
Unless it is obviously breaking something someone wants to do, it has roughly a zero cost (in maintenance) being left alone, especially since bugs can often be ignored, but a non-zero cost to remove it - as it generally will only garner complaints and whining from any remaining users.
And in open source in particular, it’s damn near impossible finding remaining users in advance, or even telling if there are no remaining users.
If no changes get made, then no friction.
And open source code can and often is just abandoned. Defacto, sometimes even de jur. User beware, use at your own risk, etc.
They have a counter balancing thing which is no one can see it to shame them.
At worst they generally just suffer reputational damage, not actual cost or lost revenue, like a business would.
https://marc.info/?l=openbsd-announce&m=150996307120987&w=2
I wonder how many memory leaks it'll take for OpenSSL to finally get their act together or for major projects to drop it entirely.
I know Gentoo did https://bugs.gentoo.org/762847
Python 3.10 did https://peps.python.org/pep-0644/
Void Linux did https://voidlinux.org/news/2021/02/OpenSSL.html
Etc.
Seems like a lot of that was due to OpenSSL breaking API compat that LibreSSL promised not to break though.
The post: "Silently sends up to 255 bytes of the client’s heap to the server."
The implementation is incredibly simple. Treat all blocks of memory as blob with a known size and then read/write into those blobs with a cursor to track progress and bounds checks on every access. Fence all serialization/deserialization through a safe low level interface. Not only do you get memory safety (which we later proved using formal reasoning) ... but when you're parsing message formats it lends itself to a declarative coding style that makes it very clear what the structure is. You can also do lifecycle things, like erasing sensitive memory with zeroes when you're done with it, making sure things don't show up in core dumps, etc. BoringSSL introduced a Crypto_bytes API that also did some of this plus bounds checking, and retrofit it into OpenSSL.
OpenSSL on the other hand is a horrific mash up of raw pointer arithmetic, ad-hoc parsers interleaved with business logic and control flow. I could never keep it straight, and it always scared me to review.
> Meeting those constraints is quite unlikely nowadays:
> NPN is a precursor to ALPN and was abandoned in 2012. It is very uncommon on internet servers now.
> Node.js 10 and later removed NPN support, and is well past end-of-life.
> Python 3.10 and later removed NPN support.
OpenSSL advisory: https://openssl-library.org/news/secadv/20240627.txt
> Severity: Low
> Issue summary: Calling the OpenSSL API function SSL_select_next_proto with an empty supported client protocols buffer may cause a crash or memory contents to be sent to the peer.
BoringSSL fix: https://boringssl.googlesource.com/boringssl/+/c1d9ac02514a1...
The heap leak was independently observed in 2014 in the Android okhttp library: https://github.com/square/okhttp/issues/437#issuecomment-358...
• https://www.adacore.com/press/spark-skein (ctrl-f for flaw)
• [PDF] https://www.cl.cam.ac.uk/archive/mjcg/Meeting.SecurityTools.... (page 12)
• [PDF] https://www.adacore.com/uploads/techPapers/SPARKSkein_SBMF.p... (page 5)
> 2024-04-23 - Discovery of SSL_select_next_proto memory unsafety while rewriting it in rust. [ https://github.com/rustls/rustls-openssl-compat/tree/main/ru... ]
In principle the result is definitely safe if your C code (which previously called OpenSSL) is safe, regardless of whether OpenSSL itself is riddled with bugs (which it likely is)
A safety focused person rewriting tricky C pointer banging code in Rust is likely to inadvertently fix bugs - what's interesting is whether you notice that as you're doing it or it's just silently not buggy any more in Rust.
A few hours ago I had some code that was doing bignum arithmetic and one call returned None where I hadn't realised it could and in a haze I thought "Oh, if it's None we can treat it as Zero" so I changed unwrap() to unwrap_or(Zero::zero()) † but nope, it was None because that function assumes we're getting integers and it's just figured out it has a proper fraction instead, so hence there is no such integer, None. Treating None as Zero meant all the related tests blew up and I realised my mistake after a couple of minutes. No, e to the power 0.5 is not 1.
† Yes that's suboptimal because it will make the zero bignum even when it doesn't need it, I should have called unwrap_or_else instead but also no I shouldn't because it's wrong.
That's what I doubt at all. I rather trust the compilers than the openssl devs.