Libtls – New TLS API from LibreSSL
openbsd.org
openbsd.org
Mixing in OS-related stuff only complicates the API.
My compiler is most commonly used together with an editor. Still I'm glad it didn't come with an editor built-in :)
> 2) You can create file descriptors not backed by any actual network connection or file on the harddrive
Indeed. It doesn't mean it is the preferred way to do it, from a code-quality or even performance point of view.
The OS used to do the networking for me. It exposes the BSD sockets API, which is pretty simple and nice but is unfortunately no longer safe to use. Yet, the kernel doesn't offer anything else.
So you're right, but you don't go far enough. It should all be in the kernel. Instead of tls_config_set_cert_file() we should be using setsockopt(2). Until we can, however, we'll have to rely on user-space libraries.
This is the second time you've said something like this today. Explain what you mean by it.
> So you're right, but you don't go far enough. It should all be in the kernel. Instead of tls_config_set_cert_file() we should be using setsockopt(2). Until we can, however, we'll have to rely on user-space libraries.
I think what you mean is that it should be in the standard library. There is no reason it should have to be in the kernel.
I wrote exactly one program that uses libssl in C directly. It is a special-purpose web server that can serve files or stream content over an HTTPS connection [1]. It was a huge pain to get my event loop to play nice with TLS because TLS does more reads/writes where a plain HTTP connection would do just reads or just writes. I also had to enable various non-defaults (SSL_MODE_ENABLE_PARTIAL_WRITE | SSL_MODE_RELEASE_BUFFERS) to get it to behave more like a regular socket. Having a more standard way to support stuff like this would be great, and it does exist in other places. For example, Python wraps libssl in such a way that you can just wrap a plain socket into a TLS socket, and it pretty much just works. It'd be great if stuff like this existed at a lower level where this new API seems to live.
However, given that the world's flawed and FILEs (and fds) aren't proper objects that end-user can create and use with their own functions (like an implementation of read(2) that'd yield to an event loop until the data's there), guess it warrants yet another ad-hoc struct. Or a bunch of functions that work on some context struct and that you're supposed to pass data chunks by yourself. Like in classic OpenSSL API.
Apache, Nginx, Node.js, curl, etc -- they all use the BIO API to provide TLS as a filter over their internal streaming APIs. Having personally worked on some of these projects, the current libtls API is not sufficient at all.
This isn't to say at all that OpenSSL's BIO API was great -- it could definitely use iteration all over -- and I hope libtls gets there, but it doesn't seem at all focused on realistic server workloads.
For example: http://funcptr.net/2012/04/08/openssl-as-a-filter-(or-non-bl... is a blog post I wrote wherein I show how to use SSL as a filter. The only reason I use a BIO is to temporarily store the same data I just got from the network and have in an std::string... it's duplication.
The world has changed. We must now consider send(2) and recv(2) to be toxic. They are fundamentally broken and cannot be used safely. The problem is, what do we use in their place? Simple sockets should continue to be simple to use, even now that we need to layer encryption on top. OpenSSL is not a practical replacement for those functions. This new API is a pretty good attempt. It may not do everything that OpenSSL does, but it doesn't need to. It only needs to make the simple things easy. If you have a different use case, you should use a different high-level API.
I think the way to understand this API is to see it as a "view", a wrapper, an abstraction. It's like a database view that hides the dozen tricky joins necessary to answer a frequently asked question. Also, like a database view, it is one of potentially many, each with a different purpose.
As a wrapper, this API exposes only a subset of the full functionality, but it's a very useful subset. Perhaps 80% of the complexity of the OpenSSL API comes from only 20% of use cases. This API exposes only a small subset of functionality, but that's what makes it simple and easy to use and it still covers 80% of use cases. If you aren't in that 80%, then this API isn't for you. But that's ok!
It seems reasonable to expect multiple layers of cryptography libraries. We ought to have a set of low-level libraries that implement the fundamental primitives in a general and orthogonal way. It would be a sort of cryptography "kernel". Most developers wouldn't use it in the same way that most don't directly make syscalls into the OS kernel, but instead use a high-lever wrapper library. To open a file, for example, we often use a wrapper like the functions in a libc, or a wrapper of a wrapper like libsqlite3.
So, if an API doesn't cover your particular use case, that doesn't mean it's bad. You just need to find a different API. The uses for cryptography are myriad. It seems unreasonable to expect a single API to cover everything while remaining simple and easy to use.
I disagree. I think it is exactly the place for an OpenSSL API replacement.
As is alluded to in this link (thanks cremno) [1], they want a critical mass of users before it makes sense to deprecate the libssl API. Deprecation seems necessary in order to perform the serious cleanup/reimplementation under the hood that they would like to do.
While new projects may have an easier time, they admit that adoption by existing projects is significant work. However, so many and large projects already exist that it seems that many of them need to be won over in order for a deprecation to be practical. Many existing projects (see other comments in this post) use this transport layer flexibility in the OpenSSL API, and not offering it with the new API may slow adoption.
Yes, like you say another API layer could be added, and it is a deeply subjective matter so let's agree to disagree... But composability is so important and only becomes more so as we tire of wheel reinventions leading to unnecessary bugs. Enabling modularity and reuse needs 1st class API attention. Being able to easily plug this new API as a source/sink onto the myriad of existing (and yet to be created) I/O frameworks would be a major win. This is my $.02 and humble appeal to the people doing the fantastic work on this lib. My thanks to all of you!
Edit: Grammar.
* tls_connect() connects a client context to the server named by host. The port may be numeric or a service name. If it is NULL then a host of the format "hostname:port" is permitted.
* tls_connect_fds() connects a client context to a pair of existing file descriptors.
* tls_connect_socket() connects a client context to an already established socket connection.
That sounds strange, as I believe we're in disagreement. I believe the flexibility should exist, and choosing the transport medium of already encrypted data should be an option for the developer. This is completely unrelated to the security critical configuration of the TLS protocol, i.e. certs, keys, ciphersuites, etc.
Yes. Which makes it useless for protocols that carry TLS. e.g. EAP. So I can't use it in my pet project: http://freeradius.org/
The new API is useful if you want to do TLS over sockets. It's completely unhelpful for everyone else.
They just added a new API that answers the most common use cases in a simple, straightforward, and internally consistent way.
Yes. Through the terrible BIO_* API.
It's imperfect, but it lets you treat TLS as a "black box" that you shove encrypted/cleartext data into, and get cleartext/encrypted data out.
This API is good for socket communication. Nothing more.
This API could be extended by adding "underlying" read and write functions. Off of the top of my head:
typedef int (tls_underlying_io)(struct tls ctx, void u_ctx, const void in, size_t inlen, void out, size_t outlen);
int tls_set_io(struct tls ctx, void u_ctx, tls_underlying_io u_read, tls_underlying_io u_write)
Where "u_read" is used by libtls() instead of calling read(fd,...), and u_write() is called by libtls instead of write(fd, ..)
Having a full standalone implementation would be even better, but this is still great as-is.
"LibreSSL: More Than 30 Days Later"
The ressl API does provide one noteworthy feature. Hostname verification. In order to make a secure TLS connection, you must do two things. Validate the certificate and its trust chain. Then verify that the hostname in the cert matches the hostname you've connected to. Lots of people don't do the latter because OpenSSL doesn't do that latter. You have to do it yourself, which requires knowing about things like CommonNames and SubjectAltNames. The good news is that popular bindings for languages like python and ruby include a function to verify the hostname. The bad news is if you pick a python or ruby project at random, they probably forget to do it. Another funny fact is that since everybody has to write this code themselves, everybody does it a little bit differently. Especially regarding handling of wildcard certificates and everybody's favorite, embedded nul bytes. Hostname verification is on by default in ressl, and the API is designed so that you always provide a hostname; there's no way to accidentally call the function that doesn't do verification.
I’m thinking that there ought to be an API defined by an IETF working group and specified in an RFC (like the IPv6 API C interface is), with obvious room for expansion for library-specific features. Then all TLS libraries could implement that API and end the madness.
INB4 xkcd 927: https://xkcd.com/927/
OCSP is generally not a well thought out feature given current attack scenarios: https://www.imperialviolet.org/2014/04/19/revchecking.html