Not sure the trade off line has been drawn in the correct place here. TLS should be implemented as a filter; the world isn't limited to files and (blocking) sockets. What about integration with event loops, for example? How does one test this API?
Mixing in OS-related stuff only complicates the API.
My compiler is most commonly used together with an editor. Still I'm glad it didn't come with an editor built-in :)
> 2) You can create file descriptors not backed by any actual network connection or file on the harddrive
Indeed. It doesn't mean it is the preferred way to do it, from a code-quality or even performance point of view.
The OS used to do the networking for me. It exposes the BSD sockets API, which is pretty simple and nice but is unfortunately no longer safe to use. Yet, the kernel doesn't offer anything else.
So you're right, but you don't go far enough. It should all be in the kernel. Instead of tls_config_set_cert_file() we should be using setsockopt(2). Until we can, however, we'll have to rely on user-space libraries.
This is the second time you've said something like this today. Explain what you mean by it.
> So you're right, but you don't go far enough. It should all be in the kernel. Instead of tls_config_set_cert_file() we should be using setsockopt(2). Until we can, however, we'll have to rely on user-space libraries.
I think what you mean is that it should be in the standard library. There is no reason it should have to be in the kernel.
The world has changed. We must now consider send(2) and recv(2) to be toxic. They are fundamentally broken and cannot be used safely. The problem is, what do we use in their place? Simple sockets should continue to be simple to use, even now that we need to layer encryption on top. OpenSSL is not a practical replacement for those functions. This new API is a pretty good attempt. It may not do everything that OpenSSL does, but it doesn't need to. It only needs to make the simple things easy. If you have a different use case, you should use a different high-level API.
I think the way to understand this API is to see it as a "view", a wrapper, an abstraction. It's like a database view that hides the dozen tricky joins necessary to answer a frequently asked question. Also, like a database view, it is one of potentially many, each with a different purpose.
As a wrapper, this API exposes only a subset of the full functionality, but it's a very useful subset. Perhaps 80% of the complexity of the OpenSSL API comes from only 20% of use cases. This API exposes only a small subset of functionality, but that's what makes it simple and easy to use and it still covers 80% of use cases. If you aren't in that 80%, then this API isn't for you. But that's ok!
It seems reasonable to expect multiple layers of cryptography libraries. We ought to have a set of low-level libraries that implement the fundamental primitives in a general and orthogonal way. It would be a sort of cryptography "kernel". Most developers wouldn't use it in the same way that most don't directly make syscalls into the OS kernel, but instead use a high-lever wrapper library. To open a file, for example, we often use a wrapper like the functions in a libc, or a wrapper of a wrapper like libsqlite3.
So, if an API doesn't cover your particular use case, that doesn't mean it's bad. You just need to find a different API. The uses for cryptography are myriad. It seems unreasonable to expect a single API to cover everything while remaining simple and easy to use.
I disagree. I think it is exactly the place for an OpenSSL API replacement.
As is alluded to in this link (thanks cremno) [1], they want a critical mass of users before it makes sense to deprecate the libssl API. Deprecation seems necessary in order to perform the serious cleanup/reimplementation under the hood that they would like to do.
While new projects may have an easier time, they admit that adoption by existing projects is significant work. However, so many and large projects already exist that it seems that many of them need to be won over in order for a deprecation to be practical. Many existing projects (see other comments in this post) use this transport layer flexibility in the OpenSSL API, and not offering it with the new API may slow adoption.
Yes, like you say another API layer could be added, and it is a deeply subjective matter so let's agree to disagree... But composability is so important and only becomes more so as we tire of wheel reinventions leading to unnecessary bugs. Enabling modularity and reuse needs 1st class API attention. Being able to easily plug this new API as a source/sink onto the myriad of existing (and yet to be created) I/O frameworks would be a major win. This is my $.02 and humble appeal to the people doing the fantastic work on this lib. My thanks to all of you!
I wrote exactly one program that uses libssl in C directly. It is a special-purpose web server that can serve files or stream content over an HTTPS connection [1]. It was a huge pain to get my event loop to play nice with TLS because TLS does more reads/writes where a plain HTTP connection would do just reads or just writes. I also had to enable various non-defaults (SSL_MODE_ENABLE_PARTIAL_WRITE | SSL_MODE_RELEASE_BUFFERS) to get it to behave more like a regular socket. Having a more standard way to support stuff like this would be great, and it does exist in other places. For example, Python wraps libssl in such a way that you can just wrap a plain socket into a TLS socket, and it pretty much just works. It'd be great if stuff like this existed at a lower level where this new API seems to live.
Apache, Nginx, Node.js, curl, etc -- they all use the BIO API to provide TLS as a filter over their internal streaming APIs. Having personally worked on some of these projects, the current libtls API is not sufficient at all.
This isn't to say at all that OpenSSL's BIO API was great -- it could definitely use iteration all over -- and I hope libtls gets there, but it doesn't seem at all focused on realistic server workloads.
For example: http://funcptr.net/2012/04/08/openssl-as-a-filter-(or-non-bl... is a blog post I wrote wherein I show how to use SSL as a filter. The only reason I use a BIO is to temporarily store the same data I just got from the network and have in an std::string... it's duplication.
However, given that the world's flawed and FILEs (and fds) aren't proper objects that end-user can create and use with their own functions (like an implementation of read(2) that'd yield to an event loop until the data's there), guess it warrants yet another ad-hoc struct. Or a bunch of functions that work on some context struct and that you're supposed to pass data chunks by yourself. Like in classic OpenSSL API.