So you'd think that if anything, these kinds of 'trivial' bugs, memory leaks and buffer overflows, would not be present in LibreSSL. Especially not in the same function. I wonder what went wrong and how it can be prevented.
Compare that to a basic unencrypted HTTP server, which you can write in ~10KiB of code. Complexity breeds these kinds of problems as a natural consequence.
I do know that verifiability is significantly easier than all of this (signatures; hashing), but it's quite possible there is no simpler solution for the secrecy aspect of TLS. I certainly wish there was more support for HTTP+integrity, for those who didn't need the secrecy component. These constant CVEs in every TLS library put our servers at risk in cases where secrecy simply isn't necessary.
But the bugs described are not complex - they are local problems that can be understood by just reading one function. The problem here is not complexity, but size (not enough people spend enough time to review all of the code) and the fact that the library is written in a language with manual memory management and unsafe memory access.
> I do know that verifiability is significantly easier than all of this (signatures; hashing), but it's quite possible there is no simpler solution for the secrecy aspect of TLS.
In this case, the affected component was related to the identity part (certificates), which are needed for verifiability. It was not directly related to secrecy.
C's simplicity is certainly a blessing and a curse. Right now, I'm of the opinion that C++ is the best we've got, even if not perfectly ideal. RAII, shared pointers and smart containers with bounds checking allows you to keep nearly all of the performance without the drastic downsides of tracing garbage collectors (obscene amounts of memory usage, nasty stalls, etc.)
And when you get into the really high-level languages (Python, Ruby), they simply don't scale.
Everyone's darling right now is Rust. I still think it's too early in its life to know how it'll play out (hot new languages follow a predictable path of gaining negative baggage and shortcomings as they increase in real-world usage), but I guess we'll see. Maybe it will indeed be a great panacea to these issues.
> In this case, the affected component was related to the identity part (certificates), which are needed for verifiability. It was not directly related to secrecy.
Sure, but I'm generalizing here.
I can (and did) write a SHA256 hashing algorithm in 3KiB of code. Combined with a package gist updater (these use signing with local keys for updates), that's enough to secure binary package downloads on the BSDs over plain-text HTTP.
Surely we can come up with a less complex system for verification alone. Something akin to signify, perhaps. I know that doesn't solve how to universally handle anonymously verifying public key authenticity, but surely we can make something less complex than the clusterfuck that is TLS and the CAs. Yet it doesn't seem there's much, if any, interest in even trying. Everyone has hitched their wagons to HTTP/2 and mandatory TLS; which to me feels like a huge step backward by means of making tech vastly more complex.
I think you need to qualify what "scale" means in this context. Otherwise java is, for all its warts, more or less as high level as ruby and python, and it's probably been proven to scale up (to big machines, or many developers) reasonably well.
> The system is built on the JVM platform ... in fact they ended up by doing all the business logic for their platform: all trades, from all customers, in all markets - on a single thread. A thread that will process 6 million orders per second using commodity hardware
https://www.reddit.com/r/programming/comments/3p6r2v/i_wrote...
I've constantly had similar experiences. I tried to write the GUI portion of my programs with Python and PyGTK. I needed to write a simple function that scaled up a bitmap from 256x224 to 512x448. A nearest-neighbor pixel doubler with no interpolation, color conversion, or anything else. It ended up running at around 30fps with nothing but the scaling, on a Core i7-2600k processor. And when I asked for help in optimizing it, nobody could really improve the Python, and they recommended I write "the slow parts" in C as a shared library, and invoke that from the Python GUI.
It's okay to be 10%, 30% or even 50% slower than C. But to be 2500% slower or more is completely unacceptable for production code. That level of performance only works on websites with small amounts of traffic, or on the simplest of applications.
The root problem here is of course dynamic typing. That makes scaling both performance and code size much more challenging.
Java is certainly vastly better, but the garbage collection and its stalls, wasteful memory usage (I've seen our production servers bog down with Tomcat eating up all the RAM on the server for simple programs), and overall extreme negatives of the language itself (no unsigned integers, limited to single inheritance, terrible support for generics [maybe this has changed recently?], etc), plus the legal issues that Oracle is saddling the language with, make it an unacceptable alternative.
C# might have been better if Microsoft ever cared about any platform other than Windows.
Both dynamic typing and garbage collection are the wrong ways to solve the problems of C. Rust is the closest I've seen to addressing the real problems, but it coming from Mozilla (with their terrible record with Firefox), and the syntax being so radically different from C, I think will chase a lot of people off.
signify also looks really promising. Get the key down to the size of a QR code, and you could separate the issue of signing from signature validation entirely ... post it on your HTTPS Twitter feed, put it on your business card, whatever.
I think it would be a positive step to get users to think about how they are verifying integrity themselves and the sources they trust.
You could even have a dozen sources and let there be a little button in the URL bar to show you which sources agree on a public key provided by a server.
I don't trust any implementation of it that uses unmanaged code. (yes, pretty much assuming everything based on openssl will be broken time and time again)
You need to use some sort of bignum function if you want to display large arc numbers in decimal. Which is what OpenSSL and LibreSSL do, and that involves creating and destroying BIGNUM objects. The problem is that someone in LibreSSL moved the free() to outside the loop body handling the arc numbers while they were cleaning up. So only the last BIGNUM used gets freed.
Oh, did I tell you that arc numbers are encoded in base 128, with the first bit of every octet used to signal whether it's the last octet of that arc number? Because they are.
It still seems questionable that they'd let any length be set, especially since you can just chain numbers together into a longer OID.
Base128 encoding seems fair though; the other varint formats are more exotic and only work with many ints at once.
I assume that's because OID nodes are essentially namespaces, giving an arbitrary upper bound to node ids would artificially and "unnecessarily" limit the size of the corresponding namespace. That looks like a "correct versus realistic" debate which was won by the "correct" side.
While Rust can export functions the same way C does it you are in unmanaged land again then.
With http://www.ponylang.org/ its much better, it can export functions like C does it, but its fully managed. The only limitation are the Data Types, you are restricted to those C knows.
Also, FWIW, I don't think Rust should be considered unmanaged for these purposes: it bridges the divide between "managed" and "unmanaged" by bringing the memory safety of the former to a language that doesn't require a complicated runtime. So strictly speaking, yes, it's unmanaged, but it doesn't suffer the worst of the problems of existing unmanaged languages like C or C++.
You can do that with most "managed" languages. You can call Python, Haskell, Go or OCaml from C, pony isn't exactly special here.
One issue in all these cases is you're carrying a whole runtime with you, possibly one per library you're using, praying they don't conflict with one another[0] and limiting the portability of the whole[1]. You're also significantly increasing the cost of using the library in question as it now brings in megabytes of runtime along for the ride.
[0] with conflicting signal handling for instance
[1] if only because all those runtimes you're bringing in might be doing stuff like spawning their own threads and the like, so that needs to be available on the target platform
Possibly less as there's a pretty high chance to get the ownership rules wrong on the callee side in a way which'll fault (because the callee will have freed the structure) rather than "just" leak memory (because the caller will not send the structure back for deallocation).
Example: https://blog.filippo.io/building-python-modules-with-go-1-5/ in the third snippet (AddDot) the author returns a Go string which may (will?) be freed next time the GC runs[0] regardless of the caller still using it.
[0] https://www.reddit.com/r/golang/comments/3ieiiu/building_pyt...
Think pass-by-value structs with only fixed size fields. Or JSON, protobufs etc. (Ok, you need the one pointer for the JSON/PB data, but it's not kept around after null check and decode)
There are (at least) three issues with that:
1. not all FFIs can handle structs, according to the Rust FFI omnibus GHC's doesn't[0]
2. the callee side must be able to express on-stack by-value structs, many (most?) managed languages don't, either not supporting them or having them only as optimisation (based on escape analysis)
3. and finally the layout (and size) of the struct must be deterministic, which further restricts (2)
> Or JSON, protobufs etc. (Ok, you need the one pointer for the JSON/PB data, but it's not kept around after null check and decode)
Depending on the callee's implementation that may be too late already, there is absolutely no guarantee the data will even go through the FFI without being released, so you'd be relying on garbage data not being zeroed/reused/moved in the interval, which would be an implementation detail.
[0] http://jakegoulding.com/rust-ffi-omnibus/tuples/#haskell
I think the FFI could well express this. Just generate a C thunk that converts the value from the stack to a suitable representation.
> 3. and finally the layout (and size) of the struct must be deterministic
The layout and padding of structs is deterministic in all C environments I know of since they end up as part of ABIs.
> Depending on the callee's implementation that may be too late already, there is absolutely no guarantee the data will even go through the FFI without being released
So you make a C FFI call from a safe language to libfoo.so function void foo(char *jsondata) and before the callee returns your safe runtime yanks the data from under it in another thread?
Yeah, you would also need the caller's FFI to guarantee against this to get safety. In absence of such guarantees you should stick to only passing pass-by-value structures!
Oh definitely, I'm just saying not all of them do (if you follow the link, the Ruby/MRI, CPython and Node FFIs all provide for that at least on the caller side)
> So you make a C FFI call from a safe language to libfoo.so function void foo(char *jsondata) and before the callee returns your safe runtime yanks the data from under it in another thread?
Yup, could happen with a concurrent GC kicking it at an inconvenient time, or with a refcounting GC.
> Yeah, you would also need the FFI to guarantee against this to get safety.
I'm not sure it could (without caller cooperation), best I can see is offer side-channels to "leak" memory (allocate non-managed memory and copy stuff into it) or recommend that you keep an internal strong reference of some sort (e.g. a global list/set).