Take the aforementioned error return inconsistencies -- how do you possibly deal with refactoring that for consistency when so many other projects are consuming OpenSSL as a library? You're trading one big problem (new clients are likely to get error return handling wrong) with another (if you modify all the returns for consistency, existing clients are now expecting the wrong return codes or, if you're smart and rename the functions to avoid that, at the very least the 3rd party consumers now need to do a lot of rewriting and deal with a painful crossover phase supporting both the old and new APIs).
You split your project into two components:
1. a new, clean, minimal core with a nice, shiny, new API; and
2. an "OpenSSL emulation layer" that loads your core library, and wraps it in OpenSSL-compatible cruft.
It'd be very similar to, say, replacing Direct3D with OpenGL, and then writing a Direct3D emulation layer that calls OpenGL.
Unless there's an actual federation behind the project that can fund it and keep it stocked with talent and oversight, something Mozilla or Apache could do, it will continue to be an unmitigated disaster.
The only reason it wouldn't have been caught in unit testing is because Apple isn't unit testing their own SSL code. If Apple doesn't commit to doing security well why do people expect OSS to?
People did rage over Apple's SSL issue, but Apple handled it swiftly, pushing out updates to all affected devices as quickly as practical. It's also worth noting it was a validation bug, not something that could leech arbitrary bits of data from a device.
I have no idea how some open-source projects get anything done, their development process is insane, but they manage to hobble along. Something as significant as OpenSSL needs to be held to a higher standard.
If the OpenSSL team can't get their act together, it would be worth it if a larger non-profit organization took the initiative and made a tested, rock-solid implementation of same with an optional compatibility layer.
I've had to integrate with OpenSSL before and the code in there is truly mad. Well written code should be boring and obvious, but OpenSSL is the polar opposite.
But how did it ever get so widely adopted, if it's so hard to use?
And even if it is widely adopted, that still shouldn't stop anyone from forking it and fixing all the problems.
Look, if this was about something fairly inconsequential, like a library for a game or a blog or something, sure, use some badly maintained, badly documented, inconsistent library. But when you care enough about security to use SSL, surely you care enough to use something that's put together in such a way that you can be sure it does what you think it does? Bad documentation and inconsistent error handling in a security package sound like an unbelievably dangerous idea. And if it's really an unmaintainable mess, like everybody says, then how can bugs possibly be fixed? How easy would it be for the NSA to insert some crucial vulnerability into the code base?
Security, moreso than anything else in computer science, needs to be clear, transparent, understandable, well documented and well maintained. If it's not, mistakes will be made and security will be compromised.
Usefulness and lack of alternatives?
2. Do you have time for a rewrite? What makes you think anyone else would?
3. The (generally correct) mindset is don't implement your own crypto. This is particularly true of something like TLS, which is complicated and has subtle requirements that are easy to screw up. Unfortunately, this means that even people who have time are discouraged from doing an OpenSSL rewrite.
Also, I've see a ton of code just invoke the OpenSSL tool instead of actually use OpenSSL library.
I tried to use the library and was down in spirit, much like the author of the blog post, but I was too disappointed to even care, and just gave in to calling the tool from my code.
An idea: create a fork, call it OpenSSL2 and clean it up, but maintain both codebases, and end support for OpenSSL1 in 2-5 years?
Back in the Linux bubble days (1997-early 2000s), lots of companies employed full time people to work on all kinds of stuff, not just profitable company project, but also needed infrastructure. Even front end projects, like Nautilus and Evolution were created by full paid teams.
There might be a bunch of companies willing to chip in a bit, but coordinating that is difficult. Something like kickstarter might be able to help, but that has its own problems: if you spend your time writing a commercial SSL thing, you can go out and sell it right now when it's ready. If some company has to wait around for the results.... it's not as nice for them.
And NSS is pretty light on the server code IIRC...
In my past as professional code auditor, I've seen OpenSSL shipped with products quite a bit, and in 99% of the cases it was openssl the command line tool that was used to access the functionality because yes, the API is that bad and confusing that people would rather pipe to an executable that use library calls...
http://directory.fedoraproject.org/wiki/Mod_nss#What_feature...
That sounds like bad advice. The huge impact of Heartbleed is precisely because of the monoculture in SSL/TLS implementations. Shit happens but this vulnerability is such a catastrophe because an estimated half a million of the top Internet websites have been vulnerable since 2012, including popular services that everybody is using and the vulnerability once discovered, is easy to use and far reaching, allowing one to steal sessions, passwords, the domain's private keys and anything important.
It really can't get any worse than this. At the very least we found out about it, better late than never and we can discuss the source-code, but it does make one wonder ... if SSL/TLS is such an important piece of infrastructure and if we need a monoculture (because apparently it's bad advice to reimplement crypto), why aren't we using a library that's the reference and that's peer reviewed by a standards committee, instead of relying on a library for which commits don't necessarily trigger sounds?
When people do implement cryptography they rely on having large numbers of other people review it. That is not a model that scales to having a large number (more than a couple dozen) of competing implementations.
The semi-official reason though is that OpenSSL doesn’t offer API compatibility between versions.
http://opensource.apple.com/source/Security/Security-55471/l...
You are aware that GnuTLS had a very similar "goto" style verification bypass bug, and that OpenSSL has had several flaws and broken stuff discovered, including the major recent "Heartbleed Bug".
Most people don't have the time or care -- even if they know how to do it. The big push for open source was when companies where sponsoring its development with a lot more paid developers than today (companies from IBM and SUN to RedHat, Corel, Novell, Eazel, VA Linux -- remember them? The bubbliest IPO in history).
That can happen not only for highly specialized code like OpenSSL (which even fewer people can understand), but even for very common and widespread application level code, including projects that one would assume would attract many people.
Gtk, for example, the GUI kit used in Gnome, has just one developer really working on it (I know, cause he complained). Compare that to the 20+ GUI teams companies like Apple and Microsoft, with dedicated technical writers for the documentation et al.
Its a pattern that you can watch on many IT-Projects.
Some "Key-People" like a project (for whatever reason) and a mass of developer follow them.
And than a thing get "standard" and more and more people use it.
It's just humanity; it doesn't just plague the IT community. :-)
Yes, reports of awful code quality in OpenSSL are not exaggerated.