Ok, so the second is, "Have there been defects found in any parts of the default install after those parts were distributed? And were they just fixed or analyzed for exploitability with an update on the website if necessary?"
1. There's never been a defect that could affect security in any OpenBSD software, from kernel to protocols, after it was released.
2. For meaningfulness, the default install supports real-world usage (and usefulness) of the system.
The good news for No 1 means that they never had to issue a patch in a protocol or software due to defects with potential to leak data, allow hijacking, or experience DOS. Knowing that, once released, every OpenBSD system was unhackable and needs no patches past functionality fixes would greatly reduce administrator overhead. Truly fire-and-forget system. If the website claim is true, that is. :)
Further, if No 2 is true, users won't have to use anything but the base install to handle day-to-day tasks on their network. Anything critical is included, usable, and secure-by-default. Users would only need to rely on 3rd party stuff for non-obvious use cases or stuff that truly doesn't belong in a baseline. So, the ports collection wouldn't even be necessary in most deployments (esp infrastructure, console desktop) as the functionality and what counts in security claims were batteries included like the competition.
I'll leave it to users and people following their bug trackers to determine if either is the case. Especially No 1 which is my main point. The mere existence of security patches would kill the web site claim. It would instead become, "Of all the defects found, only two were analyzed and shown to be exploitable over X years." Makes for a totally different impression, eh? ;)
I certainly could dig back into OpenBSD (and C) to fully understand its current situation, its code, its mitigations, every source change from vulnerability 2, and so on. Those details could benefit OpenBSD in form of contributions I attempt or the reader following along to get a better picture of insides. However, it's not necessary for evaluating my concern about the 2 vulnerability claim. We can cleverly cheat around all that knowledge by going straight to fundamentals and claim itself. Then, only acquiring (me) and/or applying (others) that knowledge if the specialist skill turns out to be required. Others and I did this with success when drowning in product and project security claims with finite time to review them. Let's apply that.
"instead of reasoning about all theoretical directions OpenBSD could have taken."
My main question is a simple, concrete one that I shouldn't have to investigate at all: what evidence is there that there have only been 2 exploitable flaws in all OpenBSD releases? To claim this, there would have to be a tracker of every defect fixed in OpenBSD since its inception, an analysis of each one's exploitability, and a result saying only two were exploitable. There's trackers like that for many projects. Where's that record for OpenBSD or, to save you time, do they even analyze every bug for exploitability?
If that doesn't exist or that analysis wasn't done, then the only 2 vulnerability claim can be rejected immediately because it wasn't proven. Instead, the claim becomes "Of all X defects found, people only showed two were exploitable remotely. (Optionally) That's a defect ratio of X per Kloc with vulnerability ratio of Y per Kloc. (Not optional) The rest either weren't assessed for severity or were shown harmless." Alternatively, the claim is totally false if even one security fix for a remote attack was done after the 2nd vulnerability was fixed. It would have to be changed to a maybe pending further, specialist analysis. This isn't theoretical: it's a direct implication of the claim they're making, an extraordinary implication, and one which deserves concrete evidence to back it.
So, I encourage people to reject it until they have that evidence and in a way they can share easily. If I have time or resources, I might attempt a thorough assessment of OpenBSD in terms of design or code. A past study I did comparing it to prior secure UNIX attempts showed some things it's missing where there's surely issues. It has some other attributes, esp code quality & protections, that help it more than others (including old ones). Nonetheless, the issue I have is an extraordinary claim that pops up repeatedly in discussions over OpenBSD vs whatever with no evidence to back it and high chance of failure. See the problem? Also, do you see how I don't need to be an OpenBSD developer to spot it?
Totally agree with that.
> do they even analyze every bug for exploitability?
I guess in a real world production used operating system there are practical limits. From http://www.openbsd.org/security.html
"Our security auditing team typically has between six and twelve members who continue to search for and fix new security holes. We have been auditing since the summer of 1996. The process we follow to increase security is simply a comprehensive file-by-file analysis of every critical software component. We are not so much looking for security holes, as we are looking for basic software bugs, and if years later someone discovers the problem used to be a security issue, and we fixed it because it was just a bug, well, all the better. Flaws have been found in just about every area of the system. Entire new classes of security problems have been found during our audit, and often source code which had been audited earlier needs re-auditing with these new flaws in mind. Code often gets audited multiple times, and by multiple people with different auditing skills."
and
"Another facet of our security auditing process is its proactiveness. In most cases we have found that the determination of exploitability is not an issue. During our ongoing auditing process we find many bugs, and endeavor to fix them even though exploitability is not proven. We fix the bug, and we move on to find other bugs to fix. We have fixed many simple and obvious careless programming errors in code and only months later discovered that the problems were in fact exploitable. (Or, more likely someone on BUGTRAQ would report that other operating systems were vulnerable to a `newly discovered problem', and then it would be discovered that OpenBSD had been fixed in a previous release). In other cases we have been saved from full exploitability of complex step-by-step attacks because we had fixed one of the intermediate steps."
> This isn't theoretical: it's a direct implication of the claim they're making, an extraordinary implication, and one which deserves concrete evidence to back it.
Fair enough. Maybe they should change the claim to something like "Only two remote holes (known at the time of the fix) in the default install, in a heck of a long time!" Then again, isn't this what most readers would expect?
" Maybe they should change the claim to something like "Only two remote holes (known at the time of the fix) in the default install, in a heck of a long time!" Then again, isn't this what most readers would expect?"
As your quote says, they actually discovered and fixed a ton of them. Users thinking that claim is true might believe their system doesn't need patching if no new features are added. So, I'd ditch the claim altogether if I were them in favor of something reflecting what you quoted. Those quotes show the real argument which is lots of prevention and constant improvement at a higher rate than competing OS's for code quality. That mindset dictates one continue to patch and upgrade: no "fire and forget."
By default OpenBSD has not opened up and ports, daemons, sockets, or other resources that would allow a remote attacker to exploit it.
If I wish to start making changes to the default install, that is, modifying /etc/rc.local to start enabling daemons/services, then I need to take responsibility for those changes, and ensure that I stay current with the security issues associated with those daemons/services.
Most making a claim like that... one that's usually wrong.... have data to back it up. Most people's position has been that I should accept it by default, accept reversed burden of proof, and disprove the claim by countless hours of inspection. Whereas, from high-security to mainstream stuff, the status quo has been something is insecure until proven otherwise. I see no reason to change it.
There have been no remote exploits in the last 8 years in the default install.
My theory is supported by another post:
https://news.ycombinator.com/item?id=10337807
So, I'm dismissing their claim entirely and will counter it with their claims linked above along with my argument. However, even without that claim, the OpenBSD team can still make a great argument based on their QA processes and track record in fixing things as you linked. It will certainly have fewer 0-days than anything else which is why I told another OpenBSD developer I recommended Xen projects use it for Dom0 wherever possible.
Btw, thanks for the links. It answers the question about their defect tracking in general. Lots of detail in there and evidence of their QA working. As I expected. I'm keeping it to share if the topic comes up in another discussion.