Not to be pedantic but the gotcha is that you can't know they're using the open source software as-is. If they run a hosted service or distribute binaries you won't know. Also with cryptography any change (diverging from the open source software) can have regressions.
It just seems like such a chicken/egg problem. Where does the actual trust come from (like holy crap web certificates seems unbelievably broken).
Ultimately it seems like it's just impossible to be 100% for sure what is running on another persons server without having access to it. Which is unfortunate.
If the client software is all that's ever supposed to see plaintext, being able to see source allows you to confirm that that is (probably) the case and then compile it yourself rather than trust that they haven't thrown an extra step in that backdoors it.
It's not perfect, but IMO this is one of the best parts of Open Source, I can audit it if I want to. Closed Source strips me of that option.
Edit: Clarification.
Sometimes it's to find out what the heck is the cause of a particular behavior in a program, sometimes it's to know for sure that the program isn't trying to do anything that I recognize as malicious in a security sensitive environment, other times it's to see exactly how a game is calculating whether or not my bullet has hit the enemy (server side calculation is more difficult to fake than client side).
Would you honestly chose a black-box solution for a business critical need, knowing that it could stop working at any time and won't let you know for sure that the code is secure by auditing it (or paying a trusted security professional to do so for you)?
I get the impression that the anti many eyes sentiment comes largely from non-programmers, am I wrong about that?
I've only heard it from programmers, generally very good ones. Anyone who is at all following the security community knows that many eyes is possible but generally very optimistic. That's why so many people were glad to see Heartbleed lead to the Core Infrastructure Initiative since that will keep the guaranteed number above zero for some key projets.
I think this is true, but I also think that a lot of people have seen statements from authoritative people to this effect and taken them farther, as a complete rejection of not just the scale of the effect of 'many eyes', but a rejection of the fundamental idea, which leads to a conclusion that the source being available is either worthless or even detrimental.
The Core Infrastructure Initiative is not at odds with the basic notion of many eyes, but augments it. Arbitrary groups (particularly groups with non-commercial motives) committing monetary resources is also enabled by open source in a way that is impossible with closed source, after all.
I can only speak for myself. I am a long-time programmer and security professional and I argue against the "many eyes" sentiment.
A significant portion of the projects that I assess code on I don't have the source. And yes, I find security-relevant bugs in that code.
There are claimed black boxes and "open" black boxes. On a linux system, do a "top" and tell me how many of those hundreds of open source programs the eyeballs have actually looked at and can testify to the absence of bugs or presence of trustworthiness?
Sure, the many eyeballs theory is appealing, but it seems more aspirational than actual.
You're also missing that often BigCorp gets more involved in open-source than random individuals. Microsoft for example is said to be the fifth largest contributor for Linux 3.0, speaking of which Red Hat, IBM and Google are regulars and now Samsung too.
Fun fact, did you know that SELinux, one of the most advanced modules for access control, was originally developed by the NSA? Yup, a little ironic, but we can use it because it is open-source and because it has been reviewed.
Closed source does not strip you of the ability to audit.
In the case of BitTorrent Sync you can use Wireshark to inspect the network traffic yourself. BitTorrent even goes so far as to purposefully use plaintext for the usage statistics it reports back so that someone could cross-verify with Wireshark. Besides Wireshark there's all sorts of tools for instrumenting, debugging, or decompiling Sync that would also fall within the realm of auditing.
As other commenters have noted, it is not exactly trivial to verify that a binary for some open source software was produced by the same open source code you audited. This leaves you with having to compile from source everything – which seems only an order of magnitude or so less annoying than Wireshark-ing or IDA-ing everything.
Perhaps a better approach to handling the security concerns of open vs. closed source software would be to take a more active approach to locking down what we run. Lets operate with the working assumption that whatever we run is hostile instead of having blind trust in open source. Lets use firewalls, containerization, selinux, etc. to essentially configure whitelists for what we allow of the software we run.
For someone that doesn't trust BitTorrent Sync because it's not open source consider locking it down. Use a firewall to only allow connections to known peers, isolate it from other processes, and restrict filesystem access. Trade some usability for security.
The kinds of things y'all are concerned about with BitTorrent Sync are the same things we should be just as concerned about for what we just apt-get'd; closed source vs. open source doesn't make a lick of difference in that respect.
On a practical level there are clearly limits to the many eyeballs hypothesis. Particularly if your assumptions about it were based on the idea that every user is an 'eyeball'. It's obvious there isn't a linear relationship between the two. None of that means it isn't still better than the alternative.
The claim is not that it is a failure of open source, but that is a failure in the "many eyeballs" claim.
Looking at two fairly famous bug hunters, Juliano Rizzo and Thai Duong finding POET, BEAST and CRIME, didn't require open source to find these bugs. BEAST was identified in 2002, but wasn't taken seriously.
Not having source is not much of a speed bump for a bug finder.
This "many eyeballs" is giving us all a false sense of security.
I agree there is an element of false sense of security, but I fail to see how the answer to that is to actively encourage our security software to be closed source.
As someone who has found bugs in software thanks to having access to the source code, I call bullshit on this. I likely would not have been able to identify the bug myself without access to the source.
It is true that you can find bugs in closed source software, but it sure as hell is a lot easier when we have access to the source.
(I'm aware that code doesn't have to be Open Source for me to have access to it, but I feel that's splitting hairs.)
Precisely!
This issue is further amplified by the fact that lots of people believe that "open source" means "trustworthy". If someone opens up their code, they must be good guys. This assumption is very easy to exploit. All a bad player needs to do is to distribute both the source and the binaries, but build latter from an alternative source. Just look how long it took for someone to actually try and verify that TrueCrypt binaries were in fact built from the source supplied. And that's for a security product with a massive installation base.
Step one in verifying their claims is providing the code. Step two is finding qualified people who are interested in reviewing it.
That review may never happen even if the source is available, but it's guaranteed to never happen if the source is not available.
So yes, it is correct to say that it is not required that the project be made open source to corroborate their claims, but the alternative of keeping it closed source and having third parties verify every release is much more logistically challenging.