I would see how much effort ingelligence agencies would need to handle the proliferation of many privately developed ciphers by the open-source community.
What do you think?
I’m in the industry - I work on cryptography research and I review cryptographic protocols for security and correctness. I can tell you when I warn people not to roll their own crypto it’s not because I’m shilling for the NSA.
It’s really very simple. Cryptography is an extremely complex discipline that requires specialized training. For the most part, engaging in complex disciplines without the requisite training results in errors somewhere. Unlike software from many other such complex disciplines, cryptosystems are 1) safety critical, and 2) compromised by extremely minute errors in design or implementation.
Speaking frankly, I don’t think it’s wise for you to try and develop your own machine learning software for a production system without training, either. But at least if you do that you’re not (likely) compromising your users. We admonish untrained engineers for doing the same with cryptography because it requires either great ignorance or great hubris to believe you can pull it off without disastrous errors along the way.
Look at the recent INTEL spectre failure, you can see how extremely complicated systems were flawed at their very bottom and few young enlighted minds were able to shake our world.
Do you really think if there were no widespread recommendations of crypto systems considered good people would suddenly start becoming expert cryptographers (and companies would suddenly pay for them doing that)? History suggests that they'd rather grab something random of GitHub/Stack Overflow or make up their own without understanding it properly. Every security conference is providing you with tons of examples of people getting it horribly wrong despite there being good free resources available providing them with the tools to avoid their failures.
So I am not saying that a newbe will do a better job than experienced experts. I am saying that we will not have many experienced experts without seeing today newbies making mistakes.
To me seems not a complicated concept.
It's not. People not wanting to learn cryptography systematically make impossible for themselves to grasp the basics of crypto. There are a lot of topics to be covered, which goes way, way beyond encrypting stuff.
People dabbling with making their own cipher or implementing DES on their own would learn exactly nothing of the cryptography. You really need to be shown a lot of stuff to understand where the industry and research were decades ago, much less where they are now, and you can't really learn that on your own, you need a guide to show you where your knowledge is still lacking.
To be brutally frank with you: most people are not qualified to implement their own cryptography, and you cannot become qualified if you skip the significant prerequisites. Much like computer science in general, cryptography is not comparable to software engineering or programming.
You can learn e.g. web development in a piecemeal, circuitous fashion. You cannot learn cryptography that way - you’d first need to understand advanced mathematics (abstract algebra, number theory, probability theory, maybe some game semantics) and computational complexity. Then you need to thoroughly understand the existing algorithms before you figure out how they can be (safely) improved.
Attempting to develop novel cryptosystems (“roll your own crypto”) without this background is irresponsible. It’s not that it’s impossible to grasp the basics, rather it’s that you need to be willing to work through a significant amount of material before you can even understand what your “unknown unknowns” might be.
To put it very succinctly, would you attempt to publish new research in computer science or mathematics without significant training in either?
If you want to learn about cryptography there's lots of resources (uni courses, books, and informal tutorials like matasano crypto challenge). If you're interested enough to learn, there's nothing stopping you. For the record, I've done all 3 and apart from some practical cryptography stuff, I really learned one thing: don't roll your own crypto - it's super tricky :-D
We will have new experts though. There are people interested enough to continue and learn and they will get their work reviewed by standard bodies one day.
Are you aware that virtually all of the researchers behind the Spectre and Meltdown attacks are academics, right? Of those not presently in academia, they still mostly have PhDs and work in private research labs.
Your comment here about “few young enlightened minds” implies that scrappiness, grit and an unorthodox perspective from outside the system enabled them to find these vulnerabilities. That’s not all the case. All the researchers involved are highly trained by the same institution your criticizing. They are quite literally in the same group of specialists and experts you hold in such contempt.
Are you sure that's not just your view of some specific group you know? People are definitely able to learn. Experts even more so - but they choose their discipline already.
Anyway, I'm not sure why this is a response to "here's an example of people whose income is directly related to the quality of their crypto implementation, see how they fail".
I can certainly tell you that you can do tons of mistakes at every layer. Protocol, primitives, composition of primitive, how do you implement them in sw or hw, how do you optimize them.
As an example you can see what happened to TLS1.2, looking after the fact basically everything was wrong with it.
Today I work for a large Internet company. I would be absolutely terrified if I had to see custom crypto implementations. There simply isn’t enough time to validate all that needs to be validated.
Back to your point, though, I’m not against the “build your own crypto”, provided that you have proven expertise. Typically, if you are really an expert, you want to consult with other experts and create a solid team that can verify your implementation.
And that being crypto-ignorant is not connected to only using known tools for real systems. In many cases, self-made crypto is done from an uninformed perspective, which is why it often is so bad. Just as using well-known crypto doesn't mean you haven't spent the time trying to understand why it is recommended and how it works internally, why alternatives are bad, or having made your own crypto and learned from that.
Some algorithms are recommended by government agencies, yes, but they mostly are not made by them, but are choosen from publicly suggested and studied algorithms. And the newer algorithms or variants considered state-of-the-art generally not government recommendations, since those processes are slower than public research - when researchers find new variants or find weaknesses in old ones software and open standards adjust far quicker than e.g. NIST recommendations.
What you’re now suggesting is very close to conspiracy territory. Most cryptosystems are developed outside the government. Moreover, if you look at the authors of many of the most prominent cryptographic primitives, they aren’t exactly fans of the NSA.
Do you think that the NSA has backdoored the algorithms to the extent that the entirety of academia is in on it? That’s what it would take for what you’re suggesting to be true, because there’s a huge incentive to catch this if it’s occurring.
I studied crypto in college on the mathematical side. RSA and ECC are based on very sound maths