Linux RNG RFC Patch: implement getrandom() in vDSO
lore.kernel.org
lore.kernel.org
Seperately, I'm also a little sad that there are no distinct RNGs for secret and non-secret output. It is an indispensable layer of defense to have separately seeded RNGs for these use-cases. That way if there is an implementation flaw or bug in the RNG that leads to re-use or predictability, it is vastly harder for that flaw to be abused. s2n, BouncyCastle, OpenSSL, and more user-space libraries use separately seeded RNGs for this reason and I don't think I could justify removing that protection.
Banks and government data processors alike are often forced to use things with FIPS’ lesser security. FIPS is where NSA is known to have sabotaged cryptography. NIST is still disgraced by the tip of the backdoor/sabotage iceberg. Yet we are still stuck with NIST, and indeed with various lesser standards which we can generally just call FIPS.
It is lame and sure, we should build better things. We should ignore FIPS where possible. We probably agree there, but it seems unreasonable to ignore all the systems which cannot be made better by intentional limitation.
Ignoring FIPS doesn’t change that many important systems do use FIPS’ cryptographic constructions. It would be nice if the U.S. government wasn’t actively sabotaging the security of standards with backdoors. It would also be nice if the U.S. government didn’t require anyone at all to use FIPS. Too bad we can’t have nice things in many important sectors.
Acceleration won’t fix the systemic issues here. Ignoring the systematic failure of (intentionally) weak FIPS standards will only further create division. Non-compliance will sideline reasonably secure modern systems in important contexts.
We shouldn’t need to fix FIPS, but what other alternative will help users whose data is protected by FIPS cryptography in the FIPS legally mandated contexts? Ignoring it isn’t going to change the law, ignoring it won’t secure those systems. OPM leadership probably really wished they had ignored FIPS. That wishful thinking won’t repair the damage to national security done by just one NSA/NIST FIPS backdoor being exploited in that single prominent example case.
My understanding of the OPM breach is that it was a classic example of poor access control combined with phishing.
Wired: https://www.wired.com/2015/12/the-years-11-biggest-hacks-fro...
IACR pre-print: https://eprint.iacr.org/2016/376
Later published in ACM: https://m-cacm.acm.org/magazines/2018/11/232227-where-did-i-...
Here’s one of the academics from the papers I linked above explaining his “pet” theory: https://mobile.twitter.com/matthew_d_green/status/1433476266... The papers explain more in detail, and the ACM publication seems reasonable to characterize as peer reviewed for some value of that depending on your impression of ACM.
You asked for “any evidence” and those papers explain the how, while Wired explains the what and reporting on the Project BULLRUN clearly lays out the why. It is undeniable that U.S. government sabotage to benefit NSA surveillance is involved in deployed Juniper devices. It is also widely discussed that those sabotaged devices were used by OPM. It is understood by many reasonable people in the know that OPM was hacked by Chinese intelligence and that they likely did this by compromising at least one Juniper device. See https://www.bloomberg.com/news/features/2021-09-02/juniper-m... for the Chinese link, for one example.
Bloomberg says: “The NSA introduces an algorithm called Dual Elliptic Curve...”
“...which it urges the National Institute of Standards and Technology to approve.”
“The Pentagon—NSA’s parent agency—insists that Juniper Networks use the algorithm as a condition for some contracts. Juniper adds it to NetScreen’s ScreenOS software.”
“This introduces an alleged backdoor that could be used to spy on select NetScreen customers, which include telecommunications providers and government agencies. Juniper includes two tweaks that it says neutralize the vulnerability.”
There are other sources which investigated this topic.
You don’t have to connect the dots, or believe the reprint from several reputable journalists or academics but I would appreciate you acknowledging that it’s not something that I made up. Many reasonable people have concluded that the NSA sabotage played some role in the OPM hack as a result of the link to Juniper.
The attack against OPM leveraged juniper bugs, which clearly exceeds your claims. I acknowledged that your claim is at least the bare minimum possible. However there is ample evidence that is goes much much further. At the time this happened, people pushed back that it was even plausible, and the paper represents a better investigation than I will be able to provide on a message board.
What I hope we can also agree on is that a complete lack of transparency means we cannot fairly adjudicate this in public to completion. Juniper, the FBI and the NSA, and indeed NIST, have not provided a full analysis to the public or even to the relevant IG as far as I understand things.
See also https://en.wikipedia.org/wiki/Office_of_Personnel_Management... for many other sources, as well as https://fcw.com/security/2021/01/lawmakers-press-nsa-for-ans... and this gem of a FOIA: https://www.muckrock.com/foi/united-states-of-america-10/inf...
So since your stated request was any evidence, I think I have shown that I didn’t make this up out of the thin air.
For encryption, you could have a FIPS-compatible TLS connection over a non-FIPS Wireguard tunnel and again be no worse off (excluding performance etc reasons), whilst gaining the benefit of not having to trust one implementation exclusively.
Are there any standards that you are aware of that would prohibit mixing FIPS and non-FIPS entropy and encryption techniques as described above?
CPUs are “Enabled” and listed as a successful operation by NSA. Standards which are implemented in hardware and software are similarly backdoored. Protocols are designed to leak enough information to exploit various things at various levels. The new push to post-quantum without using mandatory hybrid constructions that include something we at least strongly believe to be (contemporarily) secure means that we should expect problems at literally every level from hardware to the newest key exchange mechanisms.
Can we theoretically build a baroque machine that combines something we trust with something we explicitly assume is compromised? Sure, and I like XOR as much as the next cryptographically knowledgeable person.
Should we? No, absolutely not. More importantly, will everyone who is required to use FIPS? Certainly not. Is it even allowed by auditors? Doubtful but maybe you can become one and bless your own solution. It’s still not a solution for everyone, not even close.
At the point where we start to even try, might we ask ourselves why we tolerate NSA and NIST doing this to the American public? Why don’t we understand what they planned to do when someone like you proposed things like you have done? After all, there is no question that NSA was playing the long game - they just believed their own NOBUS propaganda. Whoops.
But how do these user-space libraries get the entropy for their RNG? If they read it from /dev/random or /dev/urandom or call getrandom(), it's the exact same algorithm.
To put it another way: even though this is running in user mode, it's not a user-space library; it's part of the Linux kernel, which happens to be running in user mode. If for some reason you need to make getrandom() use FIPS algorithms, you already have to patch the kernel, and when doing so you'd patch this code to match. Because this code is the getrandom() system call, which like a couple of other "fast" system calls like gettimeofday(), is implemented partially in user mode and partially in kernel mode.
I wasn't aware that FIPS required your _entropy source_ to be validated in order for your library to be validated, though. BoringSSL, for example, just reads from RDRAND, getrandom(2), or /dev/urandom. Maybe the OS/CPUs it's certified for all have FIPS-validated entropy sources?
The older e.g., 36xx series Google cert predates that requirement iirc, when you could seed from a non-FIPS kernel.
You bite off a chunk of data from the RNG and use it as the seed for a sequence based on a cryptographic hash, relying on the non-repeating qualities of these algorithms to give you a convincing random data stream that is not necessarily suitable for generating cryptographic keys.
The problem here, if there is one, is that if you implemented your RNG source as just reading from a file descriptor, the subsequent bite of that proverbial sandwich gets the entire tomato slice and most of the lettuce. If the language you picked already abstracts /dev/random, then it's just a couple lines of code difference for you. If not, well then welcome to the wide world of cryptography APIs, friend.
It would be nice to have companies collectively reject FIPS. I realize that that's a pipe dream, but we do it with patents. Lots of companies pool their patents to defend against trolls on the condition that none of those companies sue each other over patent violations.
If a bunch of companies were like "We won't require you to be FIPS, and we won't get FIPS" or something that'd be cool. Unfortunately, that can't happen with FIPS specifically as it's government mandated, but idk, maybe SOC2 or something?
I hate things so much
No American should mind, they obviously have your best interests in mind. Unless you filled out an SF-86 form that was stolen from the nice folks at OPM. Whoops.
FIPS does not impinge on me in any way whatsoever - that's my real world and that's the real world for a darn sight larger number of people than ... Americans.
I note your secret/non secret discussion for RNGs and that seems to be a good idea - many non FIPS standards also have a dichotomy like that or something more complicated. However, separately seeded pools reduces the entropy available (IANACryptographer) that's a trade off of some sort that I'm not qualified to assess.
This work is done by people I respect and I will evaluate it according to the standards I have to adhere to. None of those standards is FIPS. I'm not quite in a cargo cult state but I have to take a certain amount of things on trust when it comes to this stuff!
Currently userspace has incentive to roll their own RNG stuff. This removes that, which is good for everyone. The less incentive you give people to write code that has already been written by other, more experienced people, the better.
I would go even further and export the kernel ciphers via vDSO. Then user space could rely on those ciphers being optimized for the host CPU and side channel free instead of everybody bringing their own crypto primitives. I don't think there is a good reason why gnupg and openssl would bring different crypto primitives.
https://www.kernel.org/doc/html/latest/crypto/userspace-if.h...
This way may actually have advantages over vDSO. Maybe you can set up IV and key with the kernel and then let the kernel do the crypto without having to have them in user space memory anymore. That would be a way to reduce risk in crypto applications, as long as you can prevent an attacker who has taken over the crypto app to retrieve the keys from the kernel. Maybe seccomp can help here.
Very exciting!
The vDSO is part of the Linux kernel binary interface and considered stable. There's documentation for it on the kernel tree:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
The vDSO functions are called using the C compiler's ABI for the platform. It's completely optional infrastructure that offers higher performance, normal system calls can still be used.
The address of the vDSO shared object is passed by the kernel to the program through the auxiliary vector, a list of key-value pairs. It's located on the stack right after the environment vector. The key for the address of the vDSO is AT_SYSINFO_EHDR.
There's more useful data in there too: system page size, CPU capabilities, loaded program's own ELF header and entry point locations and even its file name, user and group IDs, some bytes of random data. In most cases glibc will be the consumer of this data but it's perfectly possible to use of it ourselves.
More about the auxiliary vector:
https://lwn.net/Articles/519085/
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...
I'm a kernel expert, so I don't know if VDSO is the right implementation, but the idea seems sound. Make it harder for people to make mistakes that break security!
On Linux, traditionally, the userland-kernel interface itself is ABI-stable. The userland code can be fully custom and doesn't even need to support dynamic linking. Syscall numbers and arguments are fixed, and application code can perform its own syscall instructions. You can then layer something like glibc on top of that, which provides its own syscall wrapper functions with a corresponding stable (userland-to-userland) ABI, but that's separate.
The vDSO has always been a step away from that. It's userland code, automatically mapped by the kernel, that provides its own system call wrappers. Applications are still allowed to make system calls manually, but they're encouraged to use the vDSO instead. Its original purpose was to allow certain functions such as gettimeofday() to be completed in userland rather than actually performing a syscall [2], but it's been used for a few other things. It's worked pretty well, but it does have the drawback that statically linked binaries no longer control all of the code in their address space. This, for instance, caused a problem with the Go runtime [3], which expected userland code to follow a certain stack discipline.
Anyway, this patch seems to me like a significant further step. Not just putting an RNG into the vDSO, which is more complicated than anything the vDSO currently does, but also essentially saying that you must use the vDSO's RNG to be secure (to quote the RFC, "userspace rolling its own RNG from a getrandom() seed is fraught"), and explicitly choosing not to provide stable APIs for custom userland RNGs to access the same entropy information.
I don't think that's necessarily a bad thing. It's not that complicated, and to me, macOS' and Windows' approach always seemed more sensible in the first place. But it's a step worth noting.
[1] https://github.com/jart/cosmopolitan/issues/426
[2] https://man7.org/linux/man-pages/man7/vdso.7.html
[3] https://marcan.st/2017/12/debugging-an-evil-go-runtime-bug/
Right; more accurately stated, you must use the vDSO's RNG to securely get random numbers quickly.
> Even before the vDSO, I think one would have made the same claim about getrandom.
Because of VM forks. But an alternative would be to expose VM fork events more explicitly to userland, so custom RNGs can take them into account.
And I'm not sure that fixing the RNG, without providing a generic way for userland code to take forks into account, is enough to avoid all vulnerabilities. After all, there will always be a window between generating random numbers and using them. As a dumb example, suppose a DSA implementation first picks a random nonce, then reads the data to be signed from some external source. If the VM forks in between those actions, and the two forks choose different data to be signed, you're in trouble no matter how good the RNG is.
I said it's a dumb example because there's an obvious fix (picking the nonce after the data to be signed). But as a nonexpert, I'd be kind of surprised if there wasn't some cryptographic protocol where forking midway through operation is inherently insecure, and the only solution is to abort and retry. Heck, couldn't that apply to TLS?
Actually, I see that Jason submitted an earlier patch [1] that does provide explicit fork events. I don't know whether this vDSO patch is meant as a replacement or a complement.
[1] https://www.spinics.net/lists/linux-crypto/msg63759.html
i think that's mostly scientific computing where you want the ability to control the RNG and even intentionally use deterministic seeds for reproducibility.
i think if the kernel is going to provide secure random numbers (which seems like a good idea), it should be through a (new) specific system call that fails unless a hardware entropy facility is available. performance seems like a secondary goal, where the primary is ensuring that people are using the right thing to generate keys and such.
This assumes you aren't starved for entropy on initialization. If you are, it would imply a constrained environment and you are better off using getrandom() then.
Since you cannot know how many bits of entropy getrandom() would give you and when it itself is reseeded with fresh entropy, you usually have your userspace PRNG sample getrandom() for some amount of time, after which it is considered initialized.
Who is “you”? Calling getrandom() extra times to get extra bits on the hope that the result is magically better is entirely useless.
All you have to do is 'merge'/'absorb' multiple getrandom() results over some time t into your userspace PRNG, with t big enough to allow for multiple reseeding events by the kernel. You are in effect getting samples of the new entries into the kernel entropy pool indirectly by doing this, sampling too frequently is wasteful however as you are just going to be sampling the kernel CSRPNG with the same entropy in its state most of the time.
Doing the above is actually the only way to properly seed your userspace CSPRNG if your userspace CSPRNG has a state larger than the kernel CSPRNG (kernel's is 512 bits I believe) otherwise you will be working with a 512 bits of entropy as an absolute maximum even if your CSPRNG is capable of holding more, at least until reseeding (both the kernel and the userspace PRNG).
The vDSO-based approach is certainly interesting because the kernel can know exactly where randomness caches are located and zap them as needed (on fork, periodically, after VM resume). But if entire pages of memory are dedicated to buffers anyway, MADV_WIPEONFORK is sufficient for now.
Actually, it does. Look at the use of v->generation.
If I call the getrandom system call, and it succeeds, I am (pretty much) guaranteed that the results are properly random no matter what state my userspace program might be in.
With vDSO, it seems we lose this critical guarantee. If a memory corruption occurs, or my process’s memory contents can be disclosed somehow (easier to do against a userspace process than against the kernel!), I don’t have truly random numbers anymore. Using a superficially similar API to the system call for this seems like a really bad idea.
Silly mistakes like the Debian randomness bug come to mind - a program can be totally well-behaved even in the face of a glaring entropy failure, in a way that’s hard for developers to detect.
Yea, that's definitely a downside of sorts. Jeffrey Walton mentioned that in the glibc discussion a few days ago: https://lore.kernel.org/linux-crypto/CAH8yC8n2FM9uXimT71Ej0m...
A mitigating factor might be that if your process memory leaks, then the secrets generated leak anyway, no matter what generated them, so maybe not as large of a difference. But of course a generator leaking means future secrets potentially leak too. I suppose frequent reseeds could mitigate this, just as they do for potential leaks in the kernel.
But anyway, I agree that compromising application memory is somewhat more possible than compromising kernel memory, though both obviously happen.
The article mentions this case too. getrandom() on my system seems to return the required amount of random bits to perform a shuffle of a deck in less time than my clock seems to have precision for; … that's … too slow?
As the post says, it's worth entertaining the idea of having the kernel provide a blessed way for userspace to do that, though I admit I've never personally seen a scenario where RNG was truly the bottleneck. But it'd still be nice to kill all the custom RNGs out there.
How frequently does the kernel populate these pages with new entropy (for every process, pre-emptively?)
Does this avoid pagefaults or other process interrupts? Don't want process interrupting every time it accesses the 'magic page of random'.
Surely these are all pain points that Intel's `RDRAND` is supposed to alleviate.