Myths about /dev/urandom (2014)
2uo.de
2uo.de
And inevitably, the appeal to authority ends up referring to these projects.
Serious question, if I submitted a patch for the man page detailing the content of this myths page, is there a chance it might go somewhere and put somewhat of an end to some of this?
https://bugs.ruby-lang.org/issues/9569
I many technical discussions, the man page becomes the compelling argument.
Worth noting: WordPress has been using the OS's CSPRNG since 4.4.
https://paragonie.com/blog/2015/10/coming-wordpress-4-4-cspr...
https://core.trac.wordpress.org/ticket/39309
https://github.com/paragonie/sodium_compat
I'm currently writing a pure-PHP libsodium polyfill that, once audited, will be proposed to shore up the security of PHP projects the whole world over. Among other things, this means that everyone will be able to adopt Argon2i for password hashing even if they support PHP 5.2.4 like WordPress does.
Ideally, everyone would get in the habit of updating to the latest versions more rapidly. But when 27% of the Internet runs software that refuses to modernize, that becomes a huge risk to the rest of the Internet.
I would say go for it. It would probably make some difference (assuming the patch were accepted). People do frequently quote the man page when arguing about this issue.
PLEASE: If you wanr people to remember your takeways, write TRUE statements, not False statements with debunkings!
https://www.skepticalscience.com/Debunking-Handbook-now-free...
I'll just point to my comments from last time:
https://news.ycombinator.com/item?id=7363188 https://news.ycombinator.com/item?id=7364121
Can you be specific about why you believe Solaris urandom would be unsuitable for any specific cryptographic task?
The fact that the "Solaris cryptographic framework team" believes something to be true is inadequate evidence for me.
I don't follow - why would you believe that something is secure despite the developers of it saying it is not? If you trust that they are competent, wouldn't you trust a competent cryptographer who says that their code is insecure? And if you don't trust that they are competent enough to make that evaluation, why would the assumption be that they are still somehow able to write secure code, even if they can't correctly identify it as such?
You can find such people outside of cryptography as well: for example, the parent (or bodyguard) who won't let their child (client) leave the house because of all the deadly things that happen every day to people who leave their houses.
It's what happens when you combine a profession that relies on a certain amount of healthy anxiety, with an anxiety disorder.
The answer is always No, but they think about it for a few hours before they reply.
That doesn't make them incompetent! The lawyer comparison is a telling one. I have a lawyer I work with on contract review that I think is amazing. But that doesn't mean he's my best source of wisdom about litigation, because litigation is a very specific speciality of law practice, and most lawyers don't do it. Just like the OS crypto developers, he has to know a lot of stuff about litigation to do his job, and I respect that. But that doesn't make him a litigator.
The LRNG developers thought they were accomplishing something quite important with the /dev/random reseeding/blocking system. But as you've seen from the man page update, the consensus is, that thing they were trying to accomplish was in fact counterproductive.
The short version is that, on Solaris, /dev/random has certain guarantees that /dev/urandom does not and so if you are generating long-term keys or high-value keying material, you should use /dev/random.
While Solaris (over time) has tried to make the differences between the two as little as possible, for a variety of reasons, they are not identical.
As just one example, one difference between the two is that, for organizations or individuals with specific security requirements, /dev/random can be configured to use only hardware-based sources registered with the kernel-level cryptographic framework by disabling the software-based provider using cryptoadm.
The fact that the "Solaris cryptographic framework team" believes something to be true is inadequate evidence for me.
They are the domain experts, authors of said material, and my friends. I'm sorry that you don't believe them, but I've known some of them almost a decade or more and I have no reason to believe they have anything other than the best interests of others in mind when they provide this guidance.
In the end, you'll have to choose what to believe on your own, all I can tell you is that the Solaris crypto team provides the guidance that "high-value" keying material should be generated using /dev/random and that I have every reason to believe that advice is sound and competent.
Further: no matter what authority you're going to appeal to, I'm still going to look at what the systems engineering details are. According to the document you sent, Solaris urandom uses a cryptographic DRBG seeded from a kernel random pool. That's what the LRNG does, too.
In fact, if I was going to take your appeal to the Solaris cryptographic framework team seriously, I would also have to concede that Linux urandom was insecure --- because the LRNG team has maintained for years that it is inferior, and only this year is finally conceding otherwise.
What am I missing? Can you be specific?
So from a programmatic standpoint, developers should use /dev/random on Solaris if they believe the material being generated is "high value". It is up to the developer to determine whether the material being generated is "high value".
No case has actually been made for why the argument is different on Solaris than it is on Linux. I'm increasingly convinced that's because there's no difference, regarding this issue, between Linux and Solaris. The generators are different, but equally safe once the generators are seeded.
As a result, organizational or contractual requirements that a system administrator may have are only guaranteed to be met when using /dev/random.
This system configuration is specific to Solaris, which is why Solaris is different than Linux.
I'm becoming increasingly convinced that there is no difference between the urandom story on Linux and the urandom story on Solaris. Not that the generators are the same, but that the differences simply do not matter.
If you don't know the specific answer, could you get one of your friends on the team to chime in? I'm reaching a threshold at which I'm going to start noisily telling people that urandom on Solaris is fine --- incidentally, a lot of very well-regarded software already agrees with me, so I feel reasonably safe joining the chorus.
As I said before, and I as I will say again, the differences do matter for some administrators with specific contractual and/or other obligations and when generating "high value" keying material.
If you choose to advise others contrary to the documented guidance that Solaris provides, that is your choice.
Always use urandom. If your contracts require you not to, revise your contracts, not your code.
Use of urandom contrary to official guidance is not recommended.
[Edited to add: it looks as though the tweeting questions-at Darren Moffat protocol has been initiated: https://twitter.com/tqbf/status/817496091759362048 . It's also worth noting (which I failed to do previously) that two of the three random providers in the blog post are described as doing their own hashing/similar processing of the entropy bytes they provide to the random pool, and the other one appears to do so from the illumos source.]
> Use of urandom contrary to official guidance is not recommended.
This is, I think, a case where using the passive voice is suboptimal. The official guidance obviously does not recommend using urandom contrary to official guidance, nor do you. Some others do recommend it, as this whole argument shows.
More substantively, this is the part of your stance that would be, as tptacek previously suggested, equally applicable to Linux urandom prior to Linux fixing their man pages. At that time, the official guidance on urandom was that it was inferior to random in general instead of solely in the one specific case of requests before the kernel CSPRNG has been seeded. If the official guidance is incorrect about when and if urandom is inferior to random, then use of urandom contrary to official guidance should be recommended.
[1]: https://blogs.oracle.com/darren/entry/solaris_random_number_...
[2]: The blog post only briefly mentions the rndc_addbytes and rndc_getbytes functions where the entropy provision and randomness-extraction bottom out, so it is possible that it just omits the details of any additional processing performed at that level. But it at least does not mention any further processing performed on the bytes from KCF providers except in FIPS mode.
When I asked Darren about this in the past (paraphrasing from memory), the response was that constraints on the implementation and/or applied by system configuration ensure that bytes from /dev/random provide the highest quality random numbers produced by the generator, and so are the most suitable for high-value keying material.
So as I understand it, yes, it's more than just applicable to the case of requests before the kernel CSPRNG has been seeded.
Also, keep in mind that on Solaris live migration may mean that your process (well, the zone hosting it anyway) is live migrated to an entirely different system and so may be hosted by a different kernel without your process ever being aware of it. So relying on assumptions about the state of the kernel is inadvisable.
If I receive any additional information I can share, I will do so.
That's the second randomness canard introduced on this subthread (the first being that there is a kind of cryptographic random number that is suitable for IVs and nonces but not for "long-term" cryptographic secrets). The two canards are related, but not identical.
I doubt the Solaris KCF team is thrilled to be virtually interposed into this argument; it is unlikely that they disagree with what I'm saying, since I'm making a pretty banal observation about FIPS cryptographic DRBGs and about the plain meaning of the KCF random code.
The Solaris urandom story is, in practical (end-user) terms, the same as urandom's story on Linux. There's some confirmation of this on Twitter, if you care to look.
Once again, all information available to me contradicts your assertions:
"Bytes retrieved from /dev/random provide the highest quality random numbers produced by the generator, and can be used to generate long term keys and other high value keying material."
"While bytes produced by the /dev/urandom interface are of lower quality than bytes produced by /dev/random, they are nonetheless suitable for less demanding and shorter term cryptographic uses such as short term session keys, paddings, and challenge strings."
https://docs.oracle.com/cd/E53394_01/html/E54777/urandom-7d....
I doubt the Solaris KCF team is thrilled to be virtually interposed into this argument; it is unlikely that they disagree with what I'm saying, since I'm making a pretty banal observation about FIPS cryptographic DRBGs and about the plain meaning of the KCF random code.
Everything I've said has been taken from either the current documentation or from conversations I've had with the crypto team.
Since they confirmed the documentation is up to date and correct, then I don't see how your assertion can possibly be correct.
The Solaris urandom story is, in practical (end-user) terms, the same as urandom's story on Linux. There's some confirmation of this on Twitter, if you care to look.
I see no confirmation on Twitter from anyone that is currently working on Solaris -- only a past member that left the organization some time ago.
Until I have independent confirmation from the team involved, I'll have to agree to disagree.
It would be even better if there was some special file with the same semantics of getrandom, so the advise could be simply "use this file, or the getrandom syscal, whatever is easier". The way it is now, no special file is foolproof.
If you can just say "actually the man page no longer says that", that's pretty significant.
Previous HN discussions:
What I think is really interesting is the question: how do you prove something is random. If you take a chunk of data that uses strong encryption, without headers or other identifiers, it should look like it has a random distribution.
That's one of the reasons for the great concern around the Snowden leaks that implied some manufactures had compromised their hardware AES chips:
https://theintercept.com/2016/01/04/a-redaction-re-visited-n...
> Every software package we examined relies on /dev/urandom to generate cryptographic keys; however, we find that Linux’s random number generator (RNG) can exhibit a boot-time entropy hole that causes urandom to produce deterministic output under conditions likely to occur in headless and embedded devices. In experiments with OpenSSL and Dropbear SSH, we show how repeated output from the system RNG can lead not only to repeated long-term keys but also to factorable RSA keys and repeated DSA ephemeral keys due to the behavior of application-specific entropy pools.
This is mentioned a little at the end of the article. Would it be a breaking change for Linux to block urandom at startup?
And every few months, send an email to Ted T'so with a patch fixing this behaviour. Maybe if every single Linux user bugs him he'll finally capitulate.
2. Perhaps somebody could create one of those preloaded email forms like those political organizations do where you fill in your name and email and it does the rest up to and including sending the email?
This causes problems in practice, allowing people to crack RSA private keys. https://factorable.net/weakkeys12.extended.pdf
Also note (as TFA indicates) that urandom not blocking on uninitialised entropy pool is mostly a Linux thing e.g. BSD urandom will block until the system the system is correctly initialised.
That excludes anything that is meant to be used by other people. Fragile security is no security, and relying on undocumented assumptions like "is not running on Linux", "starts late in the boot process" and "is not running on an VM" is incredibly fragile.
Unfortunately people who really should know better keep writing articles like this one, dispelling the "myth" that urandom works exactly as documented.
Which leads us to situations like this:
$ dmesg|grep random
[ 0.469142] random: systemd-tmpfile: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.470297] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.470325] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.470877] random: udevadm: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.470890] random: udevadm: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.471936] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.471950] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.471969] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.472132] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 0.472142] random: systemd-udevd: uninitialized urandom read (16 bytes read, 3 bits of entropy available)
[ 1.909082] random: nonblocking pool is initialized
Oops, hope none of that was for anything important, say generating a long-term cryptographic key! But of course if we're to believe TFA... "Fact: /dev/urandom is the preferred source of cryptographic randomness on UNIX-like systems."Just use getrandom. If you can, consider using only getrandom. But if your code must work on systems without getrandom or similar, before following the advice of this article, ask yourself if what you are doing is worse of blocking, or reading out a grand total 3 bits of entropy. You'd probably prefer the former.
The bug report also explains what systemd is doing here.
"systemd starts allocating those hashtables very very early on, before any process is forked off... It's par tof the initialization scheme of systemd really. Hence there's basically nothing else going on in the system, execept what is done by the kernel itself", -- Lennart Poettering, 2016-05-05, https://lists.freedesktop.org/archives/systemd-devel/2014-Ma...
"moving that blocking behavior to /dev/urandom simply does not work. The system does not boot. The reason to this issue is actually quite simple. The init process of systemd reads /dev/urandom for whatever purpose. Now, when /dev/urandom blocks during boot, systemd will be blocked too." -- Stephan Mueller, 2016-10-22, https://lkml.org/lkml/2016/10/21/982
Maybe prefix the bold sections with Myth: so it is easier to read. I was really confused as to which side of the argument the author was on at first...
But unless you are doing something very tightly integrated with Linux, you should just ignore them.
It appears the Apple and OpenBSD implementations use the getentropy syscall and then add additional entropy mixing.
https://media.ccc.de/v/32c3-7441-the_plain_simple_reality_of...
I'll have to link from media.ccc.de from now on.
1 - As in "oh just keep it there, it's easier than convincing everybody that it's useless".
Is there a good reason why /dev/random AND /dev/urandom exist?
The new version of the man page resolves this problem, and says outright that urandom is the preferred interface, and that /dev/random is obsolete; applications that run during early boot time should instead use the system call interface.
In what sense is this not a "low quality random number"? Every CSPRNG I've seen will output numbers that pass many statistical tests for randomness even if seeded with e.g. zero - is that not a "low quality random number" in the usual sense of those words?
The randomness has the same quality. It's the same pseudo-random number generation algorithm. Only in one case, the world knows your seed value, and can predict anything that you do that derives from pseudo-randomness; whereas in the other case, the world does not know your seed value, should your seed value be discovered somehow you are regularly re-seeding anyway, and the world cannot predict your actions.
I wish rngd was easier to use with multiple entropy sources, however. Even w/ RDRAND (times two -- dual CPUs), a TPM, and the TrueRNG, it's difficult to (easily) tell which of these are being used and/or if more than one is being used.
Ideally, I'd like to be able to tell rngd to use/mix the TrueRNG, RDRAND in both CPUs, the TPM, and any other entropy sources I may come up with, such as RTL-SDRs doing funky things [1]. I suppose I could just run multiple instances of rngd, though.
Anyway, I've switched to just using /dev/random for pretty much everything (where it can be configured) since, with the TrueRNG, it never blocks on me now.
Do I absolutely require it? Probably not. For $50, though, I thought "might as well try it".
That said, if you're looking for more of the how and less of the why: https://paragonie.com/blog/2016/05/how-generate-secure-rando...
Contains snippets and recommendations for C/C++, Erlang, Go, Java, (Browser) JavaScript, .NET, Node.js, PHP, Python, Ruby, and Rust.
With these instructions you'll always get a "good"[1] seed for your CSPRNG, and that includes virtual machines and clones.
[1] Of course that depends on how much you trust Intel. Don't ask the crypto mailing lists whether this is a good idea :-)
"Create a new function, get_random_bytes_arch() which will use the architecture-specific hardware random number generator if it is present. Change get_random_bytes() to not use the HW RNG, even if it is avaiable.
...
So it's much better to use the HW RNG to improve the existing random number generator, by mixing in any entropy returned by the HW RNG into /dev/random's entropy pool, but to always _use_ /dev/random's entropy pool."
http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.g...
Intels instructions perform the same as a HWRNG only it's built in to every chipset >= Ivy bridge. I actually think Linux's rng-tools incorporates it as a mix automatically.
I remember learning this when I found /dev/urandom multiple times slower than /dev/null while writing garbage data to disk.
http://www.pcworld.com/article/3117728/security/why-quantum-...
I fail to see how these might be related.
Cat videos are much too popular and a lot bigger, and since I'm paying for traffic... [0] ;-)
[0] regular traffic: ~30 MB/day, today (projected): ~3.5 GB