Hacker News Reader

It's just "because entropy used to be really difficult!" We now have relatively simple and proven algorithms (Yarrow, Fortuna) which can take 256-bits of entropy, and deliver effectively infinite streams of high quality entropy from that core without leaking any of the seed, or skewing the output. In such a system, pulling bits out of the core entropy bucket is counter-productive.

As djb says;

  Cryptographers are certainly not responsible for this superstitious
  nonsense. Think about this for a moment: whoever wrote the /dev/random
  manual page seems to simultaneously believe that

    (1) we can't figure out how to deterministically expand one 256-bit
    /dev/random output into an endless stream of unpredictable keys (this
    is what we need from urandom), but

    (2) we _can_ figure out how to use a single key to safely encrypt many
    messages (this is what we need from ssl, pgp, etc.).

  For a cryptographer this doesn't even pass the laugh test.

BSD and OS X are more modern than Linux in their choice of CS-PRNG; they used to use Yarrow, and then added Fortuna support in 2014 (behind a config flag) and will make it default in 11.0. Linux has used largely the same CS-PRNG for /dev/urandom since 1994, with some upgrades to their mixing function, to prevent DoS against /dev/random, and upgraded recently to include random bytes extracted from "RDRAM" provided by Intel CPUs.

I'm not sure why we haven't seen the underlying CS-PRNG switched wholesale to Fortuna in Linux. For example, they are still using a SHA-1 based output hash, which seems like it might be time for that to go.

The higher level take-away is that trying to avoid the modest code complexity of the CS-PRNG by going directly to /dev/random is more likely to lead to an exploitable catastrophic failure than sticking with /dev/urandom.

The advice of /dev/urandom vs. /dev/random is case-specific and doesn't apply to all operating systems. For Linux systems, in general, it would appear it doesn't matter and /dev/urandom is sufficient.

However, on systems such as Solaris, the cryptography group (whom I know personally) specifically recommends the use of /dev/random in some cases. Those cases are largely limited to specific cryptography requirements such as a hard requirement for high entropy sources.

Regardless, all of this is largely moot at this point, as all *NIX-like systems (including Solaris) seem to be standardising on the getrandom() (in Linux first) and getentropy() (in OpenBSD first) which sidestep many of the more subtle, potential issues.

Can you personally ask the Solaris cryptography group why, in detail they think this is the case? No explanation I've seen for this makes sense.

In particular: the Solaris documentation suggests that urandom is fine for nonces, and /dev/random for long-term keys. But that's the opposite of the normal threat model for randomness! You care most about high-quality continuous-availability randomness for the nonces, where biases can be devastating to security, and attackers get repeated continual bites at the apple.

If someone can explain the logic here, I'll update the page that AGL linked to in this article to account for that.

I think the Solaris people are wrong. I'm not sure, but I'd be willing to bet a small amount of money on it.

This is a very interesting point. An RNG that fixes the high bit of every word to zero still makes reasonably secure private keys, but start signing stuff with DSA and that RNG and you're screwed.

Can you personally ask the Solaris cryptography group why, in detail they think this is the case? No explanation I've seen for this makes sense.

I have, and again, most of the reasoning is Solaris-specific.

The primary difference between urandom and random is currently higher quality re-keying for /dev/random, but there are a few internal implementation differences as well.

Fundamentally, it's about constraints placed on the implementation that currently make /dev/random more "secure". In addition, for Solaris /dev/random (and getrandom(GRND_RANDOM)) when the administrator explicitly configures a specific validation mode then the bits you get come from a validated DRBG (Deterministic Random Bit Generator). /dev/urandom is not affected by that validation mode.

I think the Solaris people are wrong. I'm not sure, but I'd be willing to bet a small amount of money on it.

They're really not in this case, but in fairness, you'd have no way of knowing about this without source code access and a few decades of knowledge about Solaris crypto.

You can find a summary from two years ago written by one of the primary Solaris cryptography engineers and architects here:

https://blogs.oracle.com/darren/en_GB/entry/solaris_random_n...

...they intend to update it soon for the next Solaris release.

Yeah, I've read that page. It doesn't explain what makes /dev/random better than /dev/urandom. I'm not trying to be obtuse: I'm telling you, the Linux kernel maintainer and architect for random/urandom also believes that urandom is inappropriate for long-term keys on Linux, and he is also wrong. So that particular argument from authority is not compelling to me.

From what I can tell, the situation on Solaris is very similar to that on Linux:

* there are two separate pools that service random and urandom,

* but both use very similar generators (in particular, both use CSPRNG DRBG constructions);

* some additional care is taken in random's case to ensure initialization (a good thing)

* but that care doesn't matter once this system is fully booted,

* and random is more aggressive about reseeding,

* but that only matters for post-compromise forward-secrecy, the need for which in this case implies you are completely boned anyways.

I'm standing by what I said before: use urandom, to the exclusion of all other generators, very much including Solaris.

both use very similar generators (in particular, both use CSPRNG DRBG constructions)

Except /dev/random, which can use a different generator when the administrator configures a specific validation mode. Again, specific cryptographic requirements.

that only matters for post-compromise forward-secrecy

Not all cryptographic requirements are purely technical or even "ideal"; some reflect specific policy and/or customer requirements based on their specific cryptographic needs.

I'm standing by what I said before: use urandom, to the exclusion of all other generators, very much including Solaris.

To you, this advice is based on "authority"; to me, it's based on my long-term friendship and working relationship I've established with them over the years and my respect for them as engineers/developers/crypto experts. These are people that would run circles around most "geeks" and have been doing crypto for a decade or more and they have both the pedigree and track record to show for it.

So in the end, you're of course free to believe what you'd like, and while I agree that your advice is generally reasonable, I continue to assert that it is wrong in specific cases for Solaris. As I already pointed out, and you seem to have ignored, the results you get from /dev/urandom are not the same as /dev/random in certain configurations.

As I said before, the Solaris crypto team intends to update the blog post I linked before with more details at a later date. Perhaps that will have the elusive information you're looking for.

I'm suggesting that there are no such cryptographic requirements. I'm not alone in making that suggestion; you can, for instance, look up Thomas Pornin's comments on Crypto Stack Overflow for a similar discussion.

I'm sticking with this argument because it is a common one. There's a widespread belief that there are low-quality and high-quality random numbers (or, if you must, "random numbers suitable for one kind of cryptographic application" and "those suitable for another"). I'm pretty sure this is an urban myth.

To me, in this case, the smoking gun is the Solaris team blog post that suggests urandom is appropriate for ephemeral and short-term secrets and nonces, and than random is appropriate for long-term secrets.

Unless they're trying to communicate that random is worse than urandom, and so it's safer to use it in offline scenarios, but not in demanding online scenarios, they have the threat model exactly backwards.

I'm suggesting that there are no such cryptographic requirements.

There are indeed such requirements; look up the requirements for FIPS validation. As I mentioned before, some requirements may not be purely technical in nature. You're free to feel they're not necessary, but some organisations clearly do.

Unless they're trying to communicate that random is worse than urandom, and so it's safer to use it in offline scenarios, but not in demanding online scenarios, they have the threat model exactly backwards.

They're not; I think it's just how you're personally interpreting the text. We're just going to have to agree to disagree.

I've spent 20 minutes reading this thread. You have never actually said anything other than, "I have friends who tell me this is the way it is so I trust them." and "there are Solaris specific requirements that make Solaris different."

tptacek has responded with specificity, actually detailing how the Solaris team is, in fact, wrong in their guidance.

From my perspective we have a two-way conversation in which one person is arguing about what their friends say, and the other is trying to talk detailed cryptographic requirements.

It might help if your Solaris friends could identify what those FIPs requirements are, and how /dev/random fulfills them, but /dev/urandom does not. That would be interesting.

After all - the Linux team, who presumably also consists of pretty smart people, also believed they were right regarding /dev/random versus /dev/urandom, and it turns out it wasn't the case with them either.

tptacek has responded with specificity, actually detailing how the Solaris team is, in fact, wrong in their guidance.

No, tptacek has responded with a view that suggests beliefs about how they are wrong in their guidance, but has done so without 1) access to the actual implementation 2) the decades of experience of implementing and architecting it.

tptacek is certainly free to express opinions; but respectfully, I will trust the individuals that have implemented, architected, who are considered experts in their field, and who have maintained it for the last decade or longer over someone who has not.

After all - the Linux team, who presumably also consists of pretty smart people, also believed they were right regarding /dev/random versus /dev/urandom, and it turns out it wasn't the case with them either.

I pointed out very specifically why my assertions about Solaris are correct. When you configure the system in a specific way, the implementation for urandom vs. random can produce different results. There are also other subtle differences in the implementations.

Any further details that can be shared will likely be placed in that blog post I linked to when it is updated for a forthcoming Solaris release.

"However, on systems such as Solaris, the cryptography group (whom I know personally) specifically recommends the use of /dev/random in some cases. Those cases are largely limited to specific cryptography requirements such as a hard requirement for high entropy sources."

I'd also like to know what their specific arguments are on Solaris and for what use cases.

Linux is one of the cases where the difference matters a lot, because /dev/random is blocking. I think it's BSD where the two devices are the same.

The difference matters, but on Linux, the difference means that you need to avoid /dev/random, because the blocking behavior virtually never helps security but always reduces reliability.

"because the blocking behavior virtually never helps security but always reduces reliability."

I like how you put that. Best way to because infrastructure reliability is way too important to sacrifice over questionable security gains. Puts a QED on the whole discussion.

Of course you probably already know this, but I think the root problem that OP is driving at here is that if you use /dev/urandom then you risk getting predictable values from /dev/urandom at startup[1], e.g. when initializing your server's SSH keys (or whatever). I seem to recall this being the root cause of thousands upon thousands of home routers being having weak keys. As such, it's important to point out.

[1] That is, before enough external entropy has been gathered.

I think the new `getrandom` interface fixes that, by blocking until /dev/urandom is initialized.

Yes. Yes, it does... but it's not transparent to userspace, which I think is what tptacek was alluding to. (And, frankly, is how it should be. However, the Linux kernel maintainers are absolutely fanatical about backward compatibility for userspace, so here we are with a new syscall.)

If your cryptography requirements require high entropy for the random numbers generated, then blocking is the right answer.

In practice, that's rare, but some consumers have strict, hard requirements that can't be ignored.

Once your CSPRNG has enough entropy in its internal state it has got this entropy.

Retrieving output bytes doesn't change anything.

The view "take three bytes output, lose three bytes entropy" is simply not correct.

That doesn't match the repeated assertions of many posters here. I can only speak for Solaris knowledge-wise.

Here is a rough diagram which demonstrates it http://www.2uo.de/myths-about-urandom/structure-yes.png (from the detailed page http://www.2uo.de/myths-about-urandom/). /dev/random will block when the entropy estimate in low, but in a practical sense, the output of /dev/urandom and /dev/random are both random.

"That doesn't match the repeated assertions of many posters here" - I'm curious, what doesn't match?

Better to block than spit out low entropy data.

That's true the way you say it, and that's definitely the problem that `getrandom()` solves by blocking if the random pool isn't initialized yet. But a lot of people take that to mean "you should never get more than N random bytes from a pool that has N bytes of entropy", and that part is wrong.

All of crypto relies on being able to generate an arbitrary amount of good random bytes from a single 256-or-whatever-byte seed. Otherwise it wouldn't be safe to encrypt a long message with a short key.

libgcrypt (GPG) does the same thing OpenSSL did. It uses /dev/random to seed an internal PRNG. But you can switch to using just the system PRNG:

    gcry_control(GCRYCTL_SET_PREFERRED_RNG_TYPE, GCRY_RNG_TYPE_SYSTEM);

I don't know if they would be convinced to use /dev/urandom though. They already rank /dev/random as having lower "security" than their own PRNG.

It comes from all the posts by smart people and even the man page saying to use /dev/random if security matters. Without digging into the two's code, I also took their word for it the few times I used it in my apps many years ago. Mostly just used CRNG's while dealing with entropy myself, though, so it was limited to a few designs & countering blocking. (Wasted effort in retrospect...) It took quite a while before someone said, "Hey, they go through the same CRNG." I'm like, WTF? Why have so many sources that should be knowledgeable about this been giving bad advice?

Anyway, a number of write-ups (esp Huhn's) and people like Ptacek have been clearing up the issue well. Meme should weaken over time. Might help to straight-up delete the bad claims from whatever sources people are seeing them, though, where possible. Man page comes to mind if it hasn't been updated.

Thomas Pornin is relentlessly hunting down that meme on stackexchange.

I guess Wikipedia would be the highest-profile source for most people.

The English-speaking one mirrors the Linux man page, the German-speaking Wikipedia goes further and outright advises against /dev/urandom for "high requirements". This used to be even worse, though.

Yeah, the Wikipedia articles are a problem. I'll see what I can do when back in town on my main system.

The problem is immediately after boot before anything has seeded the entropy pool.

Like say when sshd generates host keys!

> The problem is immediately after boot before anything has seeded the entropy pool.

With getrandom() on Linux it will block if you don't set GRND_NONBLOCK and the pool isn't initialized - http://man7.org/linux/man-pages/man2/getrandom.2.html - the desired behaviour.

Which is why the getrandom() syscall is preferred; it draws from the urandom pool, but will block until a quantity of initial entropy has been fed into the pool (ie, at early boot).

Is there any actual reason why /dev/random doesn't do that?

Stubbornness of Linux kernel maintainers. My understanding that your proposed behavior is the behavior on FreeBSD and OS X, and people have proposed bringing Linux in line.

getrandom() has a mode where it acts fully like /dev/random - and it's for some (misguided but very important to some big businesses) security certifications

I'm not sure why this is supposed to be a tricky problem. Seed urandom from /dev/random, just for the side-effect of blocking, before you generate host keys. Never use /dev/random again.

AFAICT to do that reliably for all applications using /dev/urandom, you'd have to insert a step blocking all applications at startup until /dev/urandom was seeded. Even applications that have no need for anything from /dev/{u,}random.

(EDIT: I suppose one might try to replace /dev/urandom with some pipe-like thing running in userspace, but that seems error prone and rather contrary to /dev just being "devices".)

[1] Without just doing it at the kernel level, which the Linux kernel developers seemingly still stubbornly refuse to do.

But you keep explicitly telling people not to do this?

I understand what you are saying - so I hope nobody downvotes you. I've spent hours reading the HN, and Stack Exchange stuff on this - and I think the conclusion is, "/dev/random is really bad, because it blocks, breaking all sorts of programs, and 99.99% of people who use it instead of /dev/urandom aren't getting anything that they couldn't get from /dev/urandom, except the blocking behavior. The one possible exception is on a system in which /dev/urandom hasn't yet been seeded, but there are many ways to fix that, the most straighforward of which is to seed /dev/urandom from /dev/random, and then never look at /dev/random again. Or, better yet, use http://man7.org/linux/man-pages/man2/getrandom.2.html which provides a guarantee of entropy, as well as non-blocking nature once it's been properly seeded. The only time it would ever block would be on boot-up prior to it collecting the very few (approx 256?) bits required to generate effectively endless bytes."

Geht yourself a real OS. In OpenBSD the bootloader takes care oft this. Thats good enough to geht random data right wenn the kernel starts, and by the time init runs it's gathered way more entropie.

Umm, is there something wrong with your keyboard?