How to Generate Secure Random Numbers in Various Programming Languages
paragonie.com
paragonie.com
I am not convinced that random numbers were required for the security element of the gift cards, but I don't believe there were any reports of any fraudulent activity based on unknown numbers.
Developments in this arena over the last 10 years would have made my job a lot easier!
When you say "produce CSPRNG" do you mean writing a generator or simply generating numbers? Nobody[1] should be writing a CSPRNG and app developers should be using language/OS functions (ex: /dev/urandom).
> As I couldn't be certain that any of the software routes available to me were secure I ended up purchasing this http://www.protego.se/ to generate the random numbers.
Wow did you really look at that page and say, "This is going to be easier and more secure than using /dev/urandom?". Reading the rest of the info on the page, http://www.protego.se/trnsup.htm, does not give me a lot of confidence in the product. The explanation page of how it works is just a bunch of scare info against using a software solution.
[1]: Okay obviously someone has to do it but you all know what I mean...
While the page doesn't look as cool as your average SV hardware startup, and their white paper is more or less blank, don't dismiss them that easily. According to linkedin the guy beyond the company has been involved in ASIC TRNG designs at some large companies.
Also, from what I have heard there are multiple open source hardware entropy generator designs that together with minimal software processing can generate pretty good random. Now, this is not NSA-proof randomness but probably better than anything you can get from /dev/random and more than enough for, say, gift cards.
It doesn't take much to look "professional" and having HTTPS on a page that is meant to sell a security product is table stakes.
> Now, this is not NSA-proof randomness but probably better than anything you can get from /dev/random and more than enough for, say, gift cards.
I disagree. Not being able to vet how it works or seeing the code running on the chip is not better than /dev/urandom. The latter is "more than enough, for, say, gift cards".
Heck if it's good enough for SSH and GPG keys, it better be good enough for gift cards!
Anyway, now I am tempted to buy one and reverse engineer it...
The functions available from microsoft were only PRNG and not Cryptographically Secure (as far as I can remember). Even if they were 'Cryptographically Secure' not having a background in randomness theory makes it pretty hard to work out what is actually needed.
So faced with uncertainty over using VB6 to produce random numbers and finding a product that claims to produce truly random numbers I chose to mitigate the risk by transferring it. If there had been a problem down the line related to the nature of the randomness of the numbers then I would have had recourse to the third party vendor that owned the risk.
So in the same way you suggest no-one should write a CSPRNG, I decided that given my level of knowledge, the documentation available to me and the uncertainty that created we shouldn't trust ourselves to produce random numbers for a particular purpose. What if they weren't random enough? What if someone worked out a way to predict the numbers of cards that had been loaded with value but were still sat in a warehouse? Massive risk, mitigated for £150. Now that's value.
Curious how you interact with a USB device like that in VB6 ... Does it just appear as a keyboard?
And here is the contentious Ruby issue wherein the Ruby core developers insist on cargo culting according to an outdated man page despite comprehensive refutation from a number of experienced security engineers [2].
That said, the arguments against OpenSSL seem to be largely theoretical at this point ("has been a source of vulnerabilities in Ruby ..."), so the misgivings about `SecureRandom` may be slightly exaggerated.
Speaking as a foss maintainer, to other foss maintainers out there: When you're systematically adopting a defensive attitude towards someone who believes found an important flaw in your project, you're the one being rude. You're refusing to hear people out. I see this awful attitude far too often.
On Github, it's even worse nowadays where a lot of maintainers immediately fall back to locking the bug report so that they don't have to think about it anymore. At least here there is some form of discussion going on. But that's with a lot of fingers in a lot of ears, and nobody trying to be practical...
Why doesn't this surprise me...
There is huge undiscovered land of material for cross-cultural and comparative research in communication in these developer forums. Ruby and Linux mailing lists could provide material for number of really interesting studies in many fields.
Ruby community has hint of Japanese culture: design aesthetics, simplicity, importance of politeness, saving face and rigidity.
Linux community has hint of Finnish culture: practicality, "pig energy" and "management by perkele" combined with 80's competitive hacker culture.
What is the manpage we are discussing here? I think getrandom(2) and random(4) are quite good.
/dev/urandom is the interface you want.
/dev/random is unsuitable for production code because, again, it randomly blocks, which can freeze your application. Worse still: applications that use /dev/random tend to "work around" the problem by using /dev/random to seed a userspace RNG that doesn't block. That's not just inconvenient but overtly unsafe.
This is not the behavior on FreeBSD and OS X. On those systems, once a sufficient amount of entropy is estimated to have been gathered since boot, /dev/random will not block again.
Linux's /dev/random behavior (1) relies too heavily on accurate estimates of entropy and (2) implies a simultaneous fundamental mistrust in the ability of cryptographic methods to expand several hundred bits of entropy into a few megabytes of data indistinguishable from random noise and yet simultaneous belief in the strength of cryptographic algorithms for expanding maybe a kilobit of long-term key material into terabytes of data indistinguishable from random noise. (The rationale for blocking is to produce better long-term keys.)
Linux's /dev/random rekeys its internal state at a fixed entropy estimate. If state is compromised and the estimate is too optimistic, it will never recover from state compromise. If the estimate is too pessimistic, state is left vulnerable longer than necessary.
Bruce Schneier and Niels Ferguson's Fortuna for a design is more robust and consistent. A series of entropy pools all collect entorpy at the same rate, but are emptied at exponentially longer intervals. Eventually, one of the pools used to rekey will have enough entropy to recover from state compromise, and it doesn't rely on the dubious practice of entropy estimation. Now, estimating how long recovering from state compromise will take requires estimating entropy, but with a Fortuna construction we can prove that recovery will eventually happen as long as entropy is actually being collected.
If you're using TLS with AES-GCM, a Fortuna construction built using AES has an added advantage that an attack that recovers Fortuna state is almost certainly directly applicable to your TLS setup. In other words, you're depending on fewer algorithms, the failure of any one of which dooms you.
FreeBSD uses Fortuna for /dev/[u]random. OS X uses Fortuna's predecessor Yarrow, which still has the weakness of relying on entropy estimation, but at least it has two pools (one with a pessimistic rekey rate) instead of Linux's single rekey rate.
I think it would be a bad idea to forklift out the LRNG in favor of an entirely new design.
It's possible that all Linux really needs to do is fix the man page, and perhaps do something in the kernel (rather than in OS distributions) to solve seed-at-boot.
There is no reason to block; there isn't even any good way to estimate the entropy of environmental noise.
The only sane thing to do is to seed a CSPRNG with environmental noise, reseed with environmental noise as available, and never block.
You should, however, take my word for it. Or, better yet, take Daniel Bernstein's word for it. Take Adam Langley's implied word for it (he's behind crypto/rand in Go). Take Thomas Pornin's word for. You can take Thomas Pornin's word on almost anything! Take Filippo Valsorda's word for it. Or Thomas Huhn's.
The man page is wrong. Should it be fixed? Yes, it should be fixed. But, no excuses. You can't introduce security problems out of obstinacy about man pages.
I would disagree. This is a recent paper on the state of OpenSSL's userspace PRNG, for example, finding several issues: https://eprint.iacr.org/2016/367.pdf
Therefore I was really not surprised when problems related to OpenSSL random generation started to pup up later on.
Cryptographically Secure Randomness in .NET (C#)
The generally accepted solution is to use System.Security.Cryptography.RNGCryptoServiceProvider
Cryptographically Secure Randomness in Node.js
Don't use crypto.randomBytes()
>>> int
<type 'int'>
>>> int('10')
10
>>> int = 10
>>> int('10')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'int' object is not callable- Rust (no out-of-the-box crypto lib AFAIK)
- Erlang (ships-by-default 'crypto' uses openssl)
- Java 7 and below are, unfortunately, still in common use, despite being end-of-life. There is a config file to be edited, or a switch to be passed to the java executable to switch the RNG source away from /dev/random, the pre-8 SecureRandom default.
- Some obvious note about how client-side Javascript is a lost cause, in case anybody's tempted
crypto.getRandomBytes is supported by everything that matters[0], won't be a problem in ~5 years.
When is it not?
Just by being online and running for a few seconds will cause enough entropy to put the generator into a safe state, so this rarely happens in practice.
In fact Ubuntu comes prepackaged with pollinate[0] which makes this basically a non-issue for everyone that's not wearing a tinfoil hat.
1. http://manpages.ubuntu.com/manpages/trusty/man1/pollinate.1....
There's lots of stuff happening on a network that's impossible to guess and replay. This is also called noise.
Keep in mind that the CSPRNG generator only needs a random seed, then it can produce psuedo random numbers all day long. Linux will continue to reseed as more interrupts occur.
Linux will also save a 512 byte file on shutdown and seed the generator with it on the next boot up, so you immediately enter into a safe/random state.
There's lots of stuff happening on a network that's impossible to guess and replay.
Relevant (from 2011): https://www.nccgroup.trust/us/about-us/newsroom-and-events/b...
Edit: note that I did not say that other points in that article are irrelevant.
https://gist.github.com/jamwt/60f66f0fd27d349fe460051e55e79a...
(Also submitted to blog author.)
Java 8 does define more sources for SecureRandom, including NativePRNGNonBlocking, which specifically refers to /dev/urandom (http://docs.oracle.com/javase/8/docs/technotes/guides/securi...).
(ql:quickload :secure-random)
(secure-random:number 100) => 73
(secure-random:bytes 20 secure-random:*generator*) => <a simple array with 20 random bytes>
(ql:quickload :isaac)
;; FIRST TIME (one of both)
;; using (random), which on sane implementations should rely on /dev/urandom
(defvar *isaac* (isaac:init-common-lisp-random-seed))
;; using /dev/urandom directly
(defvar *isaac* (isaac:init-kernel-seed))
;; Generate bits
(isaac:rand-bits *isaac* 30)
There are probably worse examples of userspace PRNGs that attempt to be cryptographically secure than what OpenSSL provides, but we wrote this to encourage good habits. Reliance on OpenSSL's PRNG is definitely not a good habit to get into.
The "General Requirements for Inclusion" section is our basic evaluation criteria. Cross-platform APIs that wrap the operating system's CSPRNG aren't that difficult. See https://github.com/cryptosphere/sysrandom/blob/master/ext/sy...
If OpenSSL has bad RNG, then a lot of SSL traffic is in danger...
Also, do you think Java's SecureRandom is not a user space implementation? I guess it only uses /dev/[u]random for seed.
At the beginning of the blog post there are three links.
adequately -> http://www.2uo.de/myths-about-urandom/
covered -> http://sockpuppet.org/blog/2014/02/25/safely-generate-random-numbers/
elsewhere -> https://blog.cr.yp.to/20140205-entropy.html
> Also, do you think Java's SecureRandom is not a user space implementation? I guess it only uses /dev/[u]random for seed.
In Java 8, it just reads from urandom. The relevant section has a cautionary note: https://paragonie.com/blog/2016/05/how-generate-secure-rando...
Yup. Here's an example of the latter (I don't have getrandom(2) on my machine):
(defun random-bytes (len)
(declare (type (integer 1) len))
(with-open-file (rand "/dev/urandom" :element-type 'unsigned-byte)
(let ((buf (make-array len :element-type 'unsigned-byte)))
(read-sequence buf rand)
buf)))
But they still maintain a "meaningful" distinction between random and urandom, which is the most frustrating design smell.
It's so sad.
Do other OSes ensure there's enough initial entropy to safely use urandom?
2) If you managed that and were able to conclude in the negative for a specific combination of kernel, kernel revision, and hardware models and revisions, what then? Are you going to recreate all that infrastructure in your app or through a gargantuan dependency along the lines of egd or havaged (which we've already learned from experience is a poor idea)?
My point being, the question is arguably irrelevant. If available entropy at boot time (usually first boot, as the shutdown/startup process recycles entropy pools) is insufficient, better to pressure vendors (software and hardware) to address it than to add another layer of cruft or a complex mitigation. The fix will probably be out in the next year or so (indeed, with RDRAND has been out for years); the hack will linger for far longer, perhaps eventually becoming the weakest link in the chain.
For similar reasons I think the article's as well as libsodium's choice to only use the most recent arc4random implementation as shipped in recent OpenBSD releases is misguided. Every other *BSD provides arc4random, and arguably applications should always prefer to use the libc's arc4random implementation. Eventually they'll be upgraded, at which point your application will have a useless dependency adding at best needless complexity and at worst vulnerabilities.
Once you look behind the interface of arc4random, you're obliged to look behind the interface of /dev/urandom, etc, and I doubt it will look much better. Until the recent overhaul (last month), the PRNG infrastructure in Linux relied on SHA-1, which isn't much better than RC4, especially when considering that the RC4 used in the classic arc4random implementations discarded the first 1k of output and reseeded regularly. SHA-1 hasn't been recommended for cryptographic purposes for several years, and while you could make pragmatic arguments for why it was still acceptable, you could do the same for the particular use of RC4 in older arc4random implementations. I don't see libsodium skipping /dev/urandom for kernels before 4.9. At some point you have to stop bending over backwards to support platforms with poor QoS and leave things as you found them as long as they're not utterly broken. If experience with egd and, more generally, the boots-and-suspenders approach to security is any measure, classic arc4random is well within the bounds of reasonable. Simplicity is the rule of the day; experience has taught that complexity is invariably a net loss in security, especially when the weakest link remains unchanged.
However, you can also feed the system-wide PRNG with additional entropy, and for many applications this might be the better choice.
Unless an attacker gets to acctively control your PRNG results.
For example, imagine /dev/urandom is already using mouse movements as an entropy source. If you then decide to also use mouse movements, you may have weakened the randomness of the resulting stream because now some of the random bits get reused.
In the worst case, if you were the only user of /dev/urandom and output the exact same sequence of bits from the mouse movements, an XOR would completely remove all randomness from the mouse movements (All 1's get flipped to 0's, all 0's stay the same). Even without the worst case though, it could still influence a pattern into the bits due to the reuse.
The only time you'd have to worry about this is during boot. Ideally, whatever you're trying to do at boot time should be moved to post-boot time, when the kernel has taken care of all the concerns for you.
I don't know what the answer is to "If you want to generate secure random numbers during boot before urandom is seeded, how would you do it?" I assume it's "Seed urandom, then use it."
Does anyone know how other operating systems behave in that situation?
Edit: /dev/random on OpenBSD also does not block.
That's precisely why getrandom(2) is preferred on newer kernels, and the following hacky workaround from the article is given on older kernels:
For software that runs during the Linux boot, poll /dev/random until it's available.
This means /dev/urandom has been seeded and you can safely read from /dev/urandom for
all your cryptographic purposes. Don't read from /dev/random.But it's still a great feature to have in every programming language imho.
[1] https://msdn.microsoft.com/en-us/library/windows/desktop/aa3...
Just use /dev/urandom
I can't believe how many crappy engineers I've encountered who are resistant to this advice. The onpy coherent objection I've ever heard is that "It never blocs therefore it can provide random numbers with insufficient entropy", which has been adequately refuted many times now.
But still I encounter experienced, senior guys doing all sorts of Heath-Robinson crap with reading tons of entropy from TPM chips, feeding it to openssl or other systems as a seed and using those instead. It's completely unnecessary adds complexity (which in turn decreases security) and just...
What's really frustrating is people being convinced that random numbers are hard and therefore that they really need to do all this 'hard' stuff. I guess it flatters their egos.
Didn't mean to come across as negative about it, though I can see how people might get that impression!
Linux is smart enough to squirrel away some random on reboots so that it can properly seed the generator when it starts up again.
On my system it looks like it's 512 bytes:
$ wc -c /var/lib/urandom/random-seed
512 /var/lib/urandom/random-seedIt involves opening /dev/urandom, reading 8 bytes from the device, binary scan of the returned value, converting from the hex format and scaling the result to fall within a specified min to max range.
The tcllib has other PRNG procedures, and there's the "Random" package ISAAC implementation. Whether output quality of any of these procedures is adequate probably depends on the purpose.
From the comments on the ISAAC home page, the ISAAC and ISAAC-64 versions would appear adequate for most uses, but a reader more experienced in CSPRNG analysis might have more information.
It is not necessary to round trip through a hex encoding. Tcl's binary scan is capable of converting an 8 byte value directly into a 64-bit signed or unsigned integer. You can even request big or little endian interpretation of the bytes.
Check the "w" conversion flag in the man-page. For unsigned you'd add the "u" flag. For big-endian conversion use the "W" flag instead.
This is actually incorrect. Unless the size of the range is a power of two, this gives non-uniform probability. The bias might be acceptable if it's small enough, but it's something to be aware of, and, if you want to have something robust, to avoid.
A general safe strategy is to check if the raw random value is below the maximum multiple of the size of the range that's no greater than the number of possible values for the raw random value, and if not, generate a new one and repeat. (In principle, any smaller multiple (>= 1) would work as well, but taking the maximum gives the smallest number of retries.)
It's a ~$50 USB dongle, opensource and available for shipping. Anecdotally, I personally used it for an RF project that need uniform randomness and exhausted /dev/random on the target hardware. It worked fine, if a bit slow for my very specific usecase.
That said, if you want to go even cheaper, seriously consider if you can just stick with /dev/urandom. See http://www.2uo.de/myths-about-urandom/ for an interesting read on why that might just work.
If you're really paranoid, just do this:
$ echo "<bash face against keyboard multiple times>" >> /dev/random
https://github.com/waywardgeek/infnoise
https://www.tindie.com/products/WaywardGeek/infinite-noise-t...
I remember him from the CipherShed project that was trying to reboot TrueCrypt. Discussed securing the build system & distribution. He seemed smart. I have no experience with Tindie, though, so I'd appreciate feedback from anyone that knows about them. He also includes BOM, etc on the GitHub so someone can source and build it themselves.
https://en.wikipedia.org/wiki/RdRand
The potential of an NSA backdoor in it is a popular, and entirely possible, conspiracy theory, of course. You'd have to decide for yourself, and compare it to your trust in other options.
Fixed, and a bunch of suggested additions are coming later today. :)
https://paragonie.com/blog/2015/10/coming-wordpress-4-4-cspr...
I had already suggested (to Dion Hulse) that this antipattern of pinging wordpress.org to define the salts during setup be removed in favor of always generating them locally.
I don't know if/when they'll implement that (no matter what benefits a security fix might provide, WordPress always prioritizes "no BC breaks even on 0.0001% of the installed base" first), but if you're looking for a more secure CMS I happen to be building one here:
import time, sys, random
def diorandom(m, n):
start = time.time()
for x in range(0, n):
r = m.randint(0, 2**255)
elapsed = time.time() - start
return elapsed
def diorandint(n):
start = time.time()
for x in range(0, n):
r = random.randint(0, 2**255)
elapsed = time.time() - start
return elapsed
if __name__ == '__main__':
m = random.SystemRandom()
modo1 = diorandom(m, 100000)
modo2 = diorandint(100000)
print('100,000 iterations')
print('SystemRandom().randint: {}'.format(modo1))
print('random.randint: {}'.format(modo2))
Because it's not secure. Read the big fat warning in pink: https://docs.python.org/2/library/random.html