Arc4random – randomization for all occasions | Hacker News Reader

Arc4random – randomization for all occasions

openbsd.org

80 pointsedwintorok30 comments

Open articleView on HN

Link to Hackfest 2014: Theo de Raadt presented "arc4random - randomization for all occasions" video

https://www.youtube.com/watch?v=aWmLWx8ut20

via http://www.bsdnow.tv/episodes/ a great podcast with ep 64 talking about the presentation and whole lot more.

The two (for me) important slides:

http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000...

    rc4random cannot fail

    Look at the function definitions:

         uint32_t
         arc4random(void);

         void
         arc4random_buf(void *buf, size_t nbytes);

         uint32_t
         arc4random_uniform(uint32_t upper_bound);

    All 3 functions guarantee a result.  They will not fail, and cannot
    return an error.

http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000...

    Linux?

    We have encouraged Ted Ts'o to create a getentropy-like interface
    for Linux; getrandom()

    Too many options, too easy to misuse

    Interactions with signal handlers (EINTR)

    Hope applications do not directly call getrandom()...


    Bit worrying

    Really feels like there is pressure against easy-access random..

    "Use /dev/random" meme continues damaging effects

Linux's getrandom really has a bunch of error codes:

http://lwn.net/Articles/606202/

    EINVAL	An invalid flag was passed to getrandom(2)
    EFAULT	buf is outside the accessible address space.
    EAGAIN	The requested entropy was not available, and the
            getentropy(2) would have blocked if GRND_BLOCK flag
            was set.
    EINTR	While blocked waiting for entropy, the call was
            interrupted by a signal handler; see the description
            of how interrupted read(2) calls on "slow" devices
            are handled with and without the SA_RESTART flag
            in the signal(7) man page.

And a few flags:

   GRND_NONBLOCK	Don't block and return EAGAIN instead
   GRND_RANDOM		Use the /dev/random pool instead of /dev/urandom

I agree that the best interface is the one that can't fail.

getrandom() is a generic interface you can use to implement getentropy(), which is used to seed arc4random(). It is not an alternative arc4random implementation...

From the OpenBSD Manpage:

    getentropy() will succeed unless:

    [EFAULT]
    The buf parameter points to an invalid address.
    
    [EIO]
    Too many bytes requested, or some other fatal error occurred.

See: http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man2/...

Yes, getrandom is not an alternative arc4random implementation, it's a lower level interface. Even the programmers writing for Linux should actually locate arc4random implementation that they can use. And before somebody even considers calling getrandom directly he should evaluate how hard is to just implement getentropy from getrandom, it's harder than it looks at the start, if I correctly remember even Ted Tso's first attempt was wrong -- here's the better (I don't know if it's the last):

http://lwn.net/Articles/606552/

    int getentropy(void *buf, size_t buflen)
    {
            int     ret;

            if (buflen > 256)
                    goto failure;
            ret = getrandom(buf, buflen, 0);
            if (ret < 0)
                    return ret;
            if (ret == buflen)
                    return 0;
    failure:
            errno = EIO;
            return -1;
    }

Note the magical 256 number. It's important.

> EFAULT buf is outside the accessible address space.

I wonder how arc4random_buf(NULL, 42) behaves. Or what happens if you pass it a pointer to an invalid memory location. Yeah, can't fail, right...

Once we commit ourselves to write some C code some kinds of failures are expected and the ones you point to are of that kind. The topic here are actually the ones not expected by a C programmer who wishes to use the API function. I don't want this discussion to digress to talking about some other languages even if we all know they exist and can avoid the kind of errors you describe. That's still off topic.

SIGSEGV

Not a "failure" of the libc call per se and it terminates the caller app which, frankly, is the only safe thing to do.

The Linux equivalent, while it can fail could be wrapped in a libc version that emulates arc4random behaviour (provide random or kill the app really.)

arc4random has a larger range as well up to 0x100000000, whereas rand/random top out at RAND_MAX of 0x7fffffff?

All occasions, even shell scripts.

For example to choose a pseudorandom entry from a (reasonably short) list and set it as an environmental variable:

echo "main(){ arc4random();}" |gcc -x c -pipe -o arc4random -static -

b1=$(sed -n '$!D;=' .list); c1=$((b1/255)); arc4random; d1=$(($?*c1)); VARIABLE=$( sed ''"$d1"'!d;'"$d1"'q' .list ); echo $VARIABLE

It's simpler to dd from /dev/[u]random however and in OpenBSD, you'll get exactly the same quality result. Most of the benefits of arc4random() are nullified by invoking gcc...

On OpenBSD, it's even simpler to just echo $RANDOM, but that only gets you 15 bits because sh. sysctl -n kern.arandom can be used in a pinch, too.

Theoretically couldn't you do something like

  echo $(( $RANDOM << 15 | $RANDOM ))

to get 30 bits?

OK, I feel stupid. How would I get a random number using dd and /dev/urandom?

I am not sure I understand the gcc comment. Can you explain?

Assuming I cannot get a random number from dd'ing /dev/urandom then I needed a utility to do it where the system only has a small base and does not have gcc.

Unangst's suggestion to use sysctl is better (although it means I have to crunch the sysctl utility). I will be using that from now on. Thank you for the tip!

As for $RANDOM, that may be OpenBSD-specific. For example, does NetBSD have it?

This dd(1) command line would get 1 block of 4 bytes from the /dev/random file and output it to stdout. Effectively the same as a call to arc4random():

dd if=/dev/random count=1 bs=4

On Linux, you'd use /dev/urandom cause /dev/random blocks stupidly.

Agreed. But I want a pseudorandom real _number_ in decimal, to use in a shell script, not just random bytes.

I gave your solution a try.

  a=$(dd if=/dev/urandom bs=4 count=1|od -An -tx1|sed 's/ //g')
  printf %d\\n  0x$a

$RANDOM is shell-specific. bash has $RANDOM, ksh has $RANDOM, and zsh has $RANDOM, standard sh doesn't have it.

Ah, I had forgotten where this comes from. This makes sense; OpenBSD's default shell is pdksh or some derivative thereof, if I recall correctly.

Shouldn't glibc provide an arc4random? Apparently the new syscall is not for direct use [1], and you might not want to link with LibreSSL just to get arc4random [2]

[1] http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000... [2] http://www.openbsd.org/papers/hackfest2014-arc4random/mgp000...

There is a proposal to make a Posix interface that is essentially similar.[1]

[1] http://austingroupbugs.net/view.php?id=859

And now I have more work to do... :)

So is this going in? Are you using arc4random()?

That's certainly possible. I don't control what goes where, I only made the proposal.

The chances of glibc importing openbsd derived functions are practically nil unfortunately, as we saw with the strlcat saga. With the departure of Drepper, things might have changed, but strlcat is still not imported.

Things have changed enormously since Drepper left. It is a nice project to work with and many things that were vetoed are getting in.

I don't think that's true. Drepper was against strlcat for specific reasons. And in the meanwhile the C11 standard had adopted an alternative (but imo worse) API for it, based on the Microsoft stuff.

Framing this as a general rejection of APIs because they are from OpenBSD is highly disingenuous.

What is the reason for that? :(

Drepper's caustic style aside, his point ultimately was that truncation is still a problem. Whether or not you think glibc should include strlcat, this is an important point to understand.

More discussion here: http://stackoverflow.com/q/2114896/67591

Note that you can mechanically translate strcat(dst, src) to strlcat(dst, src, SIZE_MAX) with no loss of functionality or change in behavior. No worries about truncation, either. Exactly the same thing.

Now, if reading a piece of code that does strlcat(dst, src, SIZE_MAX) would make you feel uncomfortable, the question is how should you feel reading code that uses strcat?

Not specified in any standard, not much of an improvement according to the glibc maintainers, and already provided in libbsd.

Speaking of libbsd it should be updated to use ChaCha20 too: https://bugs.freedesktop.org/show_bug.cgi?id=85827

Reply on news.ycombinator.com