Hacker News Reader

Comment by mnarayan01 | Hacker News Reader

From the diff:

  -	newkey = xmalloc(sizeof(*newkey));
  +	newkey = xcalloc(1, sizeof(*newkey));

The only change is that the allocated memory is zeroed, right? (Just wondering if I'm missing something.)

View on HN

Nope. That's all.

The newkey struct in turn contains a bunch of OpenSSL context structures, and one of these includes a cleanup callback pointer for the MAC (message authentication code) in use. For most ciphers this was being initialised later but AES-GCM provides message integrity without the need for an external MAC and in this case the MAC context was being left uninitialised.

When the newkeys struct was later being cleaned up as part of a rekeying operation, the cleanup callback was called. It's address was whatever happened to be on the heap when this allocation occurred.

It certainly appears exploitable using standard techniques, though it may be complicated somewhat by OpenSSH's privilege separation architecture.

Thanks.

Is calloc(1, size) considered idiomatic for zeroing memory? While I haven't used C much lately, back when I did I don't think I realized this, and could see myself at least considering reverting the change during "code cleanup".

Yeah, it's fairly common. calloc zeroes, malloc doesn't, and you shouldn't make a significant change like that during code cleanup. I think it's somewhat more common to see malloc followed by memset, as the intent is way more clear to a reader than just calloc by itself.

calloc can be faster because it has internal knowledge of whether the memory is already zero.

Hmm.. would it make sense to replace all calls of xmalloc with xcalloc as a preventive measure?

I don't think it is advisable to blindly replace like that in a widely used, audited and tested system. You need to think about more than just the bits in memory, for example timing attacks. And who knows, your "fix" could even expose some other latent bug. It's not clear cut.

I don't get this. How could leaving memory uninitialized be better than zeroing it? Using calloc could be redundant in places, in that the allocated object might get initialized some other way right after allocation. But I don't see how it could ever be wrong. (There might have been zeros in that memory already anyway.) There are languages that have no way to allocate uninitialized memory -- are you going to say they aren't appropriate for crypto algorithms?

And besides, exposing a latent bug is a good thing, no?

There are a couple of concrete worries I'd have:

* If you malloc something very large, on Linux at least (I don't know about the BSDs), it doesn't necessarily get allocated, and only requests memory from the system when you actually access it. It's a valid pattern to, say, malloc a large amount of memory, pass it to your own allocator as a pool, and use what you need from it. If you go and zero all that memory, then you've accessed it all, resulting in much increased memory usage and maybe OOM kills. (You can argue this is an issue with the Linux overcommit system, not with malloc, but that's the system we have today and we'd need to account for it.)

* Slow code is a deterrent for people deploying crypto. If it takes 5 milliseconds for an HTTP response and 50 for an HTTPS handshake, people are going to be hesitant about deploying HTTPS, or do something silly like use it to protect only the initial login. So you do need to worry about the time that your crypto takes as an operational concern for deployment, and balance that off against hardening and paranoia. Favoring either extreme too much results in less overall security.

* There are very few languages that are truly "appropriate" for crypto algorithms. Even C only kind of counts, and you have to be very careful about how you write it. You ideally need promises from the language that certain operations are constant-time and that the optimizer is going to do what you want and neither more nor less. Most of the languages I can think of that only allocate you zeroed memory also include risks on the order of garbage collection being triggered in the middle of your algorithm depending on what your secret data is, so that makes them definitely unsuitable. And most of the non-toy crypto libraries in higher-level languages end up being bindings of carefully-tuned C libraries, and in that sense _do_ quasi-unintentionally expose a way to allocate uninitialized memory, by going through the C allocation routines.

I think that if I were a maintainer of something like OpenSSL, I would want the rule to be that any use of plain malloc, as opposed to calloc, should be commented. A simple "malloc ok" would suffice if the reason was obvious; for example, the very next lines of code initialized the object to something other than zero. If the reason were more subtle, a longer explanation should be provided. I do agree that performance concerns can be relevant (though zeroing a small block of memory that you're about to write into anyway is very cheap).

The caveat I would want everyone to be aware of is that if the object is a struct that is initialized member-wise, malloc can still be dangerous, because if a member is subsequently added to the struct, one would have to manually review all points where the struct type is allocated to add the initialization for the new member. So I would approve of malloc in such a case only if there were only one place in the program that allocated that struct type.

I'm talking about the crypto code itself. If you've written your own allocator, I agree that different rules apply for the allocator code.

"Most of the languages I can think of that only allocate you zeroed memory also include risks on the order of garbage collection being triggered in the middle of your algorithm depending on what your secret data is, so that makes them definitely unsuitable. "

You do realize that even a C program running on a modern kernel can be preempted? Unless you are running it as a high-priority real-time process on a real-time hardened kernel, and, OpenSSL is giving you hard-real-time guarantees.

But, task preemption is most likely not correlated with the contents of secret data. It's pauses that correlate with different values in secret data that allow data to leak via timing attacks.

Yes, and there are certainly cache-timing attacks between processes (and even VMs) on the same physical hardware. That said, if you're disciplined enough to avoid all its awfulness, C is one of the least-bad languages for writing crypto; other things that compile directly to native code and offer C-spirited APIs for allocation and the like might be better. Having no allocation API that doesn't zero returned buffers is somewhat correlated with being high-enough level that it's a more-bad language to write crypto in.

In other words, there may not be a good language, but some languages (or runtimes, really) are certainly worse than others.

Remember when Debian's OpenSSL was reduced to issuing a predictable small number of keys instead of using the entire keyspace? It's because someone added a patch that in effect initialized memory that was supposed to be uninitialized.

I had forgotten, but I just read up on it [0]. I haven't actually studied the code, but it appears to me from this post that your characterization is not quite correct. They didn't just add code to initialize the memory; according to [0], it would actually have been correct to do so (see the second sentence under "Links and Notes"). Instead they did something slightly different from that, which was wrong.

[0] http://research.swtch.com/openssl

That's only half correct. The problem is that they reinitialized memory.

Yes. I believe the bug rises from some code like:

    if (newkey->callback != NULL) {
        newkey->callback()
    }

If newkey structure was not zeroed, then the callback would have had some contents from previous use and the NULL check would have passed and some unintended memory position would be called.

Why didn't that crash? I understand the cleanup routine is always called and certainly someone tested the AES-GCM provider by just running it once.

Probably because the memory allocated just happened to contain only NUL bytes where that specific cleanup function pointer is stored whenever it was run in testing.

The point is that a targeted malicious attack would try and ensure that those bytes were not NUL and that it ended up calling malicious injected code on the target server.

Yeah, I've found a few bugs like this myself where the memory in question happened to be NULL, until some unrelated change elsewhere in the toolchain meant that it wasn't anymore.

The C standard doesn't require such code to be correct, even with calloc, because the NULL pointer is not necessarily represented by the zero bitpattern. While it will work "everywhere", calloc + NULL check is not semantically correct C.

The C standard defines NULL as 0:

"An integer constant expression with the value 0, or such an expression cast to type void * , is called a null pointer constant. If a null pointer constant is converted to a pointer type, the resulting pointer, called a null pointer, is guaranteed to compare unequal to a pointer to any object or function."

(Edit: the asterisk.)

Yes, but that's not what calloc does. It doesn't return memory with all values 0 (how can it? It doesn't know what structure you're allocating.), it returns all bits zero.

While I really appreciate learning from a good language-lawyering, and also acknowledge that there are many places where subtle interpretation issues have led to crashing code or even exploitable malfunctions, in practice the issue about different representations of Zero or the NULL-pointer are indeed mood.

I'll just be flamboyant now and claim that on all currently relevant platforms calloc'ing the memory of all supported numerical and pointer types will give you the numerical value 0 and also the NULL pointer.

Can someone name any CPU or microcontroller currently being produced that violates that statement? (e.g. uint64_t x; calloc(1,sizeof(x)) and void *p; calloc(1, sizeof(x)) or maybe memset(x,'0',sizeof(x))... and still x != 0 and p != NULL)? Even for gross missalignment? What about DSP platforms that don't have a useable byte-addressible memory? GPUs?

I agree. Note the fix used even depends on calloc setting a null pointer. The openbsd team generally makes a few other assumptions as well. But the comment I replied to was claiming the c standard itself makes these guarantees. That's wrong, and I think, slightly dangerous. It's fine to make assumptions, but one must always know what they are and not confuse them with guarantees.

Note that your memset call is totally bogus. '0' is not 0. You may have meant '\0', but plain 0 is perfectly legible. :)

I confirm that my inline C-code was written pretty sloppily... But obviously you clearly understood my intention :-)

> or the NULL-pointer are indeed mood

Just FYI, the word you probably mean is 'moot'.

It seems odd to see a commit in this day and age with no unit test. When you make a new newkey struct the clean-up callback should be NULL.

That's the fun with C, it would be quite unlikely the malloc call in a unit test for this didn't happen to zero initialize the struct.

Yep. You may get different results based on the code path taken, the compiler, and (maybe?) the OS. Most of the time the value at that memory location will probably be a zero byte (or a zero word/dword), but sometimes it may not.

We committed a regression test fix separately.

Reply on news.ycombinator.com