Hacker News Reader

Comment by protoman3000 | Hacker News Reader

How can you ban memcpy effectively if it’s just a simple for loop everybody can implement in less than 20 seconds?

AFAIK, memcpy is not a simple loop, in a way that it can align the copies to match the architecture. For example, with the right conditions (aligned memory), it can copy 32 or 64 bits at a time for most of the loop, and the remaining through copies of 8 bits (not specifically in that order).

You don't replace memcpy unless you do it right. The difference between systems with a bad or good memcpy is abysmal.

Fun story. I once had a program hang (running on EFI) on a fairly benal line of C code. Turns out Clang injects memcpy into your code even if you don't call it. I hadn't defined memcpy anywhere so it just hung.

This is documented[0]:

> Note that it is assumed that a freestanding environment will additionally provide memcpy, memmove, memset, and memcmp implementations, as these are needed for efficient codegen for many programs.

GCC has effectively the same stipulation[1]:

> GCC requires the freestanding environment provide memcpy, memmove, memset and memcmp.

The main thing I'm confused about is why you didn't get a linker error.

0. https://clang.llvm.org/docs/CommandGuide/clang.html#cmdoptio...

1. https://gcc.gnu.org/onlinedocs/gcc/Standards.html

> The main thing I'm confused about is why you didn't get a linker error.

EFI isn't your usual Freestanding env. Your output is a dynamically linked PE2 (Windows) executable. The linker had spots for memcpy and others to be linked in Dynamically. But they were not. Whatever was there caused a hang in TianoCore UEFI. (Which would eventually reset due to the watchdog timer)

> Turns out Clang injects memcpy into your code even if you don't call it.

so does gcc, it's pretty much a requirement of the C standard. https://gcc.godbolt.org/z/aaE5hn

> it's pretty much a requirement of the C standard

Why would the C standard mandate the presence of standard library functions in a freestanding environment? I always assumed compilers emitting calls to mem* functions were doing it because it was the easy solution just like linking to libgcc.

memmove() isn't on the list, so presumably the recommendation is to replace uses of memcpy() with memmove() due to its well-defined behaviour when source and destination overlap.

(Or with the Annex K variants -- memcpy_s(), memmove_s() -- if that floats your boat, and as this is Microsoft it surely does.)

Well, of course you can't stop out of bounds array access in C. But at least you can cover the major array operations, such as string and memory copy.

Presumably anyone sneaking in a memcpy in the backdoor would also be thwarted on code review.

On code review and/or by a linting tool. Microsoft seems to be using lots of static analysis, and recognizing a memcpy loop is pretty simple.

This is a fair point. memcpy will just be replaced by manual loops or std::copy_n or whatever. Don't know if that's better or worse.

No it will be replaced by memcpy_s which is just generally a more sane version.

Well memcpy_s is a Microsoft initiative and so Unix devs will reject it on principle.

Annex K never really got buy-in from the open source C stdlib developers—Glibc, Musl, etc.

I don’t think it’s an issue of whether it was rejected on principle, or because of its origin, I think that there just wasn’t enough interest in it. My feeling is that Annex K is not as useful without the right set of code review practices and static analysis tools. Many of the secure variants just take an extra size parameter—you would need some kind of assurance that the extra size parameter is somehow correct; without this assurance, the Annex K functions aren’t very useful. There’s not a great way to get that assurance without static analysis tools, as far as I know, because there are tons of unsafe ways to use the Annex K functions, like this:

    // Don’t do this.
    err = memcpy_s(dest, n, src, n);

If you’re just duplicating the “count” argument and pasting it in “destsz”, it will work but it won’t catch any of the errors that memcpy_s is designed to catch.

Yes, annex k was generally judged to not be very helpful in practice[0]. However, I agree that it might make more sense as a component of an environment that mandates strict static analysis and code review. (Microsoft also has other extensions with similar roles, e.g. their in and out parameters.) In that context, it makes perfect sense: microsoft wanted to upstream some components of their static analysis pipeline, but those components ended up being useless without the rest of the pipeline.

0. http://www.open-std.org/jtc1/sc22/wg14/www/docs/n1967.htm

I think the only real solution is operations that can't fail, like C++ string operations. Of course even those can fail, but do so with an exception, which IMO is easier to deal with from a security standpoint by aborting the process, which is the default behavior. And an exception is only going to occur under pathological conditions (typically out-of-memory) which really wouldn't be solvable with any practical solution.

> operations that can't fail [...] can fail, but do so with an exception, which IMO is easier to deal with from a security standpoint by aborting the process

That's exactly what annex k does. Detection of a runtime-constraint violation results in a call to a constraint handler; which handler can abort the process if you want it to.

Reply on news.ycombinator.com