Comment by haberman | Hacker News Reader

Comment by haberman | Hacker News Reader

Parent Full thread

Hi Chandler,

I should have clarified that the comments I was referring to were on this bug: https://llvm.org/bugs/show_bug.cgi?id=11763

I was surprised to see: "I don't think it's really worth warning about, considering I don't know of any real-world implementation where it doesn't work."

That sounds a lot like someone knowingly writing UB in a C program because no compiler they know of breaks the intended behavior. I doubt any memcpy() has ever guaranteed in any kind of official way that overlapping regions are ok, even if it has historically worked in practice.

I mean I'm sure it'll get fixed now, I was just surprised to see this kind of reasoning coming from someone who I presumes works on the compiler.

btw. I'm pretty sure I heard that glibc broke this behavior sometime recently. There's a new memcpy() symbol in glibc these days with a different symbol version that no longer provides correct behavior in the case of overlapping memory regions. This is all second-hand from some other people I was working with, but if I got their meaning correctly I think this issue will actually have relatively widespread impact.

That comment was actually a considered comment: when we look at what warnings we should produce, we consider cases where everyone has written code a particular way and the fact that it isn't defined is actually a language bug.

We (the Clang community members, and I suspect the GCC community members as well) have also worked to change language standards to provide guarantees relied on consistently and where the lack of guarantee has no reasonable basis. Also that comment is very old, and predates us learning about all of why this is probably something actually worth warning on. That discussion ended up in a different forum, and we do actually warn on stuff like self assignment.

Understand that it is often precisely the people working on a compiler that do deal with the cases of when we discover widespread UB in real code and whether or not the code is incorrect and must be changed or the standard is incorrect and must be fixed.

But all of that is about a warning, and not about the actual question of how the compiler and the system library interact.

Note that both Clang and GCC have at least some variants of this bug, and so I don't actually expect glibc to break this in the foreseeable future. But we're still actively working to defend against this on platforms for which we don't have a guarantee.

> I don't actually expect glibc to break this in the foreseeable future

Would you have said this if Ulrich Drepper was still in charge of glibc? Under his ownership glibc broke all sorts of things that happened to rely on memcpy behaving like memmove[1]. I'm not sure he would have been any more accommodating of non-conforming compilers than of non-conforming flash implementations.

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=12518

I can't see an argument here for why the standard should be changed. memcpy() does one thing and memmove() does another. Changing the standard so they are the same just because memcpy() gets misused in real-world programs doesn't make sense to me. If people misuse memcpy(), then have a compiler flag akin to -fstrict-aliasing where memcpy() gets rewritten to memmove() unless people compile with -fstrict-memcpy.

That seems a separate issue though from what calls the compiler should emit. I am not following why LLVM would emit memcpy() when the semantics it actually requires from libc are memmove(). Am I missing something?

It is not about making memcpy behave the same as memmove: in this case the difference would still be between exact identity (src == dest) and partial overlap.

I see, thanks for clarifying. I agree there could be reason to consider a change in that case.

LLVM is a very impressive compiler. I'm really impressed with the warning messages it's able to produce (especially compared with what was available even 2 or 3 years ago).

What is more frustrating about modern compilers (and this is not limited to LLVM) is the desire to squeeze out a tiny bit more performance by exploiting undefined behavior. I'm thinking of things like eliminating calls to memset that "don't seem to do anything user-visible," eliminating code based on signed overflow being undefined, and so on.

Most of the the undefined behavior that exists in C is there because different machines used to do different things. For example, there used to be non-two's-complement machines. It wasn't put there because C was somehow designed to be hyper-optimized. People who relied on the undefined behavior could legitimately make the same argument that you are now making-- that it worked fine on their machine, which they knew very well.

I am curious why nobody has brought up the idea of making memmove as efficiently handled as memset, rather than using undefined behavior here. UB feels like the wrong way to go.

You may be thinking of "implementation-defined behavior" which is not exactly the same as "undefined behavior".

>btw. I'm pretty sure I heard that glibc broke this behavior sometime recently.

You are perhaps thinking of this: https://sourceware.org/bugzilla/show_bug.cgi?id=12518

I'm not sure if that change broke glibc memcpy for identical arguments or just ones that were different but overlapped.

This quote suggests that Drepper wouldn't have had any problem breaking things that passed identical arguments to memcpy:

>The existence of code written by people who should never have been allowed to touch a keyboard cannot be allowed to prevent a correct implementation.

Reply on news.ycombinator.com