Doh!
416 karma · joined November 27, 2012
One thing that I keep pointing out is that the Apache provides a way for the license to be self-executing. This is really cool (although also one source of objections) and IMO super important for an Open Source project like LLVM. It avoids the need for us to force every person who ever posts a patch to sign a legal agreement before we incorporate the patch because the Apache license itself handles those kinds of issues.
Unfortunately, we wouldn't get that w/ a dual-license approach. And we would add a lot of complications due to the duality. =/
I happen to know because I was one person who donated my time. I was not in any way blocked. We were trying to find a way to help more people (and yes, companies, because companies tend to pay people) contribute to LLVM because that is one of the things that makes the project stronger.
So saying that the change was driven by these handful of contributors seems really off base. And in fact, my experience during the process was that the most motivated people were individual contributors, if anything pushing and pulling companies along (by finding good ways to address their potential concerns with any change like this).
Anyways, that was my experience from being involved in the process. YMMV of course.
We did listen, but we have specific goals that after a great deal of discussion are best addressed with the approach of the Apache 2 license. The objection was to those goals in many ways, not to the particular mechanisms.
And we really did listen to the concerns about the goals (specifically providing strong protection against patent issues) but there was strong consensus in the community that this was a real problem we wanted to address, and so we moved forward.
I am truly sad that this will cause issues w/ the OpenBSD community, but we had consensus and needed to make progress.
FWIW, this is really the only strong objection we've heard to the overall direction of this change.
This also means that if we gave her really bad direction (for the community or project), she would also be fantastic at it. The fault there lies in the direction, not the legal advice.
And as the other responder mentioned, part of this has involved directly contacting all committers in the last two years (more recent committers more than once) to ensure we have everything we need to minimize disruption and make the cut over.
Further, it shows how to do it (classifier) and gives a very good model for what bandwidth can be expected in practice. And still further, that bandwidth is entirely sufficient for attacking long lived cryptographic key material. Before, it was easy to theorize that the bandwidth would simply be too low. Now it isn't.
This is also really good because the bandwidth is really low. For data larger than a key (or where you would need to scan a lot to find the data), we can be much more comfortable with infeasibility arguments by basing them on some security margin beyond these experiments. As an example, if it would take 1 <timeunit> to exfiltrate a single credit card number even if the technique got 10 times better, maybe that means it isn't a sufficient risk. Or maybe it is. (All depends on the time unit and the risk.) But now we can make that assessment with a real baseline instead of a guess.
Still, this is very different from the first-class support for compatible debug info that Clang provides.
(To the extent possible. There are some edge cases left, but they're very rare. Essentially all the real-world code we've seen works now.)
Myself and other compiler authors I work with are much more specific in their requests. We do actually define some behavior that the standard doesn't require, and we do document this and tell our users its fine to rely on it.
Unfortunately, we also have a lot of really clear experience with this stuff that lead us to two conclusions:
1) When we identify UB in code and go look at that code, for very large bodies of code what we find are overwhelmingly bugs. Bad, complete bugs. Defining the behavior wouldn't fix the bug, it was just one of the many symptoms.
2) We have real hardware differences for a lot of simple things that make them hard or impossible to give defined behavior for. As an example, shifting left by more bits than the size of the value gives different results on several different platforms.
None of this means that we don't need to help programmers cope with this. We do indeed also make mistakes, and in fact it was these compiler authors that built a tool to help programmers find UB in their code: http://clang.llvm.org/docs/UndefinedBehaviorSanitizer.html
We (the Clang community members, and I suspect the GCC community members as well) have also worked to change language standards to provide guarantees relied on consistently and where the lack of guarantee has no reasonable basis. Also that comment is very old, and predates us learning about all of why this is probably something actually worth warning on. That discussion ended up in a different forum, and we do actually warn on stuff like self assignment.
Understand that it is often precisely the people working on a compiler that do deal with the cases of when we discover widespread UB in real code and whether or not the code is incorrect and must be changed or the standard is incorrect and must be fixed.
But all of that is about a warning, and not about the actual question of how the compiler and the system library interact.
Note that both Clang and GCC have at least some variants of this bug, and so I don't actually expect glibc to break this in the foreseeable future. But we're still actively working to defend against this on platforms for which we don't have a guarantee.
But I think Clang can actually do better here. The open source project is essentially providing integration with glibc and other libcs and we don't do a reasonable job of being conservative there where we don't know what requirements that layer places upon things like memcpy.
And neither Clang nor LLVM are generating platform agnostic code. The code only has meaning for the specific platform being targeted.
The specific cases are when it has a genuine guarantee provided by that implementation vendor that it is OK. In the case discussed on this bug, at least one implementation provides that guarantee (Darwin). The problem is that not all provide it, and further than other parts of LLVM have started assuming that the memcpy more closely matches C semantics. At least something will have to change here, but the C semantics are really relevant at all, or what is "valid C code".
The folks working on other platforms seem universally unhappy with that bug.
I don't see anyone being blasé about this. The issue comes down to something really simple:
- The compiler can emit substantially simpler code if it has a guarantee from the platform's implementation of memcpy.
- The compiler authors thought they had such a guarantee and so chose to leverage it.
Now, maybe they don't have that guarantee on all platforms. If that's the case, its flat out and simply a bug in the compiler. And you'll find folks on that bug arguing that it doesn't actually hold, which is good and important.
But it's actually quite reasonable to have a guarantee from a platform's libc that this will work and leverage that. The compiler isn't generating C code or code bound by any standard. This is about two parts of the implementation of C agreeing about what internal invariants will hold.
The problem this bug highlights (and it is a real problem, and not one anyone takes lightly) is that we have insufficient agreement about these invariants, both for certain platforms (glibc) and even within LLVM. Fixing this remains absolutely critical, but may only involve documenting the agreement.
You can definitely build obj and lib files and link them with files built by MSVC. That works well.
We also support putting DWARF debug info (the standard format) into the binaries, and the LLVM linker LLD will link them and you can debug the resulting program with the LLVM debugger LLDB (or GDB for that matter).
So, I think I have some insight into what's going on. The design and implementation of the modules support in Clang is absolutely being driven by Doug at Apple. Anyone can see that. =] However, there are some other aspects to this effort that were not the focus of an LLVM dev meeting talk. Daveed Vandevoorde has written the proposals to the committee thus far[1], and is continuing to work on the proposal and language-design side[2]. Doug is currently the one driving the implementation forward, but Clang and this implementation is completely open source, and the intent (to my knowledge thus far) is absolutely to converge with the proposed standardized feature.
[1]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2012/n334... [2]: http://youtu.be/8SOCYQ033K8
I expect lots of others will end up contributing ideas and and implementation effort long-term, even though Doug has charted the course on the implementation side and Daveed on the proposal side thus far. I don't think this is at all likely to become a vendor-specific extension with no standards support. This is something the committee is really actively pursuing with broad interest across organizations and representatives.