LLVM 16.0.0 Release
discourse.llvm.org
discourse.llvm.org
That wouldn't be so interesting, but it highlights the complexity of specifying a language and implementing a specification. For what sounds at first hearing a lot like "#define nullptr 0" the Clang release notes identify 7 syntactic cases where the C standard and the C++ standard disagree about whether "nullptr" should be acceptable in a location. Clang takes each side about equally and it sounds like they are waiting for the standards groups to fix the standards.
The C11 standard says that a null pointer constant is defined as follows: "An integer constant expression with the value 0, or such an expression cast to type void , is called a null pointer constant."
The new definition is: "An integer constant expression with the value 0, such an expression cast to type void , or the constant nullptr are all a null pointer constant."
The reason for adding nullptr, is because NULL is roughly defined as a macro that expands to "0" or "(void*)0". This causes trouble with the addition of type detection trough _Generic and now typeof.
The number of bits in a char has also been implementation defined, yet most if not all code either doesn't care, or assumes it to be 8. This is another one of those "in theory, theory and practice are the same; in practice, they're not" things.
struct S { int i; };
int S::*a = &S::i; // probably 0 under the hood
int S::*b = nullptr; // probably -1 under the hood
int S::*c = 0; // probably -1 under the hood (no typo!)
Consider using brace initializers as an ergonomic alternative: struct S d = { 0 };
struct S e = {}; /* as of C23 or C++ */https://github.com/llvm/llvm-project/tree/main/llvm/lib/Targ...
https://www.phoronix.com/news/LLVM-Xtensa-Backend
https://github.com/espressif/llvm-project/issues/4#issuecomm...
https://github.com/ziglang/zig/issues/5467#issuecomment-1465...
More discussions:
https://github.com/espressif/llvm-project/issues/4#issuecomm...
Llvm is here:
https://github.com/llvm/llvm-project/tree/release/16.x/llvm/...
But I don’t know if there should be lld as well…
The heuristics are, understandably so, somewhat conservative [2], and I presume there are some projects that could benefit from enabling `-mllvm -funcspec-for-literal-constant`, raising `-mllvm funcspec-max-clones=3` ad lowering `-mllvm -funcspec-min-function-size=100`.
[1] https://godbolt.org/z/jKo7K3jPd
[2] https://github.com/llvm/llvm-project/blob/main/llvm/lib/Tran...
I'm less familiar with Rust, but I'd have assumed trait implementations specialize there as well.
Yeah - as I understand it, traits + generics in rust monomorphize basically everything.
If anything, for rust I want the compiler to learn to do the opposite of this optimization. I'd like the compiler to be able to emit code which uses dynamic dispatch instead of monomorphization when optimizing for code size, or in cold functions. Monomorphization makes the rust compiler slower and rust binaries larger. Outside of hot loops, its often not worth it.
You can change this manually by rewriting rust functions to take dyn function pointers instead of <F: Fn> generic arguments. But I'd prefer the compiler to make that choice for me automatically based on PGO or from -Oz.
In D130810 the speedup is given as:
Performance linking some programs with --threads=8 (glibc 2.33 malloc and mimalloc):
- clang: 1.05x as fast with glibc malloc, 1.03x as fast with mimalloc
- chrome: 1.04x as fast with glibc malloc, 1.03x as fast with mimalloc
-internal search program: 1.08x as fast with glibc malloc, 1.05x as fast with mimalloc
In D133003 the speedup is given as:
Speed-up with mimalloc and --threads=8 on an Intel Skylake machine:
- clang (Release): 1.27x as fast
- clang (Debug): 1.06x as fast
- chrome (default): 1.05x as fast
- scylladb (default): 1.04x as fast
Speed-up with glibc malloc and --threads=16 on a ThunderX2 (AArch64):
- clang (Release): 1.31x as fast
- scylladb (default): 1.06x as fast
With mimalloc, a medium-sized C++ test repeatedly takes exactly 43.7s to build and link on this system, while with the default allocator it's all over the place from 46s to 49s. I would loosely characterize that as a free 10%.
[1] https://www.npopov.com/2020/05/10/Make-LLVM-fast-again.html
From the article:
I think this is the first time in quite a few releases where we end up with an overall regression. However, the situation is not quite as bad as it looks.
In particular, the large regression on the right is due to enabling C++17 by default. The close to two times slowdown in 7zip O0 builds comes down to STL headers becoming 2-3 times as large in C++17.
While this is sad, and I had to edit out some choice words on the C++ standardization committee (I hear that this gets even worse in C++20), at least this does not affect non-clang compilers (e.g. Rust) and C code.
Clang might have become slower over time, but it still has a long way to go until it gets down to MSVC's average performance.
In this particular case though, no it does not seem it matters, as the slowdown is related to C++ headers in the newer standard lib, not to the low-level compilation.
"One of the most common mistakes made by new language frontend projects is to use the existing -O2 or -O3 pass pipelines as is. These pass pipelines make a good starting point for an optimizing compiler for any language, but they have been carefully tuned for C and C++, not your target language. You will almost certainly need to use a custom pass order to achieve optimal performance."
The tl;dr is that check builds are faster (i.e. optimization quality has improved) while debug/opt builds are mixed, but slightly slower when averaged over everything.
gcc-11.2 -O0 0.02
clang-12.0.1 -O0 0.04
tcc 0.002
I wish TCC was still being maintained and supported by modern toolchains.... which says at the top:
Note: I am no longer working on TCC. Check the mailing list to get up to date information.
So, sounds pretty dead. But actually following the suggestion and taking a look at the mailing list helps:https://lists.nongnu.org/archive/html/tinycc-devel/2023-03/t...
And the source code repo seems pretty active:
Looks like they implemented it. Not sure about if they added all of the specialized containers though.
Thanks.
The problem however is that thousands of projects that still haven't updated. Gentoo developers are trying to patch the packages but as can be seen in [1] that tracks all the relevant bugs, not even half of them are fixed. Also those are fixes in Gentoo - AFAIK while the Gentoo devs are trying to upstream their patches, not everything is patchable. Note that i mention Gentoo since they were the ones that showed up in the discussions i've seen, but the same is case with all distributions (and projects like Homebrew, etc) - it is just that most likely Gentoo users were affected the most as they build everything from source and can use Clang for that.
But there seems to be two additional issues with Autoconf specifically (remember that this isn't just about Autoconf, it is just that due to how the existing Autoconf checks were written, configure scripts generated by it are the most likely to encounter the issue):
The first is that while there have been fixes for the failing checks, there hasn't yet been a new release of Autoconf that uses these checks - when i run a freshly regenerated configure scripts on my PC with the latest stable release as provided by my distribution i still get the failing checks if i force Clang 15 to use the options that were previously disabled. This means that any configure scripts generated right now will fail when those become enabled again.
The second is that some widely used macros in Autoconf[2], those to find if a library exists and to find which library exposes a symbol (e.g. if libm is needed to be linked to access the functions in math.h or they are part of the standard library - different OSes still need different options here) do it by writing and attempting to link a small C program that defines the function as (IIRC) "char the_function_name();" with code that calls that function (the function is not actually called, the check only tries to compile and link that program) and if that fails it assumes the library does not exist / does not support the function. Problem being is that the above fails with Clang 15.0.0 because of the "()" part (should have been "(void)" but when the check was written many years ago it was meant to work with pre-C89 compilers that didn't support the void keyword). This is fixed in Autoconf now but as mentioned above, no release yet so any currently made brand new configure script will have this check failing. But the more important issue is that this check seems to be a ticking bomb - at least as far as Clang is concerned - since it can trigger various errors in the future and judging from the responses by one of the Clang developers[2] (i actually recommend reading the entire thread) they seem to be of the opinion that treating this as an error in the future is something they might do as it is technically UB. As of right now there hasn't been any fix for this and from the discussions from that thread, the Autoconf developers do not seem to be able to figure out a generic solution (the only solution they found is to include prototypes for all known libc functions but this wont work if you, e.g., want to figure out if the system has curses or ncurses - as an example NetBSD needs -lcurses not -lncurses, at least last time i checked). So even if a new Autoconf release is made now, the configure scripts might still break in the future if Clang developers decide to make this an error (AFAIK even without a release some distros have patched Autoconf to fix the errors introduced and reverted in Clang 15.0.0 but even those do not have a fix for the above).
Also all these are about freshly generated configure scripts, in practice there are projects out there where their configure scripts haven't been regenerated for a long time (often because there hasn't been a new release for a long time). Though many of them can be brought up to date with "autoreconf -fi", assuming of course the fixes i mentioned above are available.
(FWIW from what i can tell this isn't a problem with GCC and the devs contemplated adding a special check for Autoconf generated code to skip the errors if they decide in the future to make these errors, but of course this only affects Autoconf - while i focus on it, Autoconf isn't the only affected program - and regardless the entire point of Autoconf is to allow whatever compiler and tools there are around instead of requiring specific ones from the user)
Considering the above i wonder if Clang 16 decided to proceed with the changes regardless. The release notes do mention some changes that might affect configure scripts[4] but i'm not sure if the notes there are just some heads up for people making configure scripts to check for potential breakage in the future or honeyed words for "technically not every configure script will break, so hey, we warned you". This is a bit amusing since if you check in the discussions i linked to, a concern others had was that the release notes in the version of Clang 15 barely made a mention of the potential for breakage.
(as a side note, all the above focus on Autoconf might sound like me ragging on Autoconf/Autotools, but this wouldn't be farther from the truth - having built a lot of software from source code, including on minimal non-Desktop non-Linux Unix-like environments like NetBSD, i absolutely love that Autotools-based releases do not require anything that isn't already there in a Unix-like environment to work and even recently decided to use it for some of my own projects because of that, which is how i found all the above. From a practical perspective it isn't that hard to work around the issues even with existing configure scripts - AFAIK some of Gentoo patches do exactly that - so despite all the above it is still my favorite build system to build stuff with as a user... as a developer, well, i haven't found it to be any worse than other build systems)
[0] https://discourse.llvm.org/t/configure-script-breakage-with-...
[1] https://bugs.gentoo.org/870412
[2] https://www.gnu.org/savannah-checkouts/gnu/autoconf/manual/a...
[3] https://lists.gnu.org/archive/html/autoconf/2022-11/msg00006...
[4] https://releases.llvm.org/16.0.0/tools/clang/docs/ReleaseNot...
Still catering for pre-C89 compilers in 2003 was questionable. Doing it 20 years later in 2023 is beyond questionable. It's not a portability problem any one of us has had to care about in decades. And it's now a source of problems because it's breaking with current tools. That defect is squarely upon them.
Portability problems are of their time, and old workarounds can be dropped once they are no longer required. Many of the Autotools checks and workarounds are for platforms which were retired decades ago. The whole lot could be deleted. When you look at the cost:benefit of them, they are utterly niche and are largely untested by anyone. When you actually look at which tools are portable in practice, the autotools are usually worse than the alternatives like CMake, because they ceased to keep up-to-date with contemporary systems.
As the other poster mentioned, the fact that the generated code is embedded in thousands of projects, all of which need independently updating is another aspect of its design which should also have been retired once the tools reached maturity. There is zero need for this, just require them to be installed like any other build dependency.
Also as i replied to the other comment, you can still regenerate the configure script if you have Autotools installed, however the benefit of Autotools is that you don't have to have Autotools installed to build a project (you can always generate those scripts on another computer).
I'm really asking about the bigger picture about the whole philosophy of what problems the Autotools are trying to solve, and why. Yes, the Autotools are ostensibly for "portability", but if you dig a bit deeper you start to appreciate firstly how outdated it is in terms of the specific portability problems it tackles, with many current portability problems being completely unaddressed, and how superficial a lot of its solutions are. At one time, the Autotools were the showcase for how to solve portability problems, but this is no longer the case.
Regarding regeneration of the scripts. Of course you can. But embedding is superficially helpful but ignores the bigger-picture issue of having outdated junk in thousands of projects' release artefacts. No other build system actively recommends embedding themselves in source releases. And we all use them without trouble. If anything, embedding generated files also exacerbated the (historically) poor compatibility story of Autoconf and Automake, back when it was being actively developed. Embedding hid this to some extent, though developers often had to have several versions of each installed. The only reason this stopped being such a problem is because the maintenance of both effectively ceased. (Yes, I know they are still nominally maintained, but they are in end-of-life maintenance at this point.)
Nobody bothered to updated not because of maintenance but because it wasn't broken. The moment it was pointed out that this is an issue the developers fixed it - this was already fixed in the repository last year.
> I'm really asking about the bigger picture about the whole philosophy of what problems the Autotools are trying to solve, and why.
Autotools, or at least Autoconf, are trying to solve the problem of figuring out at build time if something is available while minimizing the dependencies the user (builder) has to provide to what is assumed to be there.
> Yes, the Autotools are ostensibly for "portability", but if you dig a bit deeper you start to appreciate firstly how outdated it is in terms of the specific portability problems it tackles
Sorry but this "outdated" aspect is not specific, ask different people and they'll come up with different answers of what is outdated. The only thing that matters is if it works and so far it seems to work.
I'm not going to defend how Autotools/Autoconf work or are implemented, because they are a mess, but i do defend both the idea of minimizing dependencies for the users and the test-based approach they are using.
> embedding is superficially helpful but ignores the bigger-picture issue of having outdated junk in thousands of projects' release artefacts.
As i wrote, the question is a matter of if they work or not. If they work, then that is perfectly fine, they do what they are supposed to. If they do not, they can be regenerated using Autotools, which brings it back to almost the same position as other build systems that need themselves available to work.
And i write "almost" because Autotools are still better here: chances are they'll still work and if you need to regenerate the scripts you can do it in another machine that has the dependencies already available.
> No other build system actively recommends embedding themselves in source releases. [..] And we all use them without trouble.
Sorry but this comes in complete contrast with my experience actually building software with Autotools, especially on platforms that aren't your standard desktop Linux setup and do not have everything and the kitchen sink there. Projects using autotools just work there and provide the best UX of all other systems.
Hell, i mentioned i recently decided to use Autotools for some of my own projects - i didn't because they were nice or easy to use (though they weren't particularly hard either). If anything as a developer i find something like premake much easier and convenient but i also care about the user's experience and Autotools not only do much better there but also give a bigger bang (features) for buck (effort from my side and dependencies for the user).
Also Autotools do not embed themselves in the source releases, they generate script files to generate the makefiles. Autotools have dependencies for themselves that are not required to build the software releases (unless you want to regenerate the scripts).
Also note that you can still regenerate configure scripts if you have Autotools installed, so it isn't like what you describe cannot be done with how Autotools are right now. According to a comment in the discussions i linked to, Debian already regenerates the configure scripts as part of their building process.
The difference is that unlike other tools, you don't need to do that - for example i can easily download the source of some program with an outdated configure script on my main Linux PC, regenerate the script with my PC's version of Autotools and then copy the resulting package to another machine running -say- NetBSD that doesn't have Autotools (or any of its dependencies) installed and be able to build it there.
The issue here is that functions are called without a function prototype, aka the proper headers included. The compiler basically guesses what the function prototype is. This is certainly error-prone, and the fact that autoconf relies on tons of test code pieces that aren't really correct C code is if anything worrying.
It's also probably a good thing that this shows up first in clang, as most linux distros are built with gcc, so you have a testbed to fix all that stuff. gcc should eventually catch up and introduce the same default.
This is a perfect illustration of conceptual bankruptcy of the autoconf approach. Using compile/link tests and basing a decision on whether they succeeded or failed without distinguishing the reasons for failure will inevitably lead to silent false negatives.
If you are wondering what to do and what are the alternatives, here is one approach: https://github.com/build2/libbuild2-autoconf
The tests are actually fine as an idea, it is much better to perform tests to see if something is there instead of trying to guess based on other means, like OS name or whatever.
The problem isn't the idea of testing for the features you need (like checking if a header file or function is there), it is that the code generated for some of the tests was pre-C89. This was accepted by pretty much all C compilers from the 80s to today, until Clang 16 decided not to accept it anymore.
Technically they could have taken a different approach like checking the libraries themselves directly (for the case of libraries), but that'd require knowledge about the library formats and availability and i'm almost certain that the current approach gives the largest portability while requiring the least knowledge about that system.
Even now it is Clang 16 specifically that has the issue and it was already been fixed upstream last year.
However they aren't recent and LLVM usually changes a bit between releases.