Hacker News Reader

Comment by MuffinFlavored | Hacker News Reader

Does this mean the Linux x86_64 kernel could only be built with gcc before this LLVM release?

Are there any downfalls of switching from gcc -> LLVM for kernel compilation? Why might somebody want to stick with gcc, why might somebody want to switch to LLVM?

View on HN

> Does this mean the Linux x86_64 kernel could only be built with gcc before this LLVM release?

It was already possible to compile the kernel with clang/llvm. But a few releases ago, the Linux kernel dropped support for being built with older compilers which did not have support for the "asm goto" gcc extension.

The Linux kernel has always targeted gcc exclusively, and makes heavy use of gcc extensions since its very first release. The only reason clang can compile the Linux kernel is that the clang developers implemented support for the many gcc extensions the Linux kernel uses.

Interesting so this means that if someone really want llvm to support an extension (say a C attribute) that GCC supports, then you "just" have to make a contribution to the Linux kernel that use the gcc attribute and that is a minimum useful. Then LLVM will prioritize the support of said attribute.

As such, hardcore developers might become new Linux contributors just as an Indirect (but effective) way to force LLVM/clang to add support for an attribute.

Who in the world would do that rather than just supporting both compilers?

You don't seems to understand that extensions/attributes can have a usefulness that far outweight cross compiler compatibility?

Type of reasons: Performance critical; Security; Expressivity;

Each of them can be a necessary reason. E.g "polymorphism" in C through transparent unions

Rational indeed

Yes, this is in fact a viable strategy. Source: I contributed a few GCC extension in Clang forced by Linux kernel.

No but it's complicated. We shipped ARM64 clang built linux kernels with clang-4.0 in Google Pixel 2.

x86_64 was buildable up until ~4.20 Linux kernel release. (so we could build 4.4, 4.9, 4.14, and 4.19 LTS x86_64 kernels pre-clang-9). The x86_64 maintainers required the use of CONFIG_JUMP_LABEL/asm goto in the 4.20 kernel release IIRC.

Certain configs and drivers needed compiler fixes.

Pretty soon we'll be moving Android and ChromeOS' arm32 kernels over to Clang as well. CONFIG_FUNCTION_TRACER makes the compiler emit __gnu_mcount_nc calls with a non-standard calling convention just on arm32. You should have seen the Apple reviewer's reaction to the LLVM patch implementing support in LLVM for that!

There's also traces of code in the kernel that suggest it was once build-able with ICC at one point. The code looks broken to me and I suspect it doesn't work. That said, if anyone knows anyone compiling the Linux kernel with ICC, I'd love to get in touch.

I want to say Intel was advertising compiling the kernel with ICC around 2009 or so. (ICC and I parted ways around 2010).

Mainstream yes. Android has moved to clang for quite a while now, the Linux kernel is the only GPL piece left.

Fuschia seems to be non-GPL. Are Google trying to remove GPL entirely from Android?

Every major corporation is trying to get rid of GPL. On the embedded market Linux is competing against alternatives like mbed, RTOS, Zephyr, NuttX.

In a couple of more years, we will be back to days of public domain and freeware, a subset of features free and the enticing ones all closed source.

Or running in some cloud (aka mainframe), with the only open part being the browser code, or not even that depending how WebAssembly + WebGL/WebGPU happen to evolve (depending on how Google and Apple feel like driving it).

Then the anti-GPL crowd might remember the days that they got GCC, GNU and the Linux kernel, or maybe not.

Although for many of us that deal daily with commercial software that is kind of irrelevant anyway.

The biggest issue of fuchsia isn't its license, it's rather the design choice to allow proprietary driver blobs. Linux, for the exception of a few blobs here and there, rejects such an idea. In other words, the question is less about license and more about attitude / tolerance for proprietary extensions/tooling. The main power that an open source maintainer has is the design of the codebase and what to accept inside, not the power to litigate a copyleft license. The more it cooperates with proprietary software, the more likely proprietary software can establish itself and restrict user freedoms.

Google has already achieved that with Project Treble by the way.

But they had to pay for the project. For others it might not have been economic to do it. Also, in fact there were Android-custom abstraction layers before project Treble, and the Treble abstraction layer is not around the device drivers but the entire kernel including the drivers.

They're following in Apple's stead, I guess.

I am removing GCC/binutils from Android.

I wrote: https://android.googlesource.com/platform/prebuilts/clang/ho...

It has nothing to do with GPL. It has everything to do with Google having 50+ LLVM developers and 0 GCC/binutils developers.

Maybe some of them could be reassigned to the NDK tooling then.

I and other NDK users have several ideas of what is missing versus Java/Kotlin.

A lot of my teammates focus on LLVM for Android, including the NDK, which redistributes AOSP LLVM.

> what is missing versus Java/Kotlin.

I'd be happy to pass on feedback, though as worded I'm not sure if you're referring to language features or what exactly?

As mentioned already a couple of times with online discussions with Dan and AMA on Reddit, from me and others:

An NDK version of Android JetPack/AndroidX, for starters proper C++ API, instead of forcing everyone to code 90's style, each re-creating their own C++ wrappers for the Android C++ APIs exposed as C function calls.

Which by the way, given the whole security theme on Android, means more insecure code than if we had those C++ APIs to start with.

Have one shot libraries like Oboe actually be part of NDK and not in another github repo with uncertain future versus the Android platform.

For basic stuff like reading files (using the platform codecs e.g. BitmapFactory, now SAF as well), asking permissions, calling Java from C++, it would be nice if all NDK users wouldn't be force to keep writing those from scratch as rite of passage.

Actually provide build infrastructure to create NDK libraries that can be shared across Android projects, instead of forcing us to somehow cram library and header files into AAR packages while having build scripts extract them for correct placement where cmake and ndk-build expect to find them. cdep never went anywhere, while conan and vcpkg aren't the right answer when many NDK libraries need Java code to come along due to JNI wrappers for like 90% of Android APIs.

Android Studio networking debugger support, currently only supported for networking calls done in Java/Kotlin code.

Easier CMake support for multiple libraries on our projects, which is found lacking versus how easy it is on ndk-build.

As mentioned, these are just a couple of issues that we keep mentioning, always getting some kind of "thanks for the feedback, we will look into it". Just like the NDK status talk at Google IO 2019.

From the outside it doesn't look like NDK is that staffed with engineers, more like a 20% project where a little bit is done every year, like how long it took to fix the header files, adding minimal C++ support to Android Studio, or settling into CMake as the future build system.

EDIT: Almost forgot this one, fix the regressions regarding missing GCC features in clang (Android NDK tracker used to have a couple of them listed).

Thanks for the feedback, we will look into it. :P

I'm happy to pass these along to Dan and your comment is easy for me to link to, but I don't know why you're suggesting Google move LLVM compiler engineers to work on NDK, which is more of a userspace C++ framework for third party applications to interface with the Android platform. Kind of feels orthogonal to this thread which is about the llvm 9.0 release.

> fix the regressions regarding missing GCC features in clang (Android NDK tracker used to have a couple of them listed).

Now that sounds more like work for Android LLVM engineers, but it sounds like the list is removed? Could you elaborate and point me to some? (Assume anything you claim in a response I'll just ask for a link)

* EDIT * (it seems I can't reply further)

Thanks for the links. I'll pass on both links as feedback and follow up on the state of those two LLVM reported regressions.

I am suggesting that, because as I mentioned it looks understaffed from the outside, either that or there isn't that much willingness to spend resources on improving it beyond "good enough".

Given that you mentioned, that there were actually so many LLVM engineers around, I thought they could give them an hand. :)

Regarding the clang regressations:

https://github.com/android/ndk/issues/742

https://github.com/android/ndk/issues/721

There are way less than I remembered though, sorry for the confusion.

> C++ libraries etc

We do have limited resources, so we have to prioritize what gets done first. We couldn't have provided good C++ APIs before we were even able to provide good C++. We now do provide a pretty good C++ language experience, but this is still not at the top of the todo list. No one else I've spoken to thinks this is more important than the other work we're doing.

> Actually provide build infrastructure to create NDK libraries that can be shared across Android projects, instead of forcing us to somehow cram library and header files into AAR packages while having build scripts extract them for correct placement where cmake and ndk-build expect to find them. cdep never went anywhere, while conan and vcpkg aren't the right answer when many NDK libraries need Java code to come along due to JNI wrappers for like 90% of Android APIs.

This is what I've been working on for most of the year.

> like how long it took to fix the header files

This only took a single release (the rest of the time was to give users a chance to transition), and we did plenty of other things in the same release, so I'm not sure what you're getting at.

> Easier CMake support for multiple libraries on our projects, which is found lacking versus how easy it is on ndk-build.

Not sure what you mean here, but it sounds like CMake is what you don't like? Just use ndk-build then.

Thanks for jumping in.

So it is really a matter of not having the same resources as Java/Kotlin teams.

Thanks for the update on possible NDK library management.

Regarding the headers, what I am getting at, is why it was such a big issue to fix them, instead of being proper from the start, like other OSes, including embedded ones.

Anyway, thanks for improving the experience versus what it used to be in the early NDK releases.

> Regarding the headers, what I am getting at, is why it was such a big issue to fix them, instead of being proper from the start, like other OSes, including embedded ones.

Our time machine isn't working yet ;)

That's before my time, so all I can really say is that we have the benefit of hindsight here.

Surely that's just a layer of indirection over whatever the real reason is? What prevents Google from having GCC/binutils developers on staff (whether by hiring or training)?

Supporting one toolchain is already a ton of work; splitting resources between two toolchains is too costly, even for Google. If you don't think so, you underestimate the cost.

Reply on news.ycombinator.com