Hacker News Reader

> If you were a C++ developer, would you request C++17/C++20 also?

yes absolutely. I think that it is wild that some people stay with older compilers solely because they happen to be on an old distro and don't want to update. Tying compiler versions to operating system versions is absolutely braindead. A compiler is just a program that takes text and outputs a binary which is supposed to work on anything with the expected ABI - you could even cross-compile from windows if you wanted ; at least I've done the opposite (cross-compile windows binaries from a linux host) a few times.

e.g. personally I mostly use clang-13, soon 14, for development with every possible C++20 goodies, and ship software that works back to windows 7, mac os 10.13 and until recently centos:7-era linux userland (recently upgraded to 8). There is zero reason to use an older compiler.

> I don't know why GCC-12 was in the picture, but I hope all of you can understand one thing: python's source code must be compatible with GCC 6. And no so called "Threading [Optional]" features. It is all because CentOS is dead. CentOS has been sold to IBM. There is no CentOS 8 anymore. The free lunch is ended. And there wouldn't be another OSS project to replace it.

the day centos:8 ended I replaced centos:8 with rockylinux:8 in my docker image for my builds and everything continued working fine with the latest GCC and Clang versions.

Honestly your pains regarding gcc-6 are entirely self-inflicted. Just ship a more recent GCC binary with the manylinux project or something, you can build it on centos 5 if you fancy in order to get it to work on older libcs

I agree, and especially in the context of Python. My Mac comes with Python 2.7 and 3.8, but it seems like almost everyone uses Homebrew or pyenv to locally install a newer version. I rarely hear of anyone going out of their way to stick with the system version outside of specific scenarios (like "IT locks down developer laptops") or such.

And while I'm keenly aware that Python and C++ are very different languages, if someone asked if I'd insist on using the "new" Python17 (aka 3.6), like that was wildly and unreasonably bleeding edge, I'd literally laugh at them.

If everyone just use the latest macOS and the latest iOS, I'm less concerned. But I haven't heard any IT manager enforcing that. macOS 12 has come out, but usually they still allows you using macOS 11 as long as you have installed all the security bug fixes.

For macOS and iOS, the problem is more complicated. Let's say I want to publish a package to pypi.org. The package contains some binaries compiled from C++ and it requires C++17. Then what is the lowest macOS version the binary can support?? It's very tricky that if the build machine which generates the package is macOS 11, then it can have Xcode 12.5. Otherwise it has to use XCode 12.4 or lower. And in XCode 12.4 it says std::optional, which is a C++17 feature, is only supported in macOS 10.14+. But if the build machine has macOS 11 and XCode 12.5, then the binary would be able to run on 10.13 too. And in whatever case, it can't support 10.12 or lower.

I believe you don't want to build tensorflow/pytorch packages from source. So the maintainers of these two OSS projects must consider the things above. If you wonder why they don't use C++17, this is the reason. For the same reason they want to avoid C11 optional features too.

Thanks for the info. Would there likely be a downside to saying "if you want to use the precompiled packages, you need to be on macOS 11+ (but feel free to DIY if you're stuck on an older version)"?

> Then what is the lowest macOS version the binary can support??? It's very tricky [...]

No it's not. The oldest supported version of macOS your binary will run on will be the one you set in your CFLAGS with the -mmacosx-version-min=10.x flag when you build.

The OS you are running the build on and the Xcode version don't influence that, you should just run the most recent Xcode you can, build against the most recent macOS SDK available and set that flag. You can target 10.8 from Big Sur and Xcode 12 with afaik no issues.

> The OS you are running the build on and the Xcode version don't influence that

Usually it doesn't. But in this special case it does. I have set the flag you said. When C++17 wasn't in the picture, it's enough. But now we are talking about how to enable developer using the new language features. New C++ features typically need new runtime. But the old systems do have it. Then one of the difficulty is how to figure out which macOS versions have it, which doesn't.

The std::optional / std::variant case is unfortunate indeed, it's sadly an argument for using non-std versions of these types. It's also possible to cheat a bit with libc++ macros.. -D_LIBCPP_NO_EXCEPTIONS=1 does the trick (but turns exceptions into asserts).

You can also link statically against libc++.

So you are conflating tons of things here and none of it really makes sense. The reason why compiling for macOS tends to claim stuff like how C++ features are tied to operating system targets is because Apple doesn't really understand how to do toolchain architecture well and so have managed to do this awkward thing where the operating system--from their perspective--is this massive monolithic brick that contains a ton of functionality that is classically introduced by the toolchain, which I admit has some advantages but generally feels ill-intentioned.

The result of this is that Apple really assumes you are going to use the copy of libc++ that ships on the system, and it might be missing exported systems that are assumed by the library. But this isn't related to the toolchain you are using. I do remember some weird corner case with std::optional and I think it might have been that Apple forgot to mark something correctly as unsupported in a previous toolchain build? The newer compiler is thereby actually giving you the more correct understanding of what can actually be presumed to work on older systems.

But like, the core issue here is that you really shouldn't use Apple's stupid toolchain setup, nor do you have to: just embed your own copy of libc++ and call it a day. I routinely use the latest versions of Xcode's copy of clang (though I prefer to compile for macOS now using the copy of clang that comes with the Android NDK for various reasons: I recommend avoiding Xcode except to get their system headers) to target ancient systems with modern C++ features as I'm not relying on the system copy of libc++ to even exist or work (10.7's doesn't) much less be complete.

In practice compilers and the things they compile are insanely large. If we started updating them more... who knows what bugs we'd get with different GCCs on different OSes...

Up-to-date Linux distros already compile things with latest compilers. The bugs are already caught there. And in my experience, with GCC and clang, updating to new versions consistently reduces bugs.

I agree that there is little reason to not use a newer compiler to target an older distro, but there is one issue: the standard library. With newer language versions the standard library also grows. Older distros ship with older standard libraries, and they probably don't have the newer language facilities in them.

At work we sidestep this by building the new C++ standard library implementation against an old libc, and ship it with our product. I can imagine that this could be problematic for other software though, especially ones that normally ship with the disto.

> I can imagine that this could be problematic for other software though, especially ones that normally ship with the disto.

why would it be an issue ? that's how pretty much every software does on Windows and most certainly people can agree that things are much more sane there for the end-users

This is not a technical issue. But if you aim for the software to be packaged in official distro repositories, they possibly don't want you to have other backported dependencies, such as a new version of libstdc++, or you to vendor in those libraries.

(The reason this is significant is because the manylinux infrastructure is responsible for approximately 100% of the binary Python extension builds provided as wheels)

What? Windows and macOS wheels are a thing, they are built on Windows and macOS respectively, and certainly are responsible for over 0%.

I'm a little confused by this. I'm on the last Ubuntu LTS version (20.04) and the default GCC version appears to be 9.3, with 10.3 available as an option. So why are they stuck on 6?

As a user, if you build every python package from source, it's ok. But if you a maintainer of an OSS project and you need to publish binary packages for it, then you will hit the trouble. Binaries built on Ubuntu 20.04 can only support Ubuntu 20.04 and newer. So you'd better to choose an older Linux release to target broader users. Now most python packages choose CentOS 6 or 7. See https://github.com/pypa/manylinux/issues/1012 for more details. They need help!

>Binaries built on Ubuntu 20.04 can only support Ubuntu 20.04 and newer

So? Who are those people building for servers they don't control and arbitrary old versions of OSes?

Evidently the point of the manylinux project is to make it easy to distribute Python binary wheels for Linux, so that basically is their use case. Though they're not building for arbitrary old versions — they're building for a concrete set of old OS versions defined by Python standards. I think the idea is that you don't want to require everyone installing your Python module to have a full set of build tools installed, so they're providing a way to distribute a binary module that supports a guaranteed set of supported Linuxes.

The manylinux wheels distributed on PyPI need to be built against the oldest glibc imaginable, which is defined to be the glibc on some ancient version of CentOS (a different version for each different manylinux platform tag).

If you build your own wheels, of course no one’s going to stop you from building against anything.

This is how it used to be back when manylinux was separately versioned. Since https://www.python.org/dev/peps/pep-0600/, you just say which version of glibc you expect: manylinux_2_17_x86_64 etc.

Isn't it possible for more recent versions of GCC to build binaries (linked libraries or otherwise) that can be distributed and run correctly in older operating system environments?

(in other words: my understanding is that software languages and tools can continue to evolve while maintaining the ability to build backwards-compatible binaries)

I don't understand any of this logic: you can trivially run new versions of gcc on any distribution you want, and you can trivially use new versions of gcc on new distributions (which is sane) to target arbitrarily-old distributions. I routinely use bleeding edge compilers to target pretty ancient systems, and have for decades now... like, the entire point of gcc is that it is easy to compile it to run on whatever crazy system you have and then it should be able to target whatever other crazy system you have as long as you have a sysroot for it (which is of course somewhat easy to obtain as it is generally equivalent to whatever system you would have used to compile "natively" for that system).

As i see it, the real problem is the lack of a stable linux ABI for binary programs. Because linux does not have one, people need to play around with old distros and compilers to build something that will work on anything released after $SomeOldDistro.

> But does your compiler support it?

Yes, ImportC is a C11 compiler!

https://dlang.org/spec/importc.html

`_Static_assert` and `_Generic`, too.

No C compiler is complete without extensions. ImportC is no exception. It has modules, forward references, and compile time function execution.

> for now Red Hat devtoolset is the only solution. When IBM discontinued the CentOS project, most people do not understand what it means to the OSS community.

"Compilers and Runtimes" > "CentOS sysroot for linux-* Platforms" https://conda-forge.org/docs/maintainer/infrastructure.html#...

From https://conda-forge.org/docs/user/announcements.html :

>> 2021-10-13: GCC 10 and clang 12 as default compilers for Linux and macOS

>> These compilers will become the default for building packages in conda-forge.

Conda-forge specifies enough to solve for CentOS sysroot compatibility and newer GCC is already specified.

CentOS (RHEL SRPMs with RH trademarks removed + EPEL) lives on as {Rocky Linux, Alma Linux, CentOS Stream,} and SUSE is still RHEL-compatible.

https://github.com/pypa/cibuildwheel :

>> Build Python wheels for all the platforms on CI with minimal configuration.

>> Python wheels are great. Building them across Mac, Linux, Windows, on multiple versions of Python, is not.

>> cibuildwheel is here to help. cibuildwheel runs on your CI server - currently it supports GitHub Actions, Azure Pipelines, Travis CI, AppVeyor, CircleCI, and GitLab CI - and it builds and tests your wheels across all of your platforms.

Using an insecure C standard does not improve the situation, broken compilers neither.

C is broken from C11 to C26 by committee. GCC was broken from 9 to 11. Python has at least the option to use GitHub, which can eventually detect PR's with unidentifiable identifiers. Linux will need to use linters to detect homoglyphs or bidi attacks, because reviewing emailed patches is impossible.

Rocky Linux provides a perfectly good and compatible drop in replacement for CentOS 8. So does Alma Linux.

They don't handle bugs. If the bug is reproducible on RHEL, they will suggest you report the bug to RHEL. It scares people away. At this moment, the manylinux community is considering Rocky Linux and Alma Linux, but it's very hard for them to make the decision because of lacking direct support.

It was the same for CentOS.

Huh. Could Python switch to Alpine instead of Ubuntu?

The "manylinux" platform tag definition means GNU libc. musl libc systems are not within the "manylinux" Python platform, they are a separate "musllinux" platform tag.

See: PEP 600, PEP 656.

While the manylinux infrastructure project may use Alpine to produce "musllinux" binaries, Alpine will never be usable to produce "manylinux" binaries. So Alpine would only be useful in addition to another distro, not instead of another distro.

See: https://github.com/pypa/manylinux/pull/1135 .

Alpine is a supplement. It can't replace the others. It hasn't been widely accepted by the community yet. I believe many python projects are not afforded to replace libc.

Even if CentOS 8 is not anymore, we still have CentOS Stream 8.

Which has a different release model than Centos, so it's not a replacement for the business cases Centos was used (basically: long term stability and support).