Hacker News Reader

Comment by johnklos | Hacker News Reader

So much of this, such as the backpedaling on support of AVX512, seems to be at least indirectly related to the desire to "solve" the problem that occurs when Intel wants to have little (efficiency) cores that might not have the same feature set as the big (performance) cores.

Two decades ago I put a Pentium III and a Celeron on to the same dual processor motherboard (using a Celery to Nintendo cartridge adapter), and said, "Look! Different processors can run at the same time in the same system. Perhaps we should have a way for processes to express a desire to use specific CPU features, and if they request features not available on all CPUs, we'll keep them on the ones that do." Sigh.

Intel should've told OS writers years ago that heterogenous CPU features would be a thing soon, so prepare for it. Then we could've easily had AVX512 on big cores and AVX256 on little cores, and it'd've all be just fine.

View on HN

No it wouldn't because promotion of a process from an E core to a P core isn't free. Microsoft and Intel could literally do this today on existing hardware by trapping unsupported instructions and then having the OS scheduler promote the process but they don't because it takes so stupidly long and requires an immense burst of resources. All this AVX10/128 is a necessary evil because the far better option is for an E core to be able to at least run the code, even if it's not performant.

How long is "stupidly long"? I get that it's expensive to transition back and forth repeatedly, but if it's on the order of even tens of milliseconds, you only need to do it once and then forever pin that process to P cores. And you could easily have a preamble in the init that checks the instructions you need to trigger that on boot if you wanted to front-load that cost. Heck, you could have it be a filesystem attribute that's automatically set when the promotion is noticed so that you just do it with 0 overhead after the very first execution.

Executable formats like ELF and PE already allow pretty much arbitrary tags - just define one that says "This executable uses instruction set extension X" and you don't even need the cost of a trap or modifying the file/filesystem metadata at runtime.

Sure, old apps might not then have that flag, but old apps aren't using avx512 either.

Great then all processes will be running on the P-cores only, with the E-cores idling all day.

Note that even simple things like memcpy, which every single program out there will use, will use AVX512!

Actually not, on "recent"/"not too old" x86_64 architecture, "rep movsX" instructions are now hardware accelerated for short and big memory blocks. Memcpy and other basic memory operations are getting their code generated directly by the compiler which shortcuts the OS runtime.

Generally it seems things that benefit from super wide simd also benefit from threading. If productivity apps want to leave performance on the table, so be it.

But really I think the entire ecore/pcore split for avx512 is academic, as I'm not sure tying features and flexibility to wider registers makes sense even on pcores, as I'm not sure the hardware area cost is worth the benefit. I honestly wouldn't be surprised if newer architectures don't have the wider registers options on even their larger cores.

You're chasing a pretty small market IMHO of people who have datasets large enough to benefit from large registers and wider alus, but not so large it's worth it to pass it over to an even more specialized accelerator.

The benchmarks that tend to show 512-wide simd benefits may get even bigger benefits from running on a GPU.

I’m sure there are Avx512 apps that wouldn’t have this flag which poses a problem for Linux in terms of back compat.

this is already done by libraries and how new instructions are leveraged on cores that have them. During init a library checks availability of instructions and sets function pointers to relevant routines.

Yeah, and you could set the process cpu masks at init time in your own code and do the equivalent. Hell, Intel could add that init step as a flag for their compiler.

As mentioned elsewhere, I'm not sure this is hardware design being forced to adapt due to software implementation difficulties, so much as avx512 as a whole may not be worth the hardware area going forward. Even on the larger cores.

Or, intel could do like AMD and just design P cores that are space efficient enough to stuff them into a chip like E cores while still getting good performance.

AVX 10/128 doesn't sound particularly useful, because you still need to do AVX2 at 256, so you're at best skipping half the 256-bit registers. But still doing some 256-bit calculations (even if it's double pumped)

I'd think you could do some pretty good heuristics, like if the thread hit P-core instructions in the last time slice, don't schedule it on an E core for the next one. When a time slice starts on a P-core, leave the specialty instructions disabled, so you can monitor usage --- if you trap, enable it, and return; if that cost is still too high (which it might be), keep track of how many time slices in a row hit the trap, and maybe enable the P-core instruction preemptively for a few slices.

Or just, give the program more information and let it decide. If you've got some cores with avx and some without, maybe the process wants to schedule only on the avx cores. Or maybe it can schedule some threads on any core and others need an avx core. As long as the possible permutations at run time aren't too crazy, it's reasonable.

> Microsoft and Intel could literally do this today on existing hardware by trapping unsupported instructions and then having the OS scheduler promote the process but they don't because it takes so stupidly long and requires an immense burst of resources.

Why not just determine the right cpu to run on by examining the arch of the binary? Waiting for an instruction failure seems ridiculous.

Microsoft’s implementation of C standard library routines like `memcpy` is checking for a global flag, and use AVX when available. The flag is initialized on startup by testing the output of `cpuid` instruction.

For this reason, most binaries built with modern Visual C++ are technically using AVX instructions even when compiled for SSE only. It doesn’t mean these binaries gonna fail without AVX support, it only means they’re capable of using AVX when available.

Because you don't want any AVX512 binary to get stuck on a P-core forever.

Also, if application developers got to choose all of them would build their apps to request the most performance. And then how do we put anything on an efficiency core if every application claims to need to run on a P core. Back to square one it is.

Why not use a prominent OS display "power efficiency" so the user can harass devs about a calendar app running on P core

Because your calendar app happens to use a jpeg decoding library that uses "high power" instructions.

Because the binary might contain a JIT, or even more simply, it might load a DLL with new code.

Both the library and the AVX512 generating JIT binary would have the arch set to AVX512 in the executable format header.

Well, but what if it doesn't? Also, who's supposed to set that bit? The programmer, or the compiler? In either case it's possible to set it wrong.

If the compiler generates the instructions, the compiler has to set the correct arch, if the JIT does that, the author has to override the arch in the build scripts.

If the arch is incorrect, it is a bug, and program will crash on illegal instruction. Ie like if you had an ARM binary that was incorrectly set to x64 and was run on x64.

This isn't an architecture, though. It's an instruction set.

What if the JIT generates new instructions dynamically and initially there are no AVX512 instructions but later on in the process' lifetime there are?

The instruction set is the major part of an architecture spec from the point of view of an userspace application. You can design the binary header flags to express instruction sub/set when you find that reasonable. In any case, regardless if the instructions are hardcoded to the text or generated during the runtime, you need to communicate the intention to use the specific instruction set to the kernel, so it can schedule your process on appropriate cores, and check if there are such cores present in the first place.

Then it crashes with an illegal instruction exception.

The problem with that is that you'd then have processes that could only run if a big core was powered up. The main point of having little cores is so you can power down the big ones, so you want to be able to run anything on a little core that can run on a big core.

FWIW, I remember Intel manuals years ago warning that CPUID results might change across cores, but it seems software did not prepare :)

Reply on news.ycombinator.com