Hacker News Reader

Comment by CountSessine | Hacker News Reader

Jeez - how is this supposed to work with OS scheduling? The charm of big.little on arm is that the instruction sets between the big and the little cores are identical. Now the OS has to pin processes based on support for different instructions? What a ridiculous nuisance. Intel really couldn’t discipline themselves just this once and actually implement the same instructions, even if in microcode, for both types of cores?

Are the little cores’ instructions at least a complete subset of the big cores’? Are we going to have some ridiculous situations where the little cores are completely pegged but the OS can’t migrate their processes off to the big core?

Or do kernel programmers need to start chasing Intel and trap/software implement every single future AVX8192AESNISSSE instruction Intel jams into future instruction sets to provide Xeon market differentiation?

View on HN

There were some issues with big.LITTLE, like this[1] issue with cache line size differences between Big and Little cores.

1. https://www.mono-project.com/news/2016/09/12/arm64-icache/

Yeah, I remember this. But I think it was just Samsung that made this blunder, no?

NVIDIA had a much funnier (infuriating) one, where Tegra X1 had caches between the big and little cores that weren't always coherent. Their solution was to just turn off the little cores while continuing to spew marketing bullshit about it being an 8-core chipset.

Yes, you look to be right that the Samsung-designed Mongoose core in the Exynos SoC had 128-bytes cache line size, while ARM designed their generic Big cores to also use 64-bytes like their Little cores. Still though, seems like ARM did not provide guidance that these specs must match between Big & Little cores, and Samsung also didn't catch this either.

Different instruction sets are not too bad. Copy from my comment elsewhere.

> The OS can install an illegal-opcode exception handler. When a process is first run on the small CPU, the unsupported opcode will raise the exception. The exception handler can simply set the processor affinity of the process to the main CPU and put it to sleep. The OS will handle it like it normally would - putting the process in the run queue of the affined processor.

What if my program has like 1 avx2 instruction? Then we still wake up the big core and run my process there? And keep running it there? Or do we try to migrate back to the little core every once in a while? And then go back to the big core when we trap again?

Migrate back to the little core if the thread hasn't used AVX in a while.

Linux already tracks how long ago a task used AVX-512. I assume the same mechanism could be used to track AVX as well.

https://git.kernel.org/pub/scm/linux/kernel/git/tip/tip.git/...

It's not quite relevant to your point, but generally you wouldn't want to have one AVX2 instruction as they are only really useful if you have a bunch of data.

Yes, even if the process has 1 avx2 instruction, it will be scheduled to run on the main CPU. I mean you could switch the process back to the small CPU when you detect a long stretch of opcode without the avx2 instruction, but the constant switching between the main CPU and small CPU probably is not good for performance.

Thread migration only costs on the order of 100 microseconds, including the effect of cold caches. If you keep the AVX thread on the big core for at least 100 milliseconds at a time, you only lose ~0.2% performance.

Good to know the switching cost is low. Runtime profiling would be key to provide insight into when to switch back.

That sounds really slow.

There are processor architectures (MIPS) that can't do unaligned memory access where Linux will catch the fault, emulate the memory access in kernel and return to the program. On every unaligned access.

At least this only has to be done once, and frankly these AVX instructions are slow initially anyway.

Yeah, and that was suuuper slow and everyone told you to not do it for exactly that reason…

It's absurdly slow on hot paths, yet entirely unnoticeable to the vast majority of software out there.

Most performance anti-patterns in almost any program are not important at all, as most of the code is not on the hot path.

As it happens, MIPS is also practically dead as an general purpose architecture...

It also happens on 32-bit ARM, which is maybe on its way out, but still way more common.

And MIPS is slow as a dog, in part as a result of software handlers like this (software handles TLB and virtual memory walks, too, IIRC).

> At least this only has to be done once, and frankly these AVX instructions are slow initially anyway.

FWIW I believe only some of them, and those are just some (or all?) AVX-512 instructions. I think AVX2 is implemented on the main die and doesn't have to be powered up first.

https://travisdowns.github.io/blog/2020/01/17/avxfreq1.html

AVX (256 bit) instructions also suffer a penalty. Both the 256 and 512 bit instructions resulted in a slowdown for 9 microseconds featuring a quarter the instructions per clock, but the 512 bit instructions resulted in an additional penalty of 11 microseconds without executing instructions.

The first penalty was associated with voltage, and the second with frequency. Heavier 256 bit instructions would probably have resulted in the frequency transaction as well.

It only needs to be done once at the process launch. Once the processor affinity is set, it will stick to the main CPU at full speed for the entirety of the process lifetime.

It only has to happen once.

Once per process. I doubt operating systems will start modifying executables and libraries on-disk to tag them as using relevant instruction set extensions.

And generally speaking, when an application first starts issuing SIMD instructions, that's probably not the a great time to be interrupting it even if it only needs to happen once.

once per process is perfectly fine.

the issue is that once you have an active process using a feature only available on the larger cores, you can't shut off the larger cores to save power without paying a large latency to wake up that process.

You could imagine it as an ELF note of some kind. It's not a direction I've love to go in, and catching illegal exception traps seems fast enough that the more complicated design doesn't seem worthwhile.

Well, once when it's about to try to wring maximum performance out of the processor, and if you only do it once you're ensured that your smaller cores never get used…

A single trap? That's not slow.

A trap, followed by migrating the process to a different core, which may have been powered off and definitely has cold caches. Context switches don't get any worse than that.

If the core has been powered off, then that shows how rare it is.

Yes, so you pay one really bad context switch penalty per process.

Oh, I thought the suggestion was to trap to the large core every time an unsupported instruction was executed and let the process drift back to the smaller one later. Which would be slow I would assume. If you never switched back, wouldn't any process using advanced vectorized instructions (like anything using a decent libc) be permanently pinned to the large core?

> Oh, I thought the suggestion was to trap to the large core every time an unsupported instruction was executed and let the process drift back to the smaller one later.

Yes that's the idea.

> Which would be slow I would assume.

How expensive do you think a trap is? It takes about the order of 10 billionths of a second.

> If you never switched back, wouldn't any process using advanced vectorized instructions (like anything using a decent libc) be permanently pinned to the large core?

I think you can switch back next time you schedule.

> How expensive do you think a trap is? It takes about the order of 10 billionths of a second.

are you sure about that? I would expect at least a couple of orders of magnitude more just for the userspace->kernel transition.

edit: for what is worth, a syscall it takes 250ns on my (admittedly vintage) machine. That's using the lowlatency sysenter path. An interrupt is probably going to cost more.

Anyway the cost of scheduling on another core is going to dwarf that.

edit2: for reference, this was a Sandy Bridge turboing at 3.5 Ghz during the test. With spectre mitigations on (which is going to be a good chunk of that overhead).

> I think you can switch back next time you schedule.

Ok, yes, then we're on the same page. I would still think that would be slow? You'd need a full transition-to-kernel and context switch before you could execute again, which AFAIK would take at least microseconds…unless you think there would be a faster path to resume execution?

> which AFAIK would take at least microseconds…

No that's around 30 ns on modern hardware I believe.

What if the same bytes are valid instructions on both but do different things?

I don't really think that's a thing on Intel? Instructions that are unsupported generally noop or do something slowly. (Or trap.)

I thought the two cores had two different ISAs?

they support different subsets [1] of the same x86-64 ISA, so opcodes won't be reused.

[1] well, I hope the larger cores support a superset of the smaller cores, otherwise it would really be insane.

No idea. I'm just responding on the assumption that it's just two Intel architectures, one with fewer extensions.

Unless one of those things is "trap," that's just a horribly byzantine heterogeneous compute design that no general purpose OS will target.

I believe the os can turn off features in the big cores, disabling the corresponding bits in cpuid so that the two set of cores will advertise the same capabilities.

If this is going to be anything like how arm did it, intel most likely upgraded the atom CPUs to support all the instructions of the big core, probably just very slowly. For example, you will find that the lowly Cortex-A7 supports virtualization extensions. Why? Because it was paired with big cores that supported them.

The only way to make big.LITTLE work and allow task migration between them is to have them support the same instruction set exactly. So, I suspect that that is why AVX512 was removed from the big core here, and I suspect the atom cores aren't exactly garden variety either. They probably support most other extensions of modern x86 CPUs, at a slow speed. For example, AVX2 can be microcoded by running multiple pieces of the data through the ALU, one at a time.

Brings me back to coding for the ARM7TDMI on the Gameboy Advance - it supported both 16-bit THUMB[0] and the standard 32-bit instruction sets, and you would switch modes at runtime. Fun times. This does seem more annoying, though.

[0]: https://en.wikipedia.org/wiki/ARM_architecture#Thumb

The big core has SMT and AVX-512 turned off, I assume that they're just fusing off everything that the little cores don't have.

“even if in microcode”

I don’t think users would be happy if it were a lottery whether their code runs slow, using microcode instructions, or fast, on the better CPU, especially given that, I guess, the difference in instruction set is for vector instructions (because adding those is what’s makes a CPU big nowadays)

So, performance oriented code will pin its performance-critical threads to the fast CPU. Now, the question is: what code won’t try to claim the faster CPU, given that benchmarks typically run programs while there is no contention from other programs?

I'm not sure how this is different from the existing big.LITTLE "lottery", between being scheduled on a CPU with a deep speculative pipeline and a high clock rate or a CPU with less.

If the microcoded vector ops cause your code to run slow, then you will be consuming lots of CPU time, and should be migrated to the big core anyway, right?

So, performance oriented code will pin its performance-critical threads to the fast CPU. Now, the question is: what code won’t try to claim the faster CPU

Do apps have CCX pinning logic for Ryzen? No? Why would they add it for an Intel processor that probably won't even be sold in volume.

I suppose they'll leverage 'function multi-version'[1], as Clear Linux already does[2] to enable extra instructions on the big cores.

[1]: `target_clones` -> https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attribute...

[2]: https://github.com/clearlinux/make-fmv-patch

That seems hard to do with preemptive scheduling. AFAIK, the decision is on function level, and done at startup. Even if you can decide on call-time, it's only function level, and the preemption can happen mid-function.

That only works if your code is never migrated between cores.

I have to imagine the differing instruction sets means different extensions, not entirely different ISA.

I'm not a programmer, but doesn't compiled code often offer multiple codepaths, checking flags at runtime? So if your code runs on the big core, it finds avx256 is supported, and uses the avx256 codepath. If it runs on the little core, it finds avx256 is not supported and it takes a legacy floating point codepath instead.

The problem is that you want to be able to migrate processes between cores. It's not the end of the world if a number-crunching process gets stuck on the big core. But if a process detects an instruction that it's only using for convenience purposes, and that causes it to get stuck on the big core or stuck on the little cores, that's not good. In a different scenario, if it was set up so that a process starting on a little core never detected an important instruction, and was stuck in slow mode forever, that would also not be good.

It depends a lot on which instructions differ and how the feature flags work, of course. They might have set it up in a way that's fine.

Tons of asterisks here, but: You'll have to explicitly write the code to do that at runtime if APIs are provided to do so. When you build for a target, say x86, you're usually targeting the lowest common denominator that you're willing to support and the compiler might not output less supported instructions even if the CPU it's compiling on supports them.

Yes, most programs target the generic x86-64 ISA which means sse2 at most. Still some have accellerated paths that are enabled looking at feature bits. So a simple solution is for the os to mask out some of those bits for processes that intends to potentially migrate to the less featured cores, while potentially giving the option to let the user "pin" a process to the fast cores enabling the all features.

Maybe it wouldn't be possible to have several processes working in parallel, but instead a single process with 5 cores available for threads? Consider the PS3 arch for instance: it had 7 cores, 1 for management and the rest for computing. Could it be the way Intel envision its future lines of products?

> Now the OS has to pin processes based on support for different instructions?

The only complication here would be if they have differing extensions like AVX512, but that's easily solved by the OS by just advertising the common baseline. Nothing about this looks difficult to support?

by the OS by just advertising the common baseline.

What does that mean? Advertise to whom? The process/process loader? Does it mean that I can’t compile with -mavx2 anymore? What if I do?

The extensions are the whole problem.

> What does that mean? Advertise to whom?

Runtime detection is the process querying what extensions are available, and then selectively using those. You adjust what the query returns to only return the common set.

Runtime detection has been a pretty standard thing for well over a decade now - it's how we all manage to run the same compiled binaries over the years despite variability in SSE & AVX support. You don't download different versions of Chrome/Photoshop/Gimp/Premiere/Blender/Whatever compiled for different CPU micro-architectures, do you? You might if you run Gentoo I suppose, but that'd be about it.

> Does it mean that I can’t compile with -mavx2 anymore?

You already can't if you're shipping binaries to users unless you only support Skylake & newer? There's a lot of CPUs currently in use that don't support AVX2. So... you either already have this problem and you're familiar with it, or you're not doing this and it's moot.

Runtime detection is the process querying what extensions are available, and then selectively using those. You adjust what the query returns to only return the common set.

Except that they almost always do this runtime detection once, on startup, and then choose/thunk codepaths accordingly. If the OS just happens to start my avx2 process on a little core (and how is it going to know better?), that's going to turn off all of my optimizations, regardless of where the process subsequently gets migrated to.

Except nobody in 30 years of x86 dev expects to get a different answer from CPUID during runtime.

If all the cores are configured to advertise the lowest common denominator instructions it will work.

But that defeats the purpose of supporting any extensions at all in the big core that the little core doesn't support. Software will get the lowest common denominator answer and just not use avx2. So why support it in the first place? Why not just do the right thing and have uniform extension support like big.LITTLE?

Currently Intel likes to disable features on some models of Core cores for product segmentation reasons. I'm pretty sure processors sold under the Pentium or Celeron brands already have enough features fused off that they're roughly comparable to the Atoms this is being pared with. You also fuse off things like sections of cache or whole cores that have manufacturing defects.

Chips aren't designed from scratch. They're assembled out of previously designed components and in this case Core cores and Atom cores were never designed to work together.

No - but that just means that Intel shouldn’t do this at all. Either don’t support stuff like avx and avx2 in the big core by disconnecting those blocks or support a slow microcode version of avx and avx2 in the little cores. Supporting different extensions for a CPU used with modern preempting OS’s doesn’t make any sense.

> Either don’t support stuff like avx and avx2 in the big core by disconnecting those blocks

That's partly what they did. From the article: "One thing we can confirm in advance – the Sunny Cove does not appear to be AVX-512 enabled."

Maybe they also fused off AVX & AVX2 support in the Sunny Cove core as well, we'll see.

And disabling AVX in cores that otherwise support it is already a common thing - see the Pentium & Celeron lineups that Intel currently sells. They don't have AVX/AVX2, even though the cores inside them definitely could offer it.

.. and within a function because the scheduler decided to move you to a different core.

The typical x86 extension query (cpuid) is an unpriviliged user-mode instruction — unless the application takes care to ask the OS, there's no real way for the OS to select the common denominator.

Edit: Also: AVX2 is a lot older than Skylake. You're probably thinking of AVX512.

The OS could still intercept CPUID - this is, after all, what VMs do.

But it looks like Intel is doing this anyway, as Sunny Cove in this application has had its AVX-512 removed anyway: "One thing we can confirm in advance – the Sunny Cove does not appear to be AVX-512 enabled."

Running every thread in a hypervisor just to trap cpuid sounds like extreme overkill. And probably some applications want the features only available on the big core, and masking those out doesn't solve that end of the problem.

Well then you need to set up a VMM for the OS…

> You adjust what the query returns to only return the common set.

But this shoots the big core in the foot. You run at the lowest common denominator and the big core doesn't have the advantage of higher clocks. And this might just lead to a lot of software that only runs on the big core.

Imagine you wanted to fly around between multiple points but if you want the option to ever switch to a bus then your plane will circle around each airport until it's as slow as the bus. Or you can opt for "plane only".

It only shoots the big core in the foot for things that would make meaningful use of the extensions present on the big cores but not on the little ones. Which in a 7w application is what, exactly?

Anything that uses any AVX or FMA3 off the top of my head.

A better mix would have been simply using lower clocked, lower powered cores of the same type or really close derivatives of the big core where the manufacturing process and clocks are what keep power low. Not a mix of Ice Lake and Atom. But right now Intel would throw everything at the wall to see what sticks.

And it seems like a good way for developers to make sure their software stays on the big core.

The fact that they put the extensions in implies they expect meaningful use, doesn't it? If not on this specific part, on a future part with multiple big cores.

Meaningful use in a particular market != meaningful use in all markets.

AVX2/AVX-512 is great in HPC workloads, for example. But nobody is running an HPC workload on a 7w netbook, now are they?

What is useful on Xeon and what is useful on Atom are different. This is an Atom-class SoC used in Atom-class applications, not a Xeon-class one.

That doesn't address what I said at all. This is about extensions that are in this Atom-class SoC.

memcpy?

>> Does it mean that I can’t compile with -mavx2 anymore?

> You already can't if you're shipping binaries to users unless you only support Skylake & newer?

Huh? My 6+ years old gaming PC supports AVX2, and it definitely doesn't have a "Skylake & newer" CPU!

AVX2 support started at Haswell, or Intel core 3rd generation. We're at gen 10 now.

While there certainly are still a lot of systems without AVX2 support, new games (and other performance hungry software) requiring it would not be completely unreasonable.

Intel is still launching new processors with AVX2 disabled. They use AVX2 support for product segmentation and disable it on low-end parts. So a Comet Lake Pentium Gold CPU launched in Q2 2020 doesn't support AVX2.

(I recall a recent news story about how unawareness of this among people writing or documenting compilers has started to cause problems.)

Using IA for market segmentation purposes is such a classic from the "Bad Intel Ideas" basket...

That's very surprising, disappointing and extremely short-sighted from Intel.

> (I recall a recent news story about how unawareness of this among people writing or documenting compilers has started to cause problems.)

You can't blame them!

The OS can just turn off any instruction extensions by setting the XCR0 register to whatever it wants. Unsupported instructions will then cause general protection faults (SIGSEGV on Linux). Whether that's what the OS wants to do is a totally separate question. It would be kinda funny if this thing lead to the ability of a process to request a certain XCR0 that the kernel would then serve, because I've been asking for that feature for unrelated reasons for quite some time and didn't get the warmest response to it on the mailing lists.

wait for OpenMultiCore that will standardize heterogeneous cpu sets

This is not ARM; on Intel, the CPUID instruction can also be used by normal unprivileged programs. Not all programs look only at /proc/cpuinfo or the ELF auxiliary vector.

> The only complication here would be if they have differing extensions like AVX512

Which they do, Atom tops out at SSE4.2.

> Atom tops out at SSE4.2.

SSE itself tops out at 4.2, but Tremont does support the newer SHA extensions.

AVX appears to be the only missing thing, but AVX isn't even standard across Intel's other lines, either. The Pentium line doesn't support AVX either, for example, even though they are using Skylake & newer micro-architectures.

So you currently can't assume AVX support, and you still won't be able to assume AVX support. Why does this matter?

> Why does this matter?

Operating systems move threads between cores. If those cores support different features, threads that are migrated to low-feature cores might experience illegal instruction traps despite correctly checking for instruction features.

Maybe standard application code all runs on the big core, and the little cores are like the system assist processors on IBM mainframes - used to run OS jobs or specific application support code, to free up the main processor for other work (or idling).

For this to work, there would need to be a small set of undemanding tasks that account for a lot of machine time. Feeding video to the GPU? All sorts of GUI compositing and housekeeping? Handling network connections in the browser?

I don't think this is a good explanation, but it's fun to think about.

A lot of things are background tasks until suddenly, without warning, the user ends up waiting for them to happen. Take your example of handling network connections, for example: this can definitely be a background thing! Your computer might want to keep an IMAP connection open, periodically poll a CalDAV server, sync photos with your phone, etc., and all of these would be very reasonable things to run on low-power CPU cores. Kick them over to a wimpy core, throttle down the frequency to its most power-efficient setting, insert big scheduler delays to coalesce timer wake-ups, whatever. Good stuff.

But what happens when the user opens up a photo viewer app and suddenly wants those photos to be synced right now?

If your code is running on a recent iPhone -- the heterogeneous-core platform I'm most familiar with -- then the answer is that the kernel will immediately detect the priority inversion when a foreground process does an IPC syscall, bump up the priority of the no-longer-background process, probably migrate it to the fastest core available, and make it run ASAP. Then, once the process no longer has foreground work to do, it can go back to more power-efficient scheduling.

This kind of pattern is super common, and it would be way more annoying and perilous to try to split tasks into always-foreground and always-background.

The standard operating system model today is mostly to do what the application(s) request and then get out of the way quickly. I'd expect to cede all cores to applications most of the time, rather than reserving the low power cores for OS tasks.

That would be pretty disappointing if they can't handle normal processes too. It's easy to end up with a bunch of processes that are using small amounts of CPU but aren't dedicated background tasks.

Even just looking at a browser, I might have half a dozen generic tab processes open, each using a small amount of CPU. But then I navigate one to a game, and I want that particular tab to get near-exclusive access to the big core while the others use only the little cores.

Audio is a big one. You usually want to peg a core, for latency, but it doesn't have to be a particularly fast core.

They can easily disable the differing extensions in the BIG core.

Won't be BIG anymore if it's gimped, will it?

It will. The performance different between an Intel Atom and a Core i3 has basically nothing to do with differing instruction set extensions. AVX2 support is not why a Core i3 runs circles around an Atom, particularly since the overwhelming majority of instructions executed are not AVX anyway.

I'm pretty sure my computer does a lot of AVX while it matches virtual backgrounds to my video feed, or while it tries to figure out voice commands.

All that can be done with SSE and probably with x87 instructions, but, still, software will try to pick the best option.

Binaries need to target a minimum subset, but can use run time feature detection to query differences.

If a binary does that, the OS can just not migrate that binary across cores.

How would the OS know?

The only mechanism I can come up with is to detect illegal instruction traps on small cores, then flag the thread as big-core-only and re-start the execution at the bad instruction. That's not ideal but maybe workable.

(If it traps on the big core, too, it's just a bad instruction and SIGILL is raised to userspace like usual.)

I think the obvious solution is that AVX (or any other extension not in Tremont) will not be supported at all on these CPUs. These are 7W chips, after all.

I assume OS schedulers are going to have to include instruction-set support when seeking to maintain core affinity.

Reply on news.ycombinator.com