Comment by PragmaticPulp | Hacker News Reader

Comment by PragmaticPulp | Hacker News Reader

Parent Full thread

> 2. You “tolerate” this practice just long enough to win the critical initial wave of reviews and benchmarks. Interested buyers will look at those charts for years to come.

That’s not what happened here.

The AVX-512 instructions not only weren’t enabled by default, they couldn’t be enabled at all unless you went out of your way to disable the efficiency cores completely. They also wouldn’t benefit your workload unless disabling those extra cores was offset by the AVX-512 instructions on the remaining cores.

None of the benchmarks you saw in reviews or marketing material would have used these instructions unless specifically called out by the reviewers as having made all of these changes.

Benchmarks like the multi-core Geekbench would actually go down, not up, with this enabled because you’re giving up cores. Thermal performance would be worse because the efficiency cores were disabled.

Intel never marketed the part with AVX-512. It was discovered by a reviewer poking around in the BIOS of a review board.

> motherboard manufacturers can be relied upon to enable it in their “1337 OVERCLOCKZ” mode. No conspiracy is needed.

Nope. You had to disable cores to enable AVX-512 and I doubt it would show up in any gaming or consumer benchmarks as a positive.

The conspiracy theory about Intel doing this to mislead consumers not only doesn’t make sense, it’s completely wrong given how this worked and how it was discovered.

> The AVX-512 instructions not only weren’t enabled by default, they couldn’t be enabled at all unless you went out of your way to disable the efficiency cores completely. They also wouldn’t benefit your workload unless disabling those extra cores was offset by the AVX-512 instructions on the remaining cores.

If your code benefits from AVX-512, it'll probably benefit from turning off the efficiency cores too. Sixteen 256-bit AVX channels are the same as eight 512-bit channels in theoretical throughput. Because there are fewer load/store commands and fewer bunches of setup code to run, overall theoretical efficiency should be higher.

Power efficiency was a killer at 14nm. The cores would downclock into oblivion when AVX-512 executed. Given the node shrink and shutting off half the cores, I don't see why this would happen here. Doing the same calculations with less power means a lot for the kinds of workloads that actually use AVX-512. Sure, idle performance may go down a bit, but once again, if you're running the kind of application that benefits from this, that's also probably not a top consideration either.

The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores. It wouldn't be hard either, just catch the unknown instruction exception and see if it is AVX-512 then move the thread.

> If your code benefits from AVX-512, it'll probably benefit from turning off the efficiency cores too.

Maybe if you have a poorly parallelized problem where the bulk of the code is spent in AVX-512 instructions and you don’t ever do anything else on the computer, but that’s going to be rare for any consumer.

> The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores. It wouldn't be hard either, just catch the unknown instruction exception and see if it is AVX-512 then move the thread.

“It wouldn’t be hard” is a huge understatement. This is OS-level support, not something Intel can just do inside the CPU. And operating system vendors and developers are still catching up to basic Alder Like scheduler support without AVX-512 handing, so it’s not reasonable to suggest this would be easy.

There have been reports that it was possible to enable AVX512 on the big cores with the little ones enabled [0] by toggling some bits in the MSR.

For a first step it would have been enough to manually pin threads to the big cores for adventurous users, but Linux already tacks which threads use AVX (since it comes with a clock penalty, so you want to isolate them if possible), it's not unreasonable to think that auto-setting CPU mask based on that should be possible too.

[0] https://lkml.org/lkml/2021/12/2/1059

I don't see how "poorly parallelized" is relevant compared with, say, a decent OpenMP BLAS which is likely to benefit from any reduced power consumption that allows a faster clock. In general on compute nodes it's worth turning off cores dynamically which aren't being used by jobs, like on a half-full node.

> The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores. It wouldn't be hard either, just catch the unknown instruction exception and see if it is AVX-512 then move the thread.

Having actually worked on this, it’s quite a bit more complicated, although it is possible. For better or for worse, though, Intel made a decision that all cores would expose the same feature set, and now this is baked into Linux and probably other OSes at the ABI level, and changing it would be a mess.

A correct heterogeneous ABI (and frankly a good AVX512 ABI at all) would require explicit opt-in, per process, for AVX512. Opting in would change the signal frame format, the available features, and the set of allowed cores.

Boost your benchmarks with AVX-512 by subscribing to Intel+

What benchmarks? Not a lot of real programs actually use AVX512, and a lot of the ones that did discovered that it made performance worse, so they stopped.

(The issue is that AVX512 (the actual 512-bit parts, not the associated EVEX and masking extensions) may well be excellent for long-running vector-math-heavy usage, but the cost of switching AVX512 on and off is extreme, and using it for things like memcpy() and strcmp() is pretty much always a loss except in silly microbenchmarks.)

To be clear, I don't like the type of product line differentiation that Intel does, and I think Intel should have supported proper heterogenous ISA systems so that AVX512 and related technologies on client systems would make sense, but I don't think any of this is nefarious.

> What benchmarks? Not a lot of real programs actually use AVX512, and a lot of the ones that did discovered that it made performance worse, so they stopped.

This paints a picture of what happened to Intel's fab process rather than AVX-512 itself.

AVX-512 was proposed back in 2013. It was NOT designed for desktops. The original designs were for their Phi chips (basically turning a bunch of x86 cores into a GPU). These Phi chips ran between 1 and 1.5GHz, so power consumption and clocks were always matched up.

Intel wanted to move these instructions into their HPC CPUs. The problem at hand was ultra-high turbo speeds. These speeds work because the heat is a bit spread out on the chip and a lot of pieces are disabled at any given time. With AVX-512, they had 30% of the core going wide open for the vector units (not to mention the added usage from saturating the load/store bandwidth).

They wanted 10nm and then 7nm to fix these issues. At their predicted schedule, 10nm would have launched in 2015 and 7nm in 2017.

Given the introduction of AVX-512 in 2013, they had plenty of time. In fact, the first official product was Knight's landing in 2016. Skylake with AVX-512 didn't launch until 2017 when they were supposed to be on 7nm.

Intel was forced to backport their designs to 14nm++++++++++++ which forced a deal with the devil. They had to downclock the CPU to keep within thermal limits, but this slowed down EVERYTHING. Maybe they could have created a separate power plane for AVX, but this would be a BIG change (and probably politically infeasible).

What happens with the downclocking? If you run dedicated AVX-512 loads, then maybe you should have been looking at Phi instead. If not, mixed loads suffered overall because of the lower clockspeeds.

Their second revision of 10nm superfin is still a generation larger (well, probably a half-generation) than what they anticipated. There might still be downclocking, but I'd guess that it's nowhere near what previous iterations required.

TL;DR -- AVX-512 was launched two nodes too early which screwed over performance. It should become acceptable either with 10nm Superfin or 7nm when it launches so the CPU doesn't have to downclock constantly.

It's worth saying that in my life as and academic, the only program I've ever used that has commonly benefited from avx-512 in my work is Gromacs. Gromacs is a molecular dynamics program that creates beautifully graphical simulations and often appears as-is in benchmarks (the so-called ns-per-day metric). Although almost indescribably complex it's also fundamentally strangely straightforward in what it does, and avx-512 does indeed make it significantly faster.

The overwhelming bulk of my research does not use Gromacs, however. Mkl, matrix algebra and similar hybrid MPI tasks, yes, but oddly there the extensions really don't seem to do much, and frankly the inferior memory architecture of the Xeon platinum nodes we use makes itself apparent. I frequently get annoyed that Intel charge you a fortune for CPUs into which you can put a ton of ram, and then marketing makes the CPU's exclusive feature an instruction that doesn't seem to help that much in my actual workloads.

They really should just shunt it to some specialised product and use the die space for something else. I'm with Linus here.

I think the technology will be there to do the mixed instruction set stuff in 5 years, but for now I understand why Intel have played it safe.

> The real solution would be for Intel to detect the presence of AVX-512 instructions then automatically and unconditionally pin the thread to the big cores.

Wouldn't it be entirely up to the OS to decide on what core(s) a thread will run?

Theoretically, the E core can raise a #UD (undefined instruction) exception that the OS would trap (which already happens if the CPU doesn’t have AVX-512). The OS could then detect that the offending instruction is from the AVX-512 set, and move it to a P core. It could even set a flag on the thread to force it to use P cores.

Intel even touted the work they did with Microsoft about the scheduler to deal with selecting which E or P core to use.

I’m not sure why they chose to disable AVX-512 instead of doing that. Maybe that is their end goal, but it’s too much effort to have it done by release.

Comparing the facts to all the suspicious people here, it made me realize something. Our zeitgeist sure is a cynical one.

Humans seem prone to find meaning and conspiracy amidst chaos and confusing circumstances.

There's a saying I like - Humans are deterministic machines in a probabilistic world.

Intel has spent decades accumulating its reputation. Sometimes cynicism is appropriate.

Let's see: Intel

- ME https://news.ycombinator.com/item?id=21534199

- Anti Trust: https://www.networkworld.com/article/2239461/intel-and-antit...

- The cripple AMD function in their compiler: https://www.agner.org/optimize/blog/read.php?i=49

I could keep this up quite a bit longer if you want. Intel well deserves any skepticism and cynicism it is targeted with.

> I could keep this up quite a bit longer if you want. Intel well deserves any skepticism and cynicism it is targeted with.

Intel deserves skepticism and cynicism based on what is true.

All companies deserve skepticism, to be clear. AMD have their management engine too. As far as I'm concerned the only reason why AMD's ME isn't as developed as Intel's is because their products have been such a joke leading up to Zen that no one with men in suits bothered asking them for the capabilites, or something like that.

Don't forget the Intel Payola scheme: https://www.theatlantic.com/technology/archive/2010/07/dells...

Reply on news.ycombinator.com