https://www.youtube.com/watch?v=HMUWzDSAS9c
A very interesting watch if you're interested in electronics in general and in power delivery in particular (the whole YouTube channel is awesome to be honest).
Do they just not believe in compiler authors to be able to emit these kinds of hints? Is there too much legacy code that would come without the hints for it to be worthwhile? Would it just not be worth it given how often OS context switches could drop the CPU directly from regular code in one process to AVX2 code in another process?
> I observed an interesting phenomenon when executing 256-bit vector instructions on the Skylake. There is a warm-up period of approximately 14 µs before it can execute 256-bit vector instructions at full speed. Apparently, the upper 128-bit half of the execution units and data buses is turned off in order to save power when it is not used. As soon as the processor sees a 256-bit instruction it starts to power up the upper half. It can still execute 256-bit instructions during the warm-up period, but it does so by using the lower 128-bit units twice for every 256-bit vector. The result is that the throughput for 256-bit vectors is 4-5 times slower during this warm-up period. If you know in advance that you will need to use 256-bit instructions soon, then you can start the warm-up process by placing a dummy 256-bit instruction at a strategic place in the code. My measurements showed that the upper half of the units is shut down again after 675 µs of inactivity.
https://www.agner.org/optimize/blog/read.php?i=628
Agner also notes that previous generations would just flatly stall all instructions (including non-vector) once they saw 256-bit instructions, which is probably why there is no "warmup" instruction. If there is an explicit stall of all instructions then there are very few reasons to incur that until the last possible second.
But, if such "hint" ops were seen as the only viable way to get correct, high-performance behavior in these cases, and there was a commercial case to be made for it, I could see processor vendors adding them to the IA.
Another approach I haven’t seen from either Intel or AMD yet, is to copy ARM’s big.LITTLE architecture: to set up separate “AVX2 cores” that can execute AVX instructions and most basic ALU ops but not e.g. branches, put those on the other side of the die with in-wafer thermal insulation between them and the regular cores; and then throw workloads between regular and AVX2 cores in a way where the AVX2 cores heating up doesn’t mean that the non-AVX2 cores are heating up, and the CPU can go back to full Turbo Boost as soon as the workload is thrown back to the regular cores, because the respective regular core is actually quite cool.
(The performance effects of this are possible to loosely estimate, I think, by writing code that synchronously sets up a GPGPU pass on some data, executes it, retrieves the result, and then returns to executing CPU instructions for a while, in a loop; and then executing this code on an Intel CPU using its on-die IGPU as the GPU. The CPU and IGPU form somewhat-separate thermal domains already, though there’s no explicit insulation.)
Sorry to be pedantic, but... physics requires a degree of pedantry.
1. This is probably an ENERGY issue, not a power issue. Energy is the integral of power (or power is the derivative of energy: power is the change of energy over time).
2. Electronics can only really measure voltage easily. Everything else is converted to voltage to be measured. For example, to measure current, you create a resistor and then measure the voltage across the resistor. To measure power, you need to measure voltage, and multiply it with current. To measure energy, you take the integral of your power measurement. (No joke: that's how it typically works).
3. With a capacitor, you can measure your energy level. Capacitors are relatively inaccurate however (+/- 10% tolerances at the electronics level, and probably worse tolerances at the nanometer chipscale level). The voltage from a capacitor does roughly correlate to its energy level.
-----------
Issue #1: Capacitors suck at energy storage, but they're all that engineers have at these speed and sizes.
#2: Voltage is the only thing that can be measured reliably, but it takes a long time (more than a few nanoseconds, aka 10s or 100s of clock cycles) to measure voltage. Measuring other variables, such as "Energy" (voltage level of a capacitor), or "Power" (change of energy over time, or voltage * current), takes even longer to measure.
#3: Computers are fast. 4GHz computer means that you have 0.25 nanoseconds to make a decision per clock tick. Waiting on the results of any calculation on #2 will naturally take dozens or hundreds of clock ticks to accomplish.
#4: That's best case scenario, assuming that capacitors are large enough to actually hold the energy you need to perform your calculations. But in an AVX2 situation, where your CPU suddenly will use up more Power (energy/time) than expected, you need to communicate to your power-circuitry to increase the voltage or current delivery to the chip.
#5: Sending commands to the power-circuits / VRMs, or worse case to the Power Supply, can take many milliseconds (millions of nanoseconds) before the Power Supply responds. A 200W chip at 1.25V will draw 160 Amps of current (0.160 coulomb of electrons per millisecond). A 1 Farad capacitor will lose 0.16 volts per 0.16 coulombs of electrons drawn, and hint-hint, you can't physically fit a 1-Farad capacitor in a chip (https://qph.fs.quoracdn.net/main-qimg-dddab4aa574936f8a61437...).
#6: This means that the only reasonable approach with today's technology, is to wait for the VRMs or Power Supply to respond. Note: The Power Supply will naturally notice the voltage drop and automatically send more electrons down to the motherboard. But the time-scale the PSU operates is on the order of milliseconds. A millisecond is 1-million nanoseconds or ~4-million clock cycles AFTER the AVX2 units started getting used. If you run out of energy before the power gets to your chip... guess what? You get a brownout. Random cells in your chip were now turned off and the state of the CPU is a jumbled mess now.
--------
Where, how, and why to place capacitors around a chip is one of the most difficult PCB-issues an electronic engineer deals with. There's a LOT of "rules of thumb", making PCB-design / capacitors a mystic subject matter for many EEs.
Given the difficulty of the subject of power-delivery, energy calculations, and high-speed signals involved (a chip operates at the GHz band, drawing ~100 Amps on the clock-tick, and then 0 Amps between clock ticks!), I'm almost certain that there's a mistake in the capacitors somewhere.
Its a mystic subject that's very difficult to simulate, especially across varying workloads (AVX vs normal 64-bit code, etc. etc.). That's my bet for what's going wrong here.
Honestly, power-delivery is so awesome and complicated. I'm almost surprised that anything works at all these days.
I have my doubts about this being a power issue, because localized power analysis is pretty advanced in IC design and they'd totally have tested for this.
Assuming it is a power issue though, nearly all power is used in dynamic switching, so simply inserting stall cycles will resolve it.
Power rails on silicon chips have quite a lot of capacitance, at least enough for many clock cycles, so you don't even need to make every cycle in every bit of hardware a possible stall cycle - it would be enough to simply gate instruction issue I expect and let functional units drain.
But I wonder if this is a Gigabyte issue who have a history of playing around with the power delivery and using "fake phases" (to the point where they now have to advertise their boards as having 16 "real phases"). As far as I can tell many (most?) reviewers benchmarked the board with 24 core CPUs and most likely skipped on the power intensive tests.