Hacker News Reader

Comment by nzach | Hacker News Reader

I went in expecting to find 'branch prediction'[0] as the answer, but apparently things are even more complex nowadays.

[0] - https://stackoverflow.com/questions/11227809/why-is-conditio...

I don't think GPUs ever had branch prediction in the first place. You can however run into poor performance due to thread divergence, which is a similar kind of issue (with much less black magic).

>I went in expecting to find 'branch prediction'[0]

GPUs do branch prediction? I thought they didn't bother and try to minimize wasted effort by using high amounts of concurrent threads?

They do texture prefetching, which is sorta similar.

To be fair, the culprit in the article is _less complex_ than branch prediction: "with random data, bits are flipped often, and bit flips in transistors inherently draw power" is less mental gymnastics than "with random data, the cpu fails to predict the future, causing redundant speculative execution"

But why do we expect random data to result in more bit flips? That seems harder to argue than the mechanics of a basic branch prediction system.

Think about it from the other end. Why would any bits flip at all in the data path of your matrix multiplier when all the matrices are 0?

Sure, when comparing 0’s to anything else. But what about normal distribution to uniform in 0,1? The author hand waves something about signs but it’s not very well reasoned - that’s just a single bit in floats.

And what of the Pi test - I’d expect that to flip many more bits than the 1-bit one.

If the inputs are constant, then all the multiplies are constant and the only thing that toggles is the accumulation. Which explains the pi situation.

Normal vs uniform is less clear, but also not as much of a difference. The arguments about signs isn't just about a signs bit, though. The way you negate during accumulation is that you flip all the bits. Only the final float representation is sign+magnitude, the accumulation itself has two's complement steps. I don't actually know the analysis here, just pointing out that it's not that simple.

Anywhere I can read more about this float accumulation with 2’s complement?

Field effect transistors are basically a capacitor. They store energy.

If you switch a not gate's input from zero to one to zero and so on, the gate capacitance will have to charge and discharge. The entire idea behind CMOS is that if you have n and p channel transistors together, you can take advantage of the fact that electrons are more mobile than holes. Filling and draining electrons gives you a greater switching speed.

If the input stays the same, then the charge at the input inside the flip flop is the same as the charge inside the not gate. No charge differential means no electrons move, which means there is no ohmic resistance that causes the internal metal and polysilicon interconnect to heat up and less power gets lost and no switching obviously happens faster than some switching.

TL;DR If you randomize the data, you will constantly charge and discharge the capacitors.

Good point but this forum leans heavily towards software, so we are used to the latter! I have worked close to IC development so had to learn and tangle with the former idea too, was interesting.

I expected a “torch is smart enough to keep track of cases where it just initialized the C in C <= A*B+C to zero, avoiding the add” type situation but I was wrong.

That's exactly what I thought.

Reply on news.ycombinator.com