Hacker News Reader

Comment by klelatti | Hacker News Reader

Not had chance to look at this in detail but this seems more likely to be an artefact of the rounding in FMADD vs separate instructions rather than a 'flaw' in the instruction.

There seems to have been a similar issue in Chromium a few years ago.

https://bugs.chromium.org/p/chromium/issues/detail?id=401117

Where the conclusion was:

"Having said that, fmadd is doing exactly what it should do; it isn't broken. The model that gave you a result of 0x4 _is_ broken, but the hardware isn't."

View on HN

I think this is the key. My understanding is the FMADD on arm64 does what the spec says it should do, so there's nothing broken.

Next up though is whether fused multiply add is part of the IEEE floating point standards, and whether the arm implementation meets these. Chances are it does (I see references to it from 2008). If the x64 FMADD produces difference results, it's likely that either the standard is wooly in some respects and allows wiggle room (so they both meet the standard), or, the Intel/AMD implementation pre-dates the standard and isn't compliant.

I'm sure someone who knows these standards and processors better than me will be along in a minute to comment.

From IEEE 754-2019:

> The operation fusedMultiplyAdd(x, y, z) computes (x × y) + z as if with unbounded range and precision, rounding only once to the destination format. [...] fusedMultiplyAdd differs from a multiplication operation followed by an addition operation.

[elided exception discussion]

I expect that the ARM processor precisely implements this standard, as does x86-64. Any "bugs" here are the user misunderstanding how the precision is supposed to work.

The standard mandates that the result of fma is rounded as-if an infinite precision result is computed internally followed by a single correct rounding. There isn't any wiggle room.

For finite inputs, the results are unambiguously specified by the IEEE spec. For singularities involving zero/infinite inputs, the IEEE spec allows two sets of behavior, which basically correspond to x86 and everyone else.

I wouldn't be surprised if the difference in rounding is an innate property of how the math behind an FMA instruction works and that guaranteeing the same rounding (which is essentially rounding twice, once after mul and once after add) heavily impacts the performance benefits. Thus implementing FMA with an inconsistent rounding to separate mul and add instruction is in the interest of performance.

I'm just guessing here tho.

Reply on news.ycombinator.com