591 karma · joined March 8, 2017
- It’s not just ‘old’ FPUs that handle them verrry slowly. Benchmark this aspect of your target processors if this detail matters.
- Most modern processors provide a “flush to zero” denormal handling mode (and sometimes you can separately specify “denormals are zero”, relevant when e.g. loading old data). However, math libraries usually haven’t been written with this mode in mind, so you need to be careful with this.
https://pkg.go.dev/github.com/grailbio/base/simd has some work I’ve done in this vein.
If the exponent is 0, then you have the sum 1/2 + 1/4 + 1/8 + 1/16 + 1/32 + ..., from Zeno's most famous paradox (https://en.wikipedia.org/wiki/Zeno%27s_paradoxes ). If you are fortunate, you previously learned that this converges to 1, and played around with this enough in your head to have a solid understanding of why. If you are less fortunate, I recommend pausing to digest this result.
Then, if the exponent is 1, you have the sum 1/2 + 2/4 + 3/8 + 4/16 + 5/32 + ... .
What happens if we subtract (1/2 + 1/4 + 1/8 + 1/16 + 1/32 + ...) from it? We have (1/4 + 2/8 + 3/16 + 4/32 + ...) left over.
Then, if we subtract (1/4 + 1/8 + 1/16 + 1/32 + ...) from the latter, we still have (1/8 + 2/16 + 3/32 + ...) left over.
Then, if we subtract (1/8 + 1/16 + 1/32 + ...) from the latter, we still have (1/16 + 2/32 + ...) left over.
Continuing in this fashion, we end up subtracting off
(1/2 + 1/4 + 1/8 + 1/16 + 1/32 + ...) + (1/4 + 1/8 + 1/16 + 1/32 + ...) + (1/8 + 1/16 + 1/32 + ...) + (1/16 + 1/32 + ...) + (1/32 + ...) + ...
and this converges to the main sum. And, from the exponent-0 result, we know this is just 1 + 1/2 + 1/4 + 1/8 + 1/16 + ...
See also https://en.wikipedia.org/wiki/Network_effect .
It is not clear how long humans will retain control over AIs.
The tricky part is choosing in a way that puts you noticeably ahead of the unidirectional allocator re: what problems you can solve, without putting excessive mental load on yourself. I've found a pattern of "long-lived allocations on one end, short-lived allocations on the other" to work well here (which, yes, doesn't always coincide with the numerous vs. infrequent axis mentioned in my previous comment).
Given a choice, the OP implies that you should position small-but-numerous allocations next to the top, and larger infrequent allocations next to the bottom.
The hard part is recognizing them in the midst of a lot of chaff.