simonbyrne

If that's seriously what you're looking for, you should really consider Fortran. There is a reason it is still very widely used in scientific domains.

You can just wrap the literal in a conversion function, eg Float32(0x1p52), which should get constant propagated at compile time.

That's fascinating thread, thanks: https://github.com/WebAssembly/design/issues/148

Do you know what happens when you have ops with different flags? e.g. if you have (a + b) + c, where one + allows reassoc but one doesn't?

From personal experience, yes: I've seen multiple cases of scientists finding the ultimate cause of their bugs was some fast-math-related optimization.

The problem isn't necessarily the code they wrote themselves: it is often that they've compiled someone else's code or an open source library with fast-math, which broke some internal piece.

I tried to lay out a reasonable path: incrementally test accuracy and performance, and only enable the necessary optimizations to get the desired performance. Good tests will catch the obvious catastrophic cases, but some will inevitably be weird edge cases.

As always, the devil is in the details: you typically can't check exact equality, as e.g. reassociating arithmetic can give slightly different (but not necessarily worse) results. So the challenge is coming up with appropriate measure of determining whether something is wrong.

I'm not exactly sure what you're asking here, but the point is that "to machine precision" is relative: if f(x) and g(x) are O(1e200), then the absolute error of each is still O(1e185). However f(x)/g(x) will still be very accurate (with absolute error O(1e-15)).

The key thing about floating point is that it maintains relative accuracy: in your case, if you have say f(x) and g(x) are both O(1e200), and are correct to some small relative tolerance, say 1e-10 (that is, the absolute error is 1e190). Then the relative for f(x)/g(x) stays nicely bounded to about 2e-10.

However if you do f(x) - g(x), the absolute error is on the order of 2e190: if f(x) - g(x) is small, then now the relative error can be huge (this is known as catastrophic cancellation).

Herbie is a great tool, especially for teaching.

In theory, every function should do that to check things like rounding mode etc. But that would be pretty slow, especially for low-latency operations (modifying mxcsr will disrupt pipelining for example).

-ffp-contract=fast will enable FMA contraction, i.e. replacing a * b + c with fma(a,b,c). This is generally okay, but there are a few cases where it can cause problems: the canonical example is computing an expression of the form:

a * d - b * c

If a == b and c == d (and all are finite), then this should give 0 (which is true for strict IEEE 754 math), but if you replace it with an fma then you can get either a positive or negative value, depending on the order in which it was contracted. Issues like this pop up in complex multiplication, or applying the quadratic formula.

My point isn't that fast-math isn't useful: it very much is. The problem is that it is a whole grab bag of things that can do very dangerous things. Rather than using a sledgehammer, you should try to be selective and enable only the useful optimizations, e.g. you could just enable -ffp-contract=fast and -fno-math-errno.

Not necessarily: if your cospi(x) function is always returning 1.0 (https://github.com/JuliaLang/julia/issues/30073#issuecomment...), but you wrote your code assuming the result was in a different interval, then you could quite easily invoke undefined behavior.

> But then people latched on to the idea that it was caused by some kind of sonic device or hitherto unknown weapon targeting embassies.

Possibly prompted by historical precedent: https://en.wikipedia.org/wiki/Moscow_Signal

One of the interesting things about the 80-bit format was that allowed the use of 64-bit integers on a 16 or 32-bit machine. This was what Thomas Nicely was using them for when he found the Pentium FDIV bug: https://faculty.lynchburg.edu/~nicely/pentbug/pentbug.html

> or wait for optimizations to be done, e.g have pandas, numpy etc handle multi-core processors etc ?

All those exist already. Indeed, other than DataFrames.jl (the pandas equivalent) they are part of the language itself.

I disagree about practical and accessible:

- its exposition is complicated (trying to prove everything in a general base makes it difficult to understand)

- it's woefully out of date (lack of guard digits haven't been an issue for at least 25 years, extended precision hasn't been an issue for the past 10 or so, and most languages now default to having fairly strict floating point semantics)

- it gets bogged down in irrelevant minutiae (rounding modes and exception flags, while available in modern hardware, aren't really supported by any modern languages/compilers)

- it doesn't really provide any practical advice (it barely mentions binary-decimal conversion, it jumps to doubling precision and Kahan summation without suggesting any intermediate steps such as sorted or pairwise summation).

But my biggest complaint is the frequency with which users are referred to it on StackOverflow as if (1) it is a good way to learn about floating point concepts, and (2) anyone using floating point numbers should be expected to understand it all.

Deep learning is the obvious answer: 10 years ago it was largely confined to a small number of research groups and niche conferences.

Thank you! The Goldberg article is a terrible way to learn about floating point, and the frequency with which it is referred to on StackOverflow is really disheartening.

I seem to recall that part of the DoE's selection criteria is to ensure a competitive market for future contracts (since they will always be buying more supercomputers). Given the recent dominance of Nvidia in the market (see https://www.top500.org/lists/2019/06/), it probably made sense for them to ensure some contracts went to other suppliers (AMD for Frontier, Intel for Aurora) to ensure that Nvidia doesn't establish a monopoly on future tech.

Additionally, these sorts of supercomputers are also a way for governments to implicitly subsidise their tech industries: when viewed through that lens, spreading these contracts around makes a lot more sense.

Additionally the language and penalties for fare evasion often don't match that of say parking tickets, which I would put as roughly similar in terms of moral violations (i.e. taking advantage of a common good, but not endangering the safety of others, like say speeding).

Not to mention a Cray 1 used 115kW: the new raspberry pi uses 7W peak.

It's difficult to examine his comments without historical context: I haven't dealt with any code from that era, but there is still plenty of Fortran 77 code from ~20 years later where the use of gotos makes it difficult to understand control flow, e.g.: https://github.com/JuliaMath/openspecfun/blob/master/amos/zb...

That said, gotos are still occasionally very useful: jumping out of nested loops and implementing finite-state machines are both made much easier and clearer via gotos than trying to force them into more standard control flow statements.

The linux kernel also makes judicious use of gotos for cleaning up error handling: https://koblents.com/Ches/Links/Month-Mar-2013/20-Using-Goto...

Am I missing something? Other than the doc generator (which is in Python), none of the repos have been touched for a year or so.

Also, reminded me of the old CB1 cafe, which labelled itself "UK's oldest internet cafe", which amusingly hadn't updated their website since 2005: http://www.cb1.com/

I used to live around the corner: I always wondered about that.

I do miss Norfolk Street Bakery though (which I hope is still going).

Interesting: I've heard anecdotes about similar experiences in southern California, where the "good" school districts often don't allow apartments or smaller subdivided buildings (duplexes/triplexes) to be built.

Obviously that question depends on your motivation, but one good reason is that it is a great way to learn how software actually works.

R makes it very easy to see the underlying R code (you just type the function name), until you get to a ".Call" or ".Primitive": from that point, it is effectively a black box.

But as most of Julia is written in Julia, you can easily inspect and understand how functions work, all the way down. Moreover, by using the @code_* macros, you can also inspect the various stages by which the code is transformed from high level Julia code down to the actual machine code which is running on your computer.

> - You can't use a function before it's defined.

I'm not sure what you mean by this. You obviously can't call a function before it is defined, but you can use a function in another function without any problems:

    julia> foo(x) = bar(x)
    foo (generic function with 1 method)
    
    julia> bar(x) = x+2
    bar (generic function with 1 method)
    
    julia> foo(3)
    5

Why? China's per capita emissions are less than half of the US.