Why bother? The precision of double is sufficient for the vast majority of engineering problems. Seldom do you have measurements that are more precice than the calculations you do with them.
But anything which uses matrices (aka. systems of linear differential equations), always runs into some pathological cases. Quite often, they aren't even uncommon.
Systems which have vastly different time constants in the same system are ALWAYS a problem. Simulating RF circuits or phase locked loops is always an issue.
Computational fluid dynamics always has boundary conditions that challenge infinity/zero and drive the solver crazy.
Something as pedestrian as simulating basic cotton cloth accurately is problematic (the warp/weft redistributes energy on a much faster time scale than gravity pulls the cloth down).
A lot of people did a lot of research and put in a lot of programming hours to make solving these problems relatively easy without understanding the gory details.
I'm skeptical that changing how we represent the numbers will help - when the physics are icky, they're icky.
More problematic is that when a matrix is ill-conditioned it may not be all that helpful to have "an answer" since the ill-conditioning tells you there are many solutions closely satisfying the system of equations.
I've implemented a few DE solvers and sparse matrix implementations before. I think sparse matrices is one case where it would help.
Also, some floating point ops that use pade approximants (most log algorithms, sin, cos) lose onoe or two precision points (versus half for arithmetic ops) of floating point accuracy.
If you're using IEEE floating points, your systems will silently fail and (rarely) give an error like NaN. IEEE intervals will work, but often give a diagnostically useless answer. Unums have characteristic signatures that hint where to look to find the problematic calculation. You can then backtrack and redo the calculation at a higher precision and drop back to the standard precision as necessary. This is, of course, automatable.
Mostly false; see https://www.cs.berkeley.edu/~wkahan/Mindless.pdf
It is true that an arbitrary, independent sequence of n floating point operations can potentially lose up to 2n bits of precision. However, it is false that most scientific computing code has such structure. On the contrary, the loss of precision in numerical algorithms can be estimated quite well using standard techniques of error analysis, especially when there is underlying matrix structure to exploit. The growth of error in properly written numerical code should NOT grow exponentially with the number of floating point operations. It should be bounded by intrinsic properties of the underlying matrix, such as its condition number or spectral norm.
[1] http://www.johngustafson.net/presentations/Multicore2016-JLG... (see example starting p.10)
In statistics numbers often get very small because they represent probabilities. Sometimes they transform into the log domain, but that has a lot of disadvantages. You lose zero, negative numbers, addition is difficult, and you get vastly more precision near zero than near one.