An interesting piece of hardware that I have seen was used in ray tracing where high speed reduced precision FP was used to find polygons that were not visible, those where the low precision hid a change and those that were otherwise visible Conventional high precision processing was needed only for those with changes. The overall performance improvement was 100x.

We are probably close to hitting the wall for general purpose solutions on general purpose hardware, but the future is wide open for application specific hardware.