I wrote my master thesis on General purpose GPU programming. The only two reasons you want to code for the GPU is that it can be much faster for some workloads (basically identical computation which can be trivially parralized) and (in some cases) because it can use much less power.

In the particular case of matrix multiplications, lots of problems can be described as matrices (say googles website ranking) and doing calculations on huge matrices is a thing the GPU is great at. That said it is unlikely to be something most people want.

Whenever you need to implement an algorithm that is described using matrices it is much easier and faster to implement it by calling linear algebra functions instead of coding loops yourself. Neanderthal comes into picture when you have lots of numbers and still need it to be fast (nano, micro, milli)seconds instead of minutes and hours (or perhaps days). So, actually any kind of numerical software written in Clojure (and Java) can hugely benefit from this.