One of the reason CPU utilization is so low is that this is a demo. Most of the "logic" is just a straightforward computation of the mouse position, and the particle coordinates are all figured out from that in the shader engines. Real apps and games have real data that needs to be crunched.
(Edit: I just checked the source, the particle positions are actually computed in Javascript, the shaders are just straight rendering. And I don't see anything particularly clever about the implementation, it's just a bunch of array accesses. V8 is doing an amazing job on this.)
But broadly, you're right: the need for highly optimized native code to drive a modern GPU has mostly disappeared. A javascript interpreter (well, V8) is more than good enough.