Where is this useful? The bus speed across to the GPU is so slow, I thought it was only meaningful for near-autonomous operations.
However, with AMD and Intel integrated GPUs, this is about to change. AMD is doing a lot of work on HSA, which can be summarized as "GPU and CPU share same memory, and can communicate by passing pointers". I can see this kind of work being really useful in the near future.
Although in that particular context (security) you might enjoy the isolation of dedicated hardware as opposed to sharing it with others. The GPU solution though of course has the advantage of being able to adapt to new ciphers etc.
If you're attacking, having flexibility is advantageous.
So yes, crypto accelerated hardware is behind and does not support the ciphers that the crypto community (academia/industry) endorses and in all likelihood will never bother catch up since doing it on the GPU will be good enough. Even if it takes another decade.
The AES-NI instruction set was proposed in 2008 and the first intel cpus started shipping almost three years ago.[1] Soekris has had the vpnXXXX crypto accelerators since as long as I can remember.[2]
[1] http://ark.intel.com/search/advanced/?s=t&AESTech=true [2] http://soekris.com/
The fact that it was proposed in 2008 is quite telling by itself. And when intel introduced it it was in their high-end product lines, to find a processor where AES-NI is less needed would be a challenge.
A suitable integrated GPU would penetrate the market much better and ultimately support products which today use the atom processor. The very same product segment where you barely can use encryption on today (in contrast to a i7 which saturates a fast ssd without breaking a sweat in AES-encryption throughput - without hardware acceleration).
A similar solution would also most likely allow me to encrypt files on my phone without a large impact, even if the manufacturer couldn't care less about security features.