Nvidia Introduces CuDNN, a CUDA-based Library for Deep Neural Networks

In general, the question which is the fastest public implementation of spatial convolution is so heavily dependent on the kernel parameters (and even then, there are often knobs you can twiddle for a given implementation that can lead to substantial speedups), how much GPU memory you want to spend (GEMM and FFT based methods), so the best approach is probably just to benchmark a bunch of implementations for your network and just choose the best.

Then again these figures are also kind of meaningless, as the specifics of the experiments were not specified. There are a lot of parameters that affect the performance of different implementations in various ways: the number of input feature maps, the number of filters, filter width/height, input width/height. The FFT-approach tends to do well for large filter sizes and lots of input feature maps, for example.

I've played around with cudnn a bit using Theano. The Theano bindings are a work in progress, but so far they've already wrapped the convolution. Compared to conv_gemm (Theano's version of Caffe's GEMM convolution approach), it seems to be sometimes faster, sometimes slower. Soumith Chintala maintains a GitHub repository with benchmarks of various convolution implementations: https://github.com/soumith/convnet-benchmarks His results aren't that spectacular either.

In my own experiments cuDNN did pretty well for very small filter sizes (i.e. 3x3), often beating even the memory-hungry FFT approach). This is great because the top two scoring entries in the 2014 ImageNet competition made use of lots of convolutional layers with small filters.

Of course the main benefit of cuDNN is that it will become faster over time, and that it will always be adapted to the latest NVIDIA GPUs without requiring code changes (provided that they keep maintaining it).