Amazon has GPU powered instances [1]

Python has pycuda [2]

Python has OpenCL [3] amusing ran by the same person as pycuda

[1] http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using_clu...

[2] http://documen.tician.de/pycuda/

[3] http://documen.tician.de/pyopencl/

The gx2.2 instances will run you $0.650 per hour of compute time and have K520 (?) in them, which works pretty well. There are a few preconfigured instances, but if you want to make your own and go through the hassle, it will take about two hours to set up. When you're spinning up your instance, do a search for CUDA and you'll see a few Ubuntu preconfigured images with Theano+PyLearn2 already installed. Spawn them on the aforementioned GPU instance and you're good to go.

Theano supports CUDA:

http://deeplearning.net/software/theano/

http://deeplearning.net/software/theano/tutorial/using_gpu.h...