What are the biggest challenges you meet in terms of missing software infrastructure to support your NLP/ML work? Things you wish someone would implement?
Having such a runtime available as a standard lib packaged in all linux distribs would make it interesting for lowlevel vector math libs such blas / lapack / atlas and convex optimization solvers to have an implementation based on OpenCL kernels that would work (almost) as fast as the currently manually optimized C + simd intrinsics code but could also 10 to 100 as fast whenever a GPU is available on the machine without having to re-compile anything.
Some advanced and very promising machine learning algorithms (e.g. algorithms of the deep learning family) can really benefit from the computing power of vector processors such as the GPUs.
Right now everybody who wants to be able to gain the perf boost of GPUs uses CUDA but: - it's only useful on NVidia GPU machines (e.g. not on Amazon EC2 machines for instance) - it's not opensource hence has to be manually installed on every single machine you want to use it (no apt-get install from the official repos). This which makes it a heavy dependency for your pet machine learning library: be prepared to support CUDA installation problems on your project mailing list.
http://deeplearning.net/software/theano/
Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. Theano features:
- tight integration with numpy
- transparent use of a GPU
- symbolic differentiation
- speed and stability optimizations
- dynamic C code generation
- extensive unit-testing and self-verification