Writing backend agnostic code (CPU and GPU) jitted to fast LLVM is on the roadmap for python's Numba: https://github.com/numba/numba/issues/545

Currently supports both including multithreading. Its just he unified interface that is in the works.