I don't know about tensor flow in particular but are little-known methods of running "general purpose" parallel programs on GPUs. Specifically, H. Dietz' MOG, "Mimd on GPU". It's a shame the project hasn't gotten more attention imo.
See: https://en.wikipedia.org/wiki/Flynn%27s_taxonomy for explanations of terms.