in fact looks like you can use tensorflow models in spark with GPU - https://databricks.com/blog/2016/12/21/deep-learning-on-data...
https://github.com/databricks/tensorframes
Spark is just a data access layer here. It's not even remotely gpu friendly. Most people also still relies on mesos or yarn for running distributed. The library you're using matters alot. Mesos just added gpu support: http://mesos.apache.org/documentation/latest/gpu-support/
Yarn can sorta support it with node labeling for job completion but it's still kinda hacky.
The real work in this space (without the marketing) is done by IBM: http://www.slideshare.net/ishizaki/exploiting-gpus-in-spark
When spark can (without "production ready" buzzwords) run gpus like this out of the box then we're talking. For now spark needs a companion library to work with gpus though.