I've spent some time training tensorflow cnns on nvidia 1080gtx.

It works pretty well, but i couldn't reliably train production models for work on it. it's just too flaky. I mentioned this to our 'trustworthy' dell rep who suggested that a good v100 suite would surely solve all the reliability problems, what with it's greatly increased memory bus, or something...