Great write up! Even though I'm very far from understanding deep learning and the various frameworks you're testing here, benchmarking GPUs has always been a muse to me.
I'm interested to see the utilization of the underlying GPU devices when you run the MLP or CNN benchmarks (monitored with `nvidia-smi`) — the speed-up factor between the different benchmarks don't seem to be inline with the speed-up factor shown in cuDNN link[1] between a K80 and P100. I'm wondering if the P100 device is under-utilized when used with TensorFlow or CNTK.