Please see my detailed answer on this question that you can find on quora: https://www.quora.com/Why-are-GPUs-well-suited-to-deep-learn...