There are a lot of innovations recently, not just the computing power -- layer by layer unsupervised training, batch normalization, ReLU, dropouts, momentum, gated memory unit, neural language modeling, encoder-decoder just to name a few.
None of it involves the word "deep", it's all just bog standard 1980 style nets with tweaks.