Hardware Acceleration/Parallization is the next frontier. We've already seen the benefit of some pretty simple ASICs (TPU was built to be simple) as well as more general purpose accelerators. Hardware architects used to have a hard time, because often the best option was to simply wait for CPUs to get faster. Now that we've seen CPU power begin to stall it makes economic sense not only to invest in more parallel software but more appliciation specific accelerators.

CPUs/GPUs are beasts of hardware architecture, being complex mostly due to their flexibility. We can achieve higher performance with dedicated hardware (or FPGAs), and it looks like the economic reasons to do so are slowly becoming more certain.