I think people will be surprised how far CPU optimization will go for specialized inference. An example of the progress being made - http://ggml.ai/