Look up Google's TPU. Generally speaking, the Google engineering blogs are quite good. A lot of effort has gone into making all the other parts around the tensor processing fast as well. There are some AI specifics about batching requests into a single model as well.