I wonder if "faster" really matters. The implementation-level optimization seems paled when compared to productivity, ecosystem, and algorithm-level and hardware-level boost. Case in point, Caffe2 wanted to be the backend of PyTorch in FB, yet failed. MxNet claims to be "faster" than PyTorch in pretty much every way and Smola personally pushed orgs in AWS AI really hard to adopt MxNet and Gluon* yet failed. The latter failure is particularly worth mentioning: all teams in AWS AI complied and adopted MxNet but then three years later decided to move to PyTorch. All-in-all, the incremental speed boost was simply irrelevant.