Pytorch is awesome especially with automatic differentiation. We used to do the training in torch and then convert the trained model to caffe. The only problem is that the conversion script supports only torch 7. I heard that at Facebook they experiment and do the training in pytorch and then they convert the model to caffe 2. I hope they will release the conversion script.