> A forward() function gets called when the Graph is run.

Yes, the idea behind it is the same. The difference: PyTorch has a forward() function in their module class which you have to override, while in TensorFlow you can specify that yourself.

> Based on some reviews, PyTorch also shows a better performance on a lot of models compared to TensorFlow.

E.g.

http://deeplearningathome.com/2017/06/PyTorch-vs-Tensorflow-...