Disclaimer: developer of Hops. This blog basically argues that systems like Horovod (Ring AllReduce) are architecturally superior to Parameter Server models (like TensorFlowOnSpark).