This is really nice. This code looks like it is single-cluster with multiple-GPU. Are there benchmarks on how Horovod work on multi-cluster multi-GPU setting?
In the next part of the blog series, we will showcase the performance across a multi-cluster of gpus.
Thanks for the share @skun.
Author here, open to questions and feedback.