We are talking about very big models which training requires an enormous amount of hardware. Not sure how scalable HuggingFace transformer for training such models is.