Isn't it the case that training a ML model consumes vastly more power than using the trained model for inference?

If so, the problem may be somewhat overstated. Training is a dispatchable power load. It can be scheduled for periods of surplus power in regions where surplus power is periodically abundant. Training a ML model could literally follow the sun.