Nvidia should enjoy this period while it lasts. I fail to see any future where custom ASICs don't each their lunch.

There's so much opportunity for custom silicon to massively improve compute-per-watt -- for both for training and inference. Nvidia got lucky with their high-memory GPUs being early to this space, but the usefulness of that architecture has already peaked.