Think of the dev ecosystem effects in all the various permutations, e.g. from Windows or Apple to AWS to more granular like PyTorch.
Those advantages aren't easily eliminated just by more competitors or even 'better' competitors.
But holicistally, if you were to buy the entire Nvidia company today, even if those advantages continue untouched for the next 10 years, and competition doesn't catch up, and the AI boom continues ... you're still not getting your money back. That's how high the valuation is.
Nvidia's fief has a lot of folks in it, who've built sloppy stuff all over the place because they can. Most of that won't get replaced.
But the modern tooling like PyTorch and its underlying JAX doesn't need or care about this legacy. The future can build far more intimately & with deliberation to folks who haven't such large swamps around their ecosystems. We can tune systems more efficiently & carefully when our vendors don't keep us so so far away from the core hardware. Nvidia has to keep maintaining this huge footprint of software, keep it going themselves, supporting a vast legacy, while everyone else gets to innovate forward, with much better access to the hardware.
When I hit an input box on my phone, a keyboard pops up on the screen, with the first row saying QWERTYUIOP, just like the Sholes and Glidden typewriter of 1878. I'm waiting for innovation on this first, then I'll look for innovation beyond CUDA.
That random order of letters from 1878 still appearing on my screen is the definition of legacy.
PyTorch supports both ROCm and Ibex, but it's support is spotty and incomplete to the point where it becomes a pain to use despite AMD spending billions on improving its software layer.
NVIDIA is reaping a decade of design decisions on CUDA compatibility made less sloppily than its competitors.
People totally underestimate that and still think of Nvidia as a Semi company which only makes chips lol.