The moat is the products that can be built. The moat is always the product - because a differentiated product can't be a commodity. And an LLM is not a product.
Google and MSFT and Meta have already "won" because they have profitable products they can build LLMs onto. Every other company seems to be burning cash to build a product, and only ChatGPT is getting the brand recognition to realistically compete.
Building an LLM is like building a database. Sure a good one unlocks new uses, but consumers aren't buying something for the database. Meanwhile enterprise customers will shop around and drive the price of a commodity down while open source alternatives grow from in-house uses to destroy moats.
Even hardware isn't a true moat. Only Google has strong vertical integration with their TPUs, and that gives them a lead. BUT Microsoft, AWS, Meta and a whole bunch of startups are building out custom silicon which will surely put pressure on them and Nvidia to keep innovating and earning that price edge.