The real moats in this field will come from the hardware industry. It's way too expensive to train these models on general purpose compute. Vertically designed silicon that brings down the unit economics of training and inference workloads are already being designed, in industry and in academia.