There needs to be a moat. I see all these startups with a moat that is no bigger obstacle than a puddle.
A local (geographically to myself) pre-LLM (Convolutional Neural Network) company invested in a huge moat, building a large training set of human labeled data ... I knew the moment I saw an LLM that could ingest image that their moat had evaporated in an instant. A lot of the technical staff either left or lost their jobs but the company appears to be limping on hoping for an exit.
If a startup depends on a secret prompt and a few MCP servers, they have no moat.
I think the next wave of AI-based SaaS (rather than pure AI API providers) will be companies like the Legal IDE (Tritium [1]), recently featured on the homepage. Tools that innovate and use AI, but would still be innovate even without it.
[1] I have no affiliation whatsoever. I just liked the demo and the concept.