I think the only place "GPT startups" will really thrive long term is specific niche business areas where the big boys (Google/MSFT) are not likely to want to compete. For example, there was an HN post about a legal startup that used AI for various purposes. I could see that one building a sizable moat over time as "the go-to place for legal AI support" if their UI is good and very tailored to legal-specific workflows.
My primary point is that I think "generic" AI tool startups are likely to fail because the big boys will just build them into their products. E.g. a tool that just helps you write is going to have a hugely difficult time competing against the integrated functionality of Word or Google Docs (I'd be shuddering if I were Grammarly). Google and Microsoft, though, have largely stayed out of dedicated tools for highly specific verticals, and with all of the antitrust eyes on them I think they're likely to stay out of those spaces.