I think they necessarily need to specialize, as certain information is only available in the context of the domain. I think bigger context windows will hit a limit, and you'll need to have actually trained and guided the AI on specifics of the domain to be useful.

At the moment it's only the fact that public documentation is available for so many tools that it's proving useful for so many things. But what about massive, closed source, boutique enterprise systems? You can feed it docs as context, but it would be better if it were trained on docs, support tickets and internal forums then properly aligned.