I think a general way to answer this is by considering for any domain you know: What would you pay a human to do right now, that LLMs frustratingly can't, but should in theory, if only they were a bit better and more consistent?

This could mean: Instead of diving into langchain and trying to program your way out of a bad model, or trying to do weird prompts, just write a super clear set of instructions and wait for a model that is capable of understanding clear instructions, because that is an obvious goal of everyone working on models right now and they are going to solve this better than your custom workaround can.

This is not a rigid rule, just a matter of proportions. For example, you should probably be willing to try a few weird intermediary prompt hacks, if you want to get going with AI dev right now. But if most of what most people do will probably be solved by a somewhat better model, that's probably a cause for pause.

I may be misunderstanding your meaning, but I'm not convinced that "prompts as a service" is short term. I think we'll see a number of apps pop up that will be essentially that, i.e. powered by a generative AI, but with a great UX. Not everyone is good at prompting, and although it is a skill many will develop, packaging up great prompts in niche problem areas still looks like an area of opportunity to me. I'm not talking necessarily about chat experiences, but apps that can, as an example, maintain task lists for you after consuming your incoming communications.