• VENDOR LOCKIN •
It's possible that OpenAI is managing to establish a permanent competitive edge and will be the next Google. There is/was a lot of rather handwavy assumptions about open source models being competitive thanks to Stability AI and the Facebook leak, and last time I raised red flags about this on HN I got dunked on. But right now it looks more likely that LLMs are the new search engines and there won't be any competitive implementations that are both legal and that you can run yourself. It'll be APIs all the way, just like with web search engines.
Also, embeddings are model/vendor specific and expensive to compute. If you calculate 10 million embeddings using OpenAI it's going to be expensive to recalculate them all with another vendor.
• UNCLEAR PRICING •
It seems likely that OpenAI is either selling at below cost, or is at best break-even on compute. It's very unclear right now if prices are going to rise, fall or remain where they are and in fact AI prices have done all these things in just a few months. That makes it difficult to know if it's an acceptable business risk to deeply incorporate AI into your workflow.
• HUMAN BOTTLENECKS •
A lot of AI use cases that sound initially compelling actually bottleneck on human review, because it's too risky to put AI output straight into production. Some people don't care hence the wave of amusing "As an AI language model" spam, but it shows what can go wrong if you skip reviews.
• FACTUALITY •
Obvious, but after having studied this more and listened to a talk by one of the OpenAI team I think this will actually go away as a problem in the semi-near term future.
• INTEGRATION COMPLEXITY •
The limited LLM context window size means a lot of tricky workarounds are required for many use cases, which increases implementation complexity.
• TESTING DIFFICULTY •
LLMs aren't deterministic, so it's a testing nightmare. You never know when the LLM will just do something unexpected that breaks your use case or integration.
• BUSINESS UNCERTAINTY •
There's genuinely a ton of potential, the tech deserves the hype. But it can be hard to capitalize because so many people are trying to do things all at once, so where do you go that you aren't immediately drowned in VC-flush competition? And many ideas may be more naturally done as features of existing products, so if the vendor isn't doing them, should you wait or should you try developing something yourself and risk obsolesence? The pricing uncertainty compounds that, of course.