For us the complexity has all been about the infra around the AI stack. For example, having a k8s cluster for running inference for open source models (Stable Diffusion, Bark, etc.). Beyond that it’s been custom abstractions around Open AI/PaLM APIs and vector stores (pinecone and Faiss).

This stack is evolving so quickly but we are convinced it goes back to a distributed systems/infra problems when it comes to complexity, unless you’re training your own foundation models.