We're working on an open source chat gpt that runs on kubernetes. https://github.com/bionic-gpt/bionic-gpt

And you're right many companies are looking at running LLMs in house.