Ask HN: Is there a good alternative for a local ChatGPT

11 points·ludovicianul··8 comments

I would like to have something as similar as possible to ChatGPT, but deployed in a closed environment. Feed it my specific data and query it afterwards. What would you recommend?

View on HN

Keep in mind that the number of parameters on this thing goes to hundreds of billions. So, start with a local datacenter and continue from there. Facebook's Lama which is supposed to be both similar in performance and much reduced in size has only 65 billion params and weights around 220gb (as far as I remember from an article). You will need a beefy server to run this and it would better be shared or the server will be idle most of the time.

I've exaggerated a bit when saying close to ChatGPT. But my focus is getting something which already bundles the complexity, feed it data and start using it. Without running 100 seps to reach that state.

Well, you can try the hugging face models. I'm not sure if someone has already built an consumer level app on them, but at least they provide trained models for the most popular stuff out there in an easy to pip install packages.

Alternatively, you can check Eutheler AI though I haven't worked with anything of theirs. I just know that they were trying to be some kind of Open AI alternative.

1. https://www.eleuther.ai/about/

Thanks!

https://huggingface.co/

This might help you: https://www.hpc-ai.tech/blog/colossal-ai-chatgpt

A web UI for running Large Language Models https://github.com/oobabooga/text-generation-webui

There is a thread on /g/ about running your own LLM chatbots with a lot of good info.

Reply on news.ycombinator.com