Hacker News Reader

Comment by causality0 | Hacker News Reader

It makes me wonder how long it will be until open-source locally-run AI chatbots reach the GPT-4 level. Five years? Ten?

The real limiter is having enough GPU RAM to train and run a usefully large model. Everything else is just fiddly details.

Not anymore. Llama.cpp and kobold.cpp run well with low vram and non Nvidia GPUs.

Huggingface is full of finetunes now, and I believe a 33b model can be finetuned on a single 3090.

Llama.cpp is developing some kind of training, but I have no idea what the requirements will be.

To me, GPT's real "secret sauce" is how it primes the model for interactive prompting instead of just text completion.

There are "chat" and "instruct" variants of most models now.

It depends on the tasks, I’d argue some of the open source alternatives are at the gpt-4 level already, particularly for code generation.

That said, I suspect summarization, translation, etc will take time. I’d suspect under 1 year.

>I’d argue some of the open source alternatives are at the gpt-4 level already, particularly for code generation.

Like which one?

Large context Chronos 33b (and some mixes) for roleplaying type chat.

And there are some very new 65b finetunes I have not tried.

The open source bot I played with last week (Wizard Vicuna uncensored) was 85% of gpt 3.5 on a VERY hard use case (fiction stories).

Maybe a year before we are the level of stablediffusion?

When it happens, I think it’ll happen on our phones first.

Given the fundamental hardware matrix operations, that seems unlikely, unless it's something very scaled down.

its more a question of hardware progress than software probably. theres a minimum level for these behaviors to emerge