Hacker News Reader

If you're comparing to ChatGPT performance then Vicuna 13B would be a best comparison point for something Llama-based.

Vicuna 13B performance is an order of magnitude below ChatGPT for all but gimmicky conversational stuff. Try giving both somewhat large, task-based prompts with steps and see what happens.

> Vicuna 13B performance is an order of magnitude below ChatGPT for all but gimmicky conversational stuff.

Until you connect it to external resources, I tend to think of anything you do with “brain-in-a-jar” isolated ChatGPT as gimmicky conversational stuff.

ChatGPT is still going to be way more capable when you use it's API to connect to external resources.

Maybe I should have phrased that better! I didn't mean that Vicuna was comparable to ChatGPT, just that it's the best Llama-based comparison you can make (since it's at least been conversationally trained).

Isn't ChatGPT a 165B parameter model?

No. OpenAI haven't disclosed parameter count of GPT-3.5 or GPT-4, which are models used by ChatGPT. You may be thinking of GPT-3, which is indeed a 175B parameter model.

Ah, interesting. Thought GPT-3.5 had the same structure as GPT-3, for some reason. GPT-4 would obviously be different.

GPT-3.5 is likely a finetuned Curie 13B using output from the full size GPT-3 175B.

Reply on news.ycombinator.com