Hacker News Reader

Comment by SeanAnderson | Hacker News Reader

I'm curious if this will give better results than llama 7B? Llama 7B felt like a toy that, while cool to be able to run locally, did not feel useful in any way when contrasted to the state of GPT. Here's hoping for better and/or release of larger parameter models with low performance requirements soon :)

EDIT: my first question times out when ran online, seems like huggingface is getting hugged to death.

View on HN

Even if it doesn't initially, the fact that it's being released so permissively is massive - stable diffusion was made far more powerful by being hackable at all levels and I can't imagine we won't see the same here.

I imagine things like control nets that restrict output to parsable types, LoRa style adaptations that allow mixable "attitudes", that sort of thing.

Very different underlying architecture from diffusers, ofc. But the action of open source is the same - a million monkeys with a million xterms and so forth.

I'm really hoping for the ability to load in different sets of trained material as embeddings/textual inversions like in Stable Diffusion. Imagine scanning in some of your favorite philosophy and design books and throwing them with small weighting as a little flavor to your answer. The crossovers between LLM and Stable Diffusion type models (like Loras) is such a fascinating space to explore.

This already exists for LLaMA, I'm sure it'll be implemented for StableLM as well.

https://github.com/lxe/simple-llm-finetuner

good looks on the link! I was experimenting with script writing the other day and thought "gee, I really wish I could finetune on Beckett plays specifically".

I don't know if anyone else has experienced this same tipping point, but when I used to have ideas, I would look them up and discover that implementing them was probably out of scope. These days, I think "wouldn't it be cool..." and immediately stumble on a way to make it happen, by accident.

This has been around for GPT models for a while in the form of "soft prompts", which are rather approximate to textual inversion in the Stable Diffusion space.

They have on their team people from Anthropic which have stuff like Claude Instant which is likely running a very light model, ie the tricks from Anthropic have likely been incorporated into the model they’re presenting here

If you're comparing to ChatGPT performance then Vicuna 13B would be a best comparison point for something Llama-based.

Vicuna 13B performance is an order of magnitude below ChatGPT for all but gimmicky conversational stuff. Try giving both somewhat large, task-based prompts with steps and see what happens.

> Vicuna 13B performance is an order of magnitude below ChatGPT for all but gimmicky conversational stuff.

Until you connect it to external resources, I tend to think of anything you do with “brain-in-a-jar” isolated ChatGPT as gimmicky conversational stuff.

ChatGPT is still going to be way more capable when you use it's API to connect to external resources.

Maybe I should have phrased that better! I didn't mean that Vicuna was comparable to ChatGPT, just that it's the best Llama-based comparison you can make (since it's at least been conversationally trained).

Isn't ChatGPT a 165B parameter model?

No. OpenAI haven't disclosed parameter count of GPT-3.5 or GPT-4, which are models used by ChatGPT. You may be thinking of GPT-3, which is indeed a 175B parameter model.

Ah, interesting. Thought GPT-3.5 had the same structure as GPT-3, for some reason. GPT-4 would obviously be different.

GPT-3.5 is likely a finetuned Curie 13B using output from the full size GPT-3 175B.

Reply on news.ycombinator.com