Classifier Free Guidance works on LLMs with a significant boost in performance | Hacker News Reader

Classifier Free Guidance works on LLMs with a significant boost in performance

arxiv.org

More granular prompting like this is a huge deal... If they ever release the code for it. Even if its cherry picked, the effect on that "Dragon flying over France" prompt example is huge.

Also, I noticed Stability and Coreweave provided funding for the compute. That actually makes me kinda suspicious, like they are going to keep the implementation to themself.

Since many of the advancements in the diffusion Community (lora etc) are applicable to the text generation community, I don't see how this won't hit the local llama scene within the next month or so.

It appears to be a one-line (or, if you prefer readability, one-function) change - my guess is 80% chance by EOD and 95% chance by end of week :^)

Correct me if I'm wrong, but the change seems to be:

    return_logits = generated_logits + cfg_scale * prompt_logits

Pardon my ignorance, but what exactly does "Classifier Free Guidance" mean, and what are logits?

NLP folks are so slow to implement EVERYTHING

Where the hell are negative prompts? Where are word weights? Where is ANYTHING that I can do in Automatic1111?

There are no technical reasons why they can't be implemented.

The author of the paper has already shared their implementation and are willing to contribute it to hf-transformers: https://github.com/huggingface/transformers/issues/24536

This is basically a balanced audio cable, but in a bajillion dimensions

Reply on news.ycombinator.com