Hacker News Reader

Comment by Garcia98 | Hacker News Reader

I've been following open source LLMs for a while and at first glance this doesn't seem too powerful compared to other open models, Flan-Alpaca[0] is licensed under Apache 2.0, and it seems to perform much better. Although I'm not sure about the legalities about that licensing, since it's basically Flan-T5 fine-tuned using the Alpaca dataset (which is under a Non-Commercial license).

Nonetheless, it's exciting to see all these open models popping up, and I hope that a LLM equivalent to Stable Diffusion comes sooner than later.

[0]: https://github.com/declare-lab/flan-alpaca

View on HN

Sounds like you might be the right person to ask the “big” question.

For a small organization or individual who is technically competent and wants to try and do self-hosted inference.

What open model is showing the most promise and how does it’s results compare to the various openAI GPTs?

A simple example problem would be asking for a summary of code. I’ve found openAI’s GPT 3.5 and 4 to give pretty impressive english descriptions of code. Running that locally in batch would retain privacy and even if slow could just be kept running.

Google's Flan-T5, Flan-UL2 and derivatives, are so far the most promising open (including commercial use) models that I have tried, however they are very "general purpose" and don't perform well in specific tasks like code understanding or generation. You could fine-tune Flan-T5 with a dataset that suits your specific task and get much better results, as shown by Flan-Alpaca.

Sadly, there's no open model yet that acts like a Swiss knife and gets good-enough results for multiple use cases.

Iterating on the question, what model/weights would be the most appropriate for the specific use case of code generation right now?

Sorry for the late reply, as I said Flan-UL2 (or Flan-T5 if you want lighter models) fine-tuned against a dataset like CodeAlpaca's[0] is probably the best solution if it's intended for commercial use (otherwise LLaMa should perform better).

[0]: https://github.com/sahil280114/codealpaca

Their goal isn't to make a powerful model. It's to show how well compute-optimal models do on test-loss as a function of increasing model size. This function can be used with some caveats to forecast the test-loss of larger models for which compute-optimality becomes more important.

Reply on news.ycombinator.com