Hacker News Reader

Well I was able to run the original code with the 7B model on 16GB vram: https://news.ycombinator.com/item?id=35013604

The output I got was underwhelming, though I did not attempt any tuning.

The author just made an update that makes the generation much better, even with the 7B model:

https://twitter.com/ggerganov/status/1634310199170179075

I tried it out myself (git pull && make) and the difference in results are day and night! It's amazing to play with, although you should prompt it differently than ChatGPT (more like the GPT-3 API).

parameter tuning is pretty necessary, according to anecdotes. People on twitter have got good results by changing the default parameters.

For 13b and 30b, it really needs high temperature to produce good outputs.

Reply on news.ycombinator.com