Hacker News Reader

harisecon RedPajama: Reproduction of LLaMA with friendly license

Same here. If you believe the following research (which I do), the ability to perform complex reasoning is likely to be from training on code:

https://yaofu.notion.site/How-does-GPT-Obtain-its-Ability-Tr...

I think it's essential to increase the quantity of code tokens.

harisecon Gpt4all: A chatbot trained on ~800k GPT-3.5-Turbo Generations based on LLaMa

This is a common misunderstanding; fine-tuning a model does not mean teaching the model new information. Fine-tuning is used to adapt the model to perform better in a specific task or domain using the information it already has, in a specialized way (like a chatbot for question/answer interactions). Training a LLM on new data is extremely expensive so it's not possible.

harisecon Using ChatGPT Plugins with LLaMA

Bard is a lightweight model version of LaMDA. LLMs are very expensive to run.

harisecon Sam Altman didn’t take any equity in OpenAI, report says

Buried in the article is this gem: "The latest language model, GPT-4, has 1 trillion parameters." - have no idea if it's true or not.

harisecon Elon Musk, Sam Altman, and OpenAI

Hidden inside the article: "The latest language model, GPT-4, has 1 trillion parameters."