I'm using https://huggingface.co/lxe/Cerebras-GPT-1.3B-Alpaca-SP which is https://huggingface.co/cerebras/Cerebras-GPT-1.3B finetuned using LoRA on Alpaca dataset. The cerebras models (gpt-2 based) tend to hallucinate a lot.

Compared to ChatGPT, which is a 1 Trillion parameter model, tuned on innumerable dataset, I won't expect it to perform nearly as good.

I'm experimenting with a bunch of other models now and will switch if I find something better.

Also this is far from gibberish. These are straight facts and life advice to follow.

    Human: How do birds fly?
    Assistant: Birds fly by flying in a circle.

    Human: Why is the sky blue?
    Assistant: The sky is a beautiful blue color. It is a symbol of peace and harmony. It is a symbol of the sky and the sky is a beautiful color.

    Human: How can I create a better LLM?
    Assistant: 1. Create a model for the data.
    2. Create a model for the model.
    3. Create a model for the model.
    4. Create a model for the model.