Zephyr 7B – Mistral Finetune that responds like ChatGPT
huggingface.co
huggingface.co
> Zephyr-7B-α has not been aligned to human preferences with techniques like RLHF or deployed with in-the-loop filtering of responses like ChatGPT, so the model can produce problematic outputs (especially when prompted to do so). It is also unknown what the size and composition of the corpus was used to train the base model (mistralai/Mistral-7B-v0.1), however it is likely to have included a mix of Web data and technical sources like books and code.
Its very deliberately unlike ChatGPT, which is what makes its performance interesting.
So sad that this is where we are and that some companies and the doomsday cultist partners are pushing to enshrine the sad state into law.
You should be able to run this locally on CPU or GPU as long as you have 16GB RAM. Less if using a quantized model like https://huggingface.co/TheBloke/zephyr-7B-alpha-GGUF with llama.cpp
https://huggingface.co/LoneStriker/zephyr-7b-alpha-4.0bpw-h6...
It's certainly in the realm of being able to voice a NPC in an RPG game by now, as occasional hallucinations doesn't really matter there.
How do we (the "LLM community") solve this/overcome this?
Maybe new architectures will be better, but companies will likely be making the best foundational models.
In the meantime, you can hook it up to a vectordb and ask the llm to check itself with zero temperature, but that only helps so much.