Hacker News Reader

I've got a 3.xBIL model running on a 8GB version which is fine-tuned to be better than 13BIL models; heres how I did it: https://christiaanse.ca/posts/running_llm/

That's not memory swapping but something else right? I ask because it looks like the new mistral model but slightly different.

It’s fine tuned minstrel model based on orca Microsoft model.