https://ollama.ai/library/yi/tags
for anyone who tinkers with these via ollama.
I have a 64gb M1 Max Macbook Pro and I think I'm limited to 4 bit quantized models? Can someone elucidate my options here? Can I do 6 or 8?
for anyone who tinkers with these via ollama.
I have a 64gb M1 Max Macbook Pro and I think I'm limited to 4 bit quantized models? Can someone elucidate my options here? Can I do 6 or 8?
ollama run yi:34b-chat-q2_K # 2-bit ollama run yi:34b-chat-q4_0 # 4-bit ollama run yi:34b-chat-q8_0 # 8-bit