Hacker News Reader

What did you do to enable Metal?

I use LM Studio and 0.2.9 now has a checkmark for it

How much RAM do you have? Ollama comes with a note: "this model requires 48GB of RAM".

How do you use 8x7B?

running it with LM Studio. No command line and has a huggingface browser, very user friendly.

Use ollama [1]

$ ollama run mixtral

[1] https://github.com/jmorganca/ollama

could you please eli5 how metal works

its just a co-processor included in Apple devices that has relevant components of a GPU, can accelerate graphics and general computation tasks

Apple’s chip architecture has a variety of co-processors alongside the CPU that make it the most perfect for transformers, in a convenient laptop form factor with fanless heat dissipation and low energy footprint