Hacker News Reader

Comment by minimaxir | Hacker News Reader

8 GB of RAM with local LLMs in general is iffy: a 8-bit quantized Qwen3-4B is 4.2GB on disk and likely more in memory. 16 GB is usually the minimum to be able to run decent models without compromising on heavy quantization.

View on HN

But 8GB of Apple RAM is 16GB of normal RAM.

https://www.pcgamer.com/apple-vp-says-8gb-ram-on-a-macbook-p...

I concur. I just upgraded from m1 air with 8gb to m4 with 24gb. Excited to run bigger models.

> m4 with 24gb

Wow, that is probably analogous to 48GB on other systems then, if we were to ask an Apple VP?

Not sure what Apple VPs have to do with the tech but yeah, pretty much any core engineer you ask at Apple will tell you this.

Here is a nice article with some info about what memory compression is and how it works: https://arstechnica.com/gadgets/2013/10/os-x-10-9/#page-17

It's been a hard technical problem but is pretty much solved by now since its first debut in 2012-2013.

I've heard good things about how macOS handles memory relative to other operating systems. But Linux and Windows both have memory compression nowadays. So the claim is then not that memory compression makes your RAM twice as effective, but that macOS' memory compression is twice as good as the real and existing memory compression available on other operating systems.

Doesn't such a claim... need stronger evidence?

Interestingly it was AI (Apple Intelligence) that was the primary reason Apple abandoned that hedge.

It's 4-bit quantized (Q4_K_M, 2.5 GB) and still works well for this task. It's amazing. I've been running various small models on this 8 GB Air since the first Llama and GPT-J, and they improved so much!

macOS virtual memory works well on swapping in and out stuff to SSD.

Reply on news.ycombinator.com