Hacker News Reader

This may happen if you chose a model which is larger then your available unified memory.

Which version of llama2 did you choose? And how much unified memory do you have?

Just the default (7B parameters?), which apparently requires at least 8GB of RAM, which is how much I have. It seems to be running fine now... it's just the installation process that caused it to completely die.

if it requires 8gb of ram and you have 8gb of ram that leaves nothing left for the other processes that need to run and the gpu which also shares the ram.

I’m just surprised that the process can successfully eat all of the memory so badly that the OS actually dies. I thought there was protection for stuff like that.

There's not, mlock() will crash your machine from userland happily

Reply on news.ycombinator.com