Hacker News Reader

I've run local video generation models on an 8GB graphics card and know firsthand that nothing runs smoothly when memory is insufficient. So seeing 14GB of weights crammed into 2GB of RAM is impressive.

If running continuously for over an hour (like an overnight batch task), will a fanless MacBook Air overheat and throttle? Can the SSD handle the continuous weight reads and sustained output speeds?

Great work, congratulations on the release!

View on HN

Thank you very much!

I think it will throttle quite soon, but I haven't tried runs longer than 30minutes with this engine.

However, there is no constant load on ssd or gpu. i/o and gpu work are alternating and there is a brief idle periods for each i/o and gpu during inference (because gpu waits for i/o and after that i/o waits for gpu)

This is where MoEs shine though. You don't need all experts in memory at once. Diffusion inference doesn't have sparse inference.

Reply on news.ycombinator.com