Hacker News Reader

Additional question is what kind of local hardware would be required for this? 7B parameters sounds very light weight, but I'm not sure. (Edit: The download is 33 GB).

Edit x2: As usual I'm in a twisty maze of pip packages that don't work together, with obscure errors about missing modules, even though I followed the instructions on the page to the letter. I really wish people didn't use Python for this stuff. A simple C/C++ program would be so much better.

View on HN

It's about 16 GiB at Q8 quants (combining both the image and language parts). (Meaning, community quantized models from HuggingFace).

I think it will technically run on anything that has enough memory. I just tried it on a standard laptop (dual-channel DDR5), and it took about 3 minutes for a 512x512. If you'd want to run it at interactive speeds, you would want a GPU (one which fits this in VRAM).

> "I really wish people didn't use Python for this stuff. A simple C/C++ program would be so much better."

You mean besides stable-diffusion.cpp ?

> You mean besides stable-diffusion.cpp ?

Yes, thanks, I didn't know about that. Will try it.

Reply on news.ycombinator.com