Hacker News Reader

Full thread

evolveyourmind

Any benchmarks?

View on HN

Some people have timed it here, it looks like it's taking 15-20s/it (dependent on quant and hardware).

https://github.com/leejet/stable-diffusion.cpp/issues/1

I have compiled it with the command:

cmake .. -DGGML_CUBLAS=ON -DCMAKE_CUDA_COMPILER=/opt/cuda/bin/nvcc

to use my NVIDIA GeForce RTX 2060 SUPER.

I have converted the model to use FP16.

With these choices, the time per iteration is between 8.5 s and 9 s and the total time for making an image is around 200 s.

That seems a lot worse than a 2060 SUPER with PyTorch in A1111.

https://vladmandic.github.io/sd-extension-system-info/pages/... (search for 2060 SUPER)

This is not surprising, because the GPU support in GGML is said to be preliminary and it is optimized for being run on CPUs.

Seeing the times reported by other people, it seems that using the GPU with GGML, instead of the CPU, still provides a speed improvement, but it is small.

Nevertheless, I have appreciated that after following exactly the instructions of this project everything was up and running after a few minutes and it could be tested.

Past attempts to install all the environment needed to run such models have required much more work.

Reply on news.ycombinator.com