Hacker News Reader

Comment by skykooler | Hacker News Reader

Is there a way to run things like this with an AMD graphics card? Every Stable Diffusion project I've seen seems to be CUDA focused.

View on HN

That's because Stable Diffusion is built with PyTorch. Which isn't optimized for anything but CUDA. Even the CPU is a second class citizen there. Let alone AMD or other graphics.

Not saying PyTorch doesn't run on anything else. You can but those will lag and some will be hackish.

Looks like Nvidia is on its way to be the next Intel.

Part of this is simply that AMD does a TERRIBLE job of writing tooling and software for anything that isn't just "render these triangles for this videogame". Doing raw compute things with AMD GPUs just seems limited to those building supercomputers apparently. Their promised "cross GPU" solution in ROCm is only available on a tiny fraction of the GPUs they make, seemingly without architecture excuses for why it isn't available on 5000 series cards, it took them YEARS to provide a backend that blender could actually use, productively and without crashes and bugs, and their drivers are just in general very fragile.

It's weird to me how much lip service AMD puts into making cross platform, developer friendly, free and open GPU compute standards, and then turn around and just not do that.

From the Arch wiki, which has a list of GPU runtimes (but not TPU or QPU runtimes) and arch package names: OpenCL, SYCL, ROCm, HIP,: https://wiki.archlinux.org/title/GPGPU :

> GPGPU stands for General-purpose computing on graphics processing units.

- "PyTorch OpenCL Support" https://github.com/pytorch/pytorch/issues/488

- Blender re: removal of OpenCL support in 2021 :

> The combination of the limited Cycles split kernel implementation, driver bugs, and stalled OpenCL standard has made maintenance too difficult. We can only make the kinds of bigger changes we are working on now by starting from a clean slate. We are working with AMD and Intel to get the new kernels working on their GPUs, possibly using different APIs (such as CYCL, HIP, Metal, …).

- https://gitlab.com/illwieckz/i-love-compute

- https://github.com/vosen/ZLUDA

- https://github.com/RadeonOpenCompute/clang-ocl

AMD ROCm: https://en.wikipedia.org/wiki/ROCm

AMD ROcm supports Pytorch, TensorFlow, MlOpen, rocBLAS on NVIDIA and AMD GPUs: https://rocmdocs.amd.com/en/latest/Deep_learning/Deep-learni...

RadeonOpenCompute/ROCm_Documentation: https://github.com/RadeonOpenCompute/ROCm_Documentation

ROCm-Developer-Tools/HIPIFY https://github.com/ROCm-Developer-Tools/HIPIFY :

> hipify-clang is a clang-based tool for translating CUDA sources into HIP sources. It translates CUDA source into an abstract syntax tree, which is traversed by transformation matchers. After applying all the matchers, the output HIP source is produced.

ROCmSoftwarePlatform/gpufort: https://github.com/ROCmSoftwarePlatform/gpufort :

> GPUFORT: S2S translation tool for CUDA Fortran and Fortran+X in the spirit of hipify

ROCm-Developer-Tools/HIP https://github.com/ROCm-Developer-Tools/HIP:

> HIP is a C++ Runtime API and Kernel Language that allows developers to create portable applications for AMD and NVIDIA GPUs from single source code. [...] Key features include:

> - HIP is very thin and has little or no performance impact over coding directly in CUDA mode.

> - HIP allows coding in a single-source C++ programming language including features such as templates, C++11 lambdas, classes, namespaces, and more.

> - HIP allows developers to use the "best" development environment and tools on each target platform.

> - The [HIPIFY] tools automatically convert source from CUDA to HIP.

> - * Developers can specialize for the platform (CUDA or AMD) to tune for performance or handle tricky cases.*

>Looks like Nvidia is on its way to be the next Intel.

There's a non-trivial part of several industries who do not want to see that happen, because NVidia treats its customers and partners orders of magnitude worse than Intel ever did.

Stable Diffusion works fine for me on my RDNA2 gfx1031 ( RX6700 XT ) under Debian. If you are on a Linux kernel with amdgpu, have your distro's latest version of ROCm HIP runtime, and a relatively recent AMD GPU, you just need to replace the default pytorch with the ROCm HIP version.

    # Install your distro's HIP runtime and rocminfo
    $ apt install rocminfo hip-runtime-amd # for debian and derivatives

    # Confirm ROCm is installed properly and confirm your gpu is supported
    $ rocminfo
    ROCk module is loaded
    ...                 
      Name:                    gfx1031                            
      Marketing Name:          AMD Radeon RX 6700 XT  
    ...

    # Must uninstall any non-ROCm torch libs
    $ pip3 uninstall torch torchvision torchaudio

    # Install latest ROCm pytorch, see website for more versions
    $ pip3 install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/rocm5.1.1

    # Required env for unofficially supported newer gpu's like gfx1030 & gfx1031 that give error "hipErrorNoBinaryForGpu", can be added to /etc/environment to make permanent
    $ export HSA_OVERRIDE_GFX_VERSION=10.3.0

    # Test torch HIP is working
    $ python3
    >>> import torch
    >>> torch.cuda.is_available()
    True
    >>> quit()

    # Run Stable Diffusion workloads like normal...

Hope this points people in the right direction with AMD GPUs.

Blender can also take advantage of the installed HIP runtime for render acceleration in the latest versions too once you enable the setting in preferences.

You can run it on an Intel CPU if that helps: https://github.com/bes-dev/stable_diffusion.openvino

I followed this guide and it works well:

https://www.travelneil.com/stable-diffusion-windows-amd.html

https://www.travelneil.com/stable-diffusion-updates.html

Tedious to set up, but the author does a great job of explaining all of the steps.

Reply on news.ycombinator.com