Follow on, great question (separately from John) can it run on Vulkan?
He was saying he has a hard time finding stuff that doesn’t only run on CUDA or ROCm and sees this as a huge opportunity
You can pull down the repo and run a few easy commands to get inference up and running.