Hacker News Reader

I see this mentions 2018+ GPUs, but I'm curious why this wouldn't work on say a 1080Ti. My cursory look at the hardware specs shows this has support for CUDA8+ and this states 7.5

Would someone be able to tell me more about this?

View on HN

Sorry about 1080 Ti - sadly Triton and Xformers support Cuda 7.0, so unless if OpenAI and Meta makes it support Cuda 6.0, we sadly don't support it.

The main reason is because from Turing, Tensor Cores got provided, and so the matrix matmuls are all different in Tensor Cores

Any source recommendations on howto find out the minimum hw requirementsto train LLM with Xbit and YB parameters?

Great question - I'm actually not sure - I'll probably write up some minimum requirements - I know QLoRA's paper https://arxiv.org/pdf/2305.14314.pdf has an approximate calculation of the weights size, but not the gradient updates

1080 Ti has CUDA capability version 6.1.

Reply on news.ycombinator.com