The M40 does not support 4bit, so it's basically useless for LLMs.
The P40 24GB is only $200, supports 4bit, and is about 80% the speed of a 3090 (surprisingly) for LLM purposes.
The P40 24GB is only $200, supports 4bit, and is about 80% the speed of a 3090 (surprisingly) for LLM purposes.
Good to know, thanks !