Used 3090 are getting really cheaps on the second hand market.
Then if you only need VRAM, the Tesla M40 are even cheaper at 100€ per unit, which has 24GB of VRAM.
The P40 24GB is only $200, supports 4bit, and is about 80% the speed of a 3090 (surprisingly) for LLM purposes.
Good to know, thanks !