70B at f16 needs about 120-160GB (4 A100 40GB or 2 A100 80GB).

Quantization still seems to have some issues for the 70B model.