I've quantised 2B & 7B Instruct models here https://huggingface.co/iAkashPaul/gemma-2b-it-gguf & https://huggingface.co/iAkashPaul/gemma-7b-it-gguf