It looks like you can - the repo for VQ-GAN has instructions for this:

https://github.com/CompVis/taming-transformers#training-on-c...