Yes, but possibly not the entire model, hypothetically for instance some fine-tuning on compression and then distillation.
I can imagine some uses for this. Imagine having to archive a massive dataset where it’s unlikely any individual image will be retrieved and where perfect accuracy isn’t required.
Could cut down storage costs a lot.
And how much compute time/power does “decompressing” take compared to a jpg?