GPT4[0] is actually very good with base64 to the point where it makes perfect sense.

I'd be interested in how well you could finetune 3.5 to use different compression.

[0] - https://platform.openai.com/playground/p/hfLUBCTE8RrRYPIRxEe...