Oh that discussion thats been going on in the last few day is so silly. The vae is hardly any kind of bottleneck. You barely gain anything by working with the latents directly