This is interesting, although the results are underwhelming IMO. I actually had the same idea - finding the latent space of instrument sounds and using that for synthesis - a couple of years back. After countless hours of research I managed to turn it into a commercial software instrument called "GalaXynth" [1]. For me at least, it turned out that the "automatic" latent space (i.e. discovered by autoencoding) isn't that interesting musically, therefore I turned to hand-designing the latent space, which was a gargantuan task that I'm not sure I would do over again if I knew how hard it would be. Anyway if anyone is interested in this type of thing you should get in touch!