ESP32-S3 is a great USB host. I have a custom version where I use a Daisy Seed 3 as the ADC/DAC (it uses I2S for audio, not USB) that I built with around 4ms total latency. If you turn off the amp model (which is pointless, but) it's around 2.6ms. I also patched the iRig HD X firmware so its analog latency is around 1.2ms, practically achieving the same performance as the custom Daisy Seed 3 interface.