My guess is inadequate driver support for the USB mic. I'm running a focusrite 2i2 3rd gen, and although I have gripes with the drivers, the performance is miles beyond the default windows drivers and is fairly solid.
Due to the various problems with USB microphones, I generally recommend something like a Scarlett Solo + SM57 or SM58 as an entry point if you're buying your own gear. This has a price tag around US$250, once you include a XLR cable. Just like there's diminishing returns when you buy expensive interfaces/preamps/mics, there's also a sharp dropoff in quality and experience once you go below this baseline.
Even though you can find cheaper microphones and cheaper interfaces, I haven't found any that I can reliably recommend.
I've actually never encountered the issue with USB mics not accurately reporting latency because I don't think I've ever tried to record with one. Back in the 90s, in high school, my drummer friend and I went in together on a couple SM57s and a Radio Shack 4-channel mixer so we could record our jam sessions into a boombox tape deck (later a PortaStudio). I still have one of those SM57s and just pull that out whenever I've needed a mic.