I agree it feels "worse" than a physical screen - it is certainly less crisp. But I ran an experiment, and created a doc with font sizes from 1-15 pt, then viewed it on my physical 24" 4k monitor, and on a mirrored Mac display in the AVP set to the same (virtual) dimensions and distance away.
Interestingly, text became illegible (without leaning closer in either case) for me at around the 6pt mark for both of them. Objectively speaking, the size of font I can read is pretty equivalent, although I find myself preferring fonts about 2pts larger on the AVP than on my physical monitor.
I suspect this is because the physical display is able to benefit from precise pixel alignment and subpixel rendering, which the AVP can't. But the "average density" feels pretty equivalent... it somehow feels more an issue of optical sharpness than of "pixel size" (even I know the two are related.)