I think this might come down to heart rate. I've found that when I bike home at like 10 pm from somewhere say 30 minutes away, I'm fine now with sleep but when I first started biking back from somewhat far places I couldn't sleep well all night.
https://supermemo.guru/wiki/Circadian_phase I follow this graph from him and I've found that if I exercise at the time marked as "sports" even 1 hour isn't enough for my heartrate to lower which leads to a worse siesta nap. I'm hoping that long-term as I get more aerobically fit it becomes less of a problem.
More generally, I don't think the argument is that strenuous exercise impairs sleep. More that it effects your circadian rythm. iirc from talking to Woz, for him, sports make him more tired and bring sleep phase earlier and I think my experiences are somewhat similar, though both of us exercise in late morning.