+1. In Ableton on Windows you can get your latency down to ~40ms without a dedicated sound card using ASIO. Mac's drivers are even better with sub ~20 ms on my m2 pro IIRC.
Just to be clear that you're measuring apples to apples with OP:
You are measuring less than 40ms roundtrip latency on your Mac. Is this correct?