Not sure about M1, but on the Macbook Pro 14" with an M1 Max using 8 threads I transcribed a 44 minute podcast in 16 minutes. So about 3x "real time" speed.
What model are you using? I guess large, as my M1 Max takes about 1.4 min for a 4 min file (35% of total time)?