You can as well, make separate builds for M1 and Intel. The size of each binary will be smaller.