Are you comparing a binary being run under dynamic binary translation with a native binary?
Not really an honest comparison, if that's the case.
Why? The Java community has only just been working on ARM64 support at all over the last few years, and it's still a little limited, and macOS ARM64 support is only out a couple of weeks and only from non-Oracle third-party builders I believe.
Depends which version you have installed. It's a taken a while for the native versions to reach the mainstream channels, so unless you've specifically installed an M1 build you probably have an x86 build being run under translation.
I see a 4.1x speedup in compiling (including tests), excluding dependency resolution:
Azul Native M1 java JDK11:
./sbt "clean;compile;test:compile"
254.86s user
11.84s system
519% cpu
51.375 total
Azul java JDK11:
./sbt "clean;compile;test:compile"
490.04s user
59.48s system
269% cpu
3:23.81 total