It's similar to ia32 vs amd64. From amd64, you can ensure that SSE exists and guarantee that certain legacy features don't need to be coded around. Plus additional GP registers are always nice and you know they're there (despite usually leaving that to a compiler to manage).
With aarch64, you can ensure that VFP3+ and NEON both exist for FPU capabilities and choose between them at your behest (FP accuracy vs speed). You also get a few cryptographic features (AES, a couple SHA's and finite field arithmetic). However, with x86 you're able to use the CPUs (more or less) with feature flags since you can test for features in real-time. ARM doesn't offer this to near the degree (since there is no BIOS/UEFI in most boards), so you either code/compile directly for a certain CPU and it's features or use something like U-Boot/Linux's device tree spec files to hint at a boards feature set. You also have different entry points, clock speeds, memory maps, etc. This is why you see most Linux/FreeBSD/Android releases released per-board vs generic installers like with x86.