IMHO this is still a problem, it is easy to get RPi-style devices but for development a more powerful device would be great, however this is much harder to get. Sure, cross-compilation works but is usually tedious to setup and work with.
I've had the chance to work on one of these powerful ARM servers for some time. I was connected via ssh and mostly working with tmux+vim, I could compile natively with all these available cores and memory - development was a breeze.
Nevertheless there were some pain points compared to x86: Often no ARM64 binary packages, except for the packages provided by the distro. No precompiled binaries. Software wasn't working/building e.g. because Fedora on ARM64 uses 64K page size. perf didn't support as many performance counters as on x86. Sometimes I would've really liked to use rr (the reverse debugger) but it only supports x86. Pretty sure I've encountered more pain points that I don't remember and also I think I didn't discover all of them.
Development is almost exclusively done on x86. So it is only natural that x86 is the most tested and optimized architecture. This wouldn't be a problem if your offering would be much better/cheaper in some way and although I think ARM servers can compete in certain areas it doesn't seem to provide enough benefit to be worth the trouble. I agree with Linus that they should be focusing on developers and software and let the machines bubble up into the server segment.