Personal anecdata: I tried using ARM based server and didn't find any pain.
Compatibility issues?
Performance issues?
Something else?
Many open source projects don’t even have build tooling or testing on anything but x64.
That’s a lot of pain, even discounting buggy and poorly tested drivers.
As for which libraries, I’m too lazy to do a detailed search. All I can report anecdotally is that we very briefly tried ARM at dayjob on our codebase about a year ago and it seemed everything was immediately broken. Especially interop between high level languages and C libraries. The team tasked with it said almost imm fiat would “way too hard, let’s quit this experiment”.