The Java RSA implementation apparently had a timing side-channel attack vulnerability for many years that was only discovered this year[1].
I don't know if there were more serious issues with the Java RSA implementation, in the past, but the same cannot be said about their ECDSA implementation. It was completely broken for a while before it was discovered[2].
Bad implementation can happen with any algorithm, but if the algorithm is simpler and built to be more foolproof like Ed25519, that's just far less likely. Unfortunately most of these libraries are not written by a professional cryptographer who codes without bugs, and even Daniel J. Bernstein, a cryptography expert who IS famous for writing software without almost no bugs[3], criticized RSA and ECDSA[4] and designed Ed25519.
To be honest, if even djb doesn't want to implement RSA and ECDSA then I honestly think nobody should.
---
[1] https://www.rapid7.com/db/vulnerabilities/redhat_linux-cve-2...
[2] https://www.cryptomathic.com/blog/explaining-the-java-ecdsa-...