I use ed25519 and don't have this issue.