I'd argue the RFC is already there and it's RFC 8032, which defines "32 bytes string" <-> "public key" and "32 bytes string" <-> "private key" APIs.
Then you can use your preferred format to serialize a 32 bytes string if you need a text-safe encoding. (If you don't you're done!) For example, you can use RFC 4648.
I am positive every language has Base64 code, and if there's an Ed25519 library out there that can't accept 32 byte strings we should talk to the author because it's broken as it doesn't implement RFC 8032.
What else would a serialization RFC say, that is not already said in RFC 8032? Deciding that keys are encoded with Base64 and not with, say, hex seems like a weird thing to force on people.