It is not a default by any means, you can write it down to yaml as plain text too without any processing[0]. Just for some reason people decided to always use base64 without any actual reason behind it.
Edit: There is reason to do it explained above[1] - to store binary data in the yaml, but doesn't change fact you don't have to use it.
[0] https://kubernetes.io/docs/concepts/configuration/secret/#ba...