Hacker News Reader

Comment by agwa | Hacker News Reader

This is unnecessarily complicated. The beautiful thing about Ed25519 keys (private and public) is that they are simply opaque byte strings. Indeed, here is Go's definition of ed25519.PublicKey:

  type PublicKey []byte

So if your application uses exclusively Ed25519, you can just pass around the raw bytes (encoded as base64 as desired). There is no need to muck around with ASN.1.

And if you do need to transmit keys in ASN.1 structures, Go's standard library has functions for that:

https://golang.org/pkg/crypto/x509/#MarshalPKIXPublicKey

https://golang.org/pkg/crypto/x509/#ParsePKIXPublicKey

The author has just written worse versions of those functions.

View on HN

Came here to say this. Please don't use PKIX, ASN.1, and Base64-of-DER-without-PEM-headers for Ed25519. Those are all extra complexity from protocols that were misguidedly built with runtime agility, or from a different time in cryptography engineering in which we really felt the need to put dynamic types on everything. [0][1]

An Ed25519 public key is a 32 bytes sequence. An Ed25519 private key (called seed in crypto/ed25519 for unfortunate historical standardization reasons) is a 32 byte sequence. That's it.

Here they are encoded in Base64.

    public key: 5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=
    private key: shhKyGTvTeLjXGDCjEQgHRA7ps3LRNNfoO5S714kinU=

age [2] similarly uses Curve25519 keys encoded with Bech32 to make them easier to copy-paste and read aloud. Look how nice they look compared to PKIX blobs!

    $ age-keygen
    # created: 2021-04-23T12:31:14-04:00
    # public key: age1gek0nrawzg9fhkrzcmt4ql7au0n6hwflz7lqqc8wwcvefn2vssgsa4uulp
    AGE-SECRET-KEY-19JFPFAY2DF3HJP5DFGVSY4A4G4YSRHG4ZCJMKNC5MFD9C9ZN5LCSG8VTDL

I'll think about how to make the crypto/ed25519 godoc point people in the right direction once Go 1.17 enters feature freeze.

[0] https://www.imperialviolet.org/2016/05/16/agility.html

[1] https://buttondown.email/cryptography-dispatches/archive/cry...

[2] https://github.com/FiloSottile/age

This makes sense, but if I'm handing these things around probably some of the code [gasp] isn't in Go. So someone should write an RFC for the case where you don't need algorithm agility, and then devs don't need to know or care in the slightest what 25519 keys actually are, they just need to call APIs to serialize & deserialize them.

I'd argue the RFC is already there and it's RFC 8032, which defines "32 bytes string" <-> "public key" and "32 bytes string" <-> "private key" APIs.

Then you can use your preferred format to serialize a 32 bytes string if you need a text-safe encoding. (If you don't you're done!) For example, you can use RFC 4648.

I am positive every language has Base64 code, and if there's an Ed25519 library out there that can't accept 32 byte strings we should talk to the author because it's broken as it doesn't implement RFC 8032.

What else would a serialization RFC say, that is not already said in RFC 8032? Deciding that keys are encoded with Base64 and not with, say, hex seems like a weird thing to force on people.

Really?! I read 8032 because people told me I should and all the strings in there are bit or octet strings, anyone looking for textual serialization is going to come up empty. Also 8032 is way too heavyweight for mere mortals. Also bear in mind that lots of languages aren't as transparent as Go and their devs, given something purporting to be a public key, aren't going to look inside, they're just going to ask "where's the API to serialize/deserialize this so I can post it on the Web?" So yeah, I agree it would be a short RFC, but once it existed, people would arrange for those APIs to exist.

I think we might have different ideas of the purpose of RFCs, but I don't expect end users to read them. What I'm saying is that RFC 8032 already defines a binary encoding for Ed25519 keys (the octet strings you've found). How to encode a binary string in text is IMHO orthogonal, and has nothing to do with Ed25519. Similarly, PKIX definex binary DER encodings, and PEM defines how to encode DER as text.

DER : PEM = Ed25519 binary encoding : Base64

Adam's post about agility is a good read, but it's worth remembering that it makes a prediction which did not in fact come true.

> When we try to add a fourth (TLS 1.3) in the next year, we'll have to add back the workaround, no doubt.

Adam is talking about the unsafe downgrade fallback, traditionally done by web browsers, not so long ago even falling back to SSLv3 which was long obsolete.

But in fact today, if your web browser connects to a remote server proposing TLS 1.3 and the remote server just silently drops the connection because it can't conceive of such a thing, the browser goes "Huh, I guess TLS doesn't work on that server" not "Let's try again with TLS 1.2" because the design in TLS 1.3 actually works, even if getting there was a heroic effort. A compliant TLS 1.2 server will do TLS 1.2, and sufficiently non-compliant ones just break and are now presumably very rare.

As a non-crypto person who coincidentally just spent a couple days mucking with this in Go earlier this week, I absolutely agree with the author's assertion that it's very hard to find complete examples, and even articles that don't assume a decent baseline of domain expertise.

fwiw my own thought/research path, after choosing Ed25519, looked something like this:

1. "How do I generate the key pair? I should probably use openssl since I trust that and have used it in the past." After googling, I see I can use `openssl genpkey -algorithm ed25519` for the private key.

2. "OK, that gave me a pem file. How do I deal with that in Go?" From here I found the same blainsmith.com article as OP, which is overall very helpful but unfortunately contains the unnecessary asn1.Unmarshal steps.

3. "Hmm, this looks overly complex." Luckily I took some extra time to look at other search results for "ed25519 pem golang" and found the crypto/x509 functions linked above.

In retrospect, if I had just used ed25519.GenerateKey(nil) for step 1, I would have saved a ton of time. But this really wasn't initially clear to me at all. Exactly as the OP said, "Because as I stumbled through the undergrowth figuring this stuff out in the usual way, via Google and StackOverflow, I did not convince myself that the path I was following was the right one." And in this case the OP is a highly respected software engineer with his own wikipedia page, so at least I'm in good company for tripping all over this process :)

Since Microsoft strapped OpenSSH into Windows 10, we can assume ssh-keygen is the most ubiquitous ed25519 management tool; anything using OpenSSH format will be widely supported.

Does this run just about everywhere?

    ssh-keygen -t ed25519 -f testkey

That likely runs everywhere, but in my mind it doesn't solve the core problems here:

* The keys are each really just 32 byte sequences. So any tool that serializes them in a more complicated structure is just adding unnecessary complexity, if the calling code only cares about ed25519 keys anyway.

* Lack of end-to-end examples in documentation and blog posts.

Fair enough, thanks, will adopt those. An interesting question is why heavy googling and stackoverflow diving looking for "how to publish ed25519" and related things failed to turn this up.

Call me old fashioned, but I browse and search the documentation first. Starting from the index page. "pkix - Package pkix contains shared, low level structures used for ASN.1 parsing and serialization of X.509 certificates, CRL and OCSP."

Right, what path would have led you to that doc page. I read a lot of doc pages and none of the paths I followed led to that one.

Just one click away. Go to https://golang.org/

click [Packages]

Scroll down until you see crypto packages. Now you should see package named pkix and description of the package: pkix - contains shared, low level structures used for ASN.1 parsing and serialization of X.509 certificates, CRL and OCSP.

Call me weird, but I tend to start by searching for what I think I'm looking for. I was looking for "serialize 25519" or something like that. None of "pkix", "ASN.1", "CRL", X.509, o "OCSP" were in my vocabulary when I started.

I see.

1. You needed to learn how the cryptographic serialization happens in practice. Including terminology.

2. You forgot to look again after you had learned all you needed to learn and wrote your own.

It's easy to wander from learning to coding.

I think the problem you had is basically that's in uncommon to be distributing bare public keys in the most common cryptography contexts. Public keys are nearly always part of certificates.

I looked at the @bluesky identity post, and it seems to me that what's being described in the granting process is very close to a self-signed certificate.

To link two identities, couldn't you create a self-signed certificate with two SANs that identify your identity on P1 and P2 and post the same certificate to both of them, then send links to the two posts to the ledger?

An X.509 certificate is already a structured method to create a payload consisting of a public key, various claims as to identity of the subject, and a wrapping signature.

A fair question. I've opened an issue in Go suggesting some documentation improvements: https://github.com/golang/go/issues/45726

The trouble with using just a bag of (potentially Base64-encoded) bytes is that it flirts with cryptographic doom: one relies on users not to make mistakes.

For example, is 5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ= a Base64-encoded Ed25519 public key? Sure. Is it a Base64-encoded HMAC-SHA-256 key? Sure! It’s 256 bits, after all. Is it a valid AES-256 key? Yes again.

OTOH, (public-key (ed25519 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly not an HMAC private key. For that matter, (secret-key (hmac-sha-256 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly not an Ed25519 public key. And (secret-key (aes-256 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|)) is clearly neither a public Ed25519 key nor a private HMAC-SHA-256 key. Software expecting one and getting one of the other will break.

Why does this matter? Because using a public value such as a public key as an HMAC signing key means that anyone can generate a signature, and using a public value such as a public key as a secret encryption key means that anyone can decrypt the encrypted material.

Why use the unusual formats I suggested above? Because this:

    (public-key
      (ed25519 |5uW7anEGF1nIjGfp5pS2kiN0cn2mGYkuSa+TCBoFIbQ=|))

is clearly preferable to this:

    -----BEGIN PUBLIC KEY-----
    MDYwBQYDK2VwAy0ANXVXN2FuRUdGMW5JakdmcDVwUzJraU4wY24ybUdZa3VTYStUQ0JvRkliUT0
    -----END PUBLIC KEY-----

The conventional PKIX/PEM is logically equivalent to your s-expression there; I assume you're arguing for using the conventional packing rather than proposing a new s-expression based encoding?

For your purposes, I think that your choice of PEM is probably the best. As you note, it is logically equivalent to the s-expression (it is typed, not a bag of bytes), and it has the advantage of being a well-known format.

If æsthetic concerns are a higher priority than interoperability (say, for a personal hobby project), then I do recommend folks adopt the format I gave an example of. It has some history behind it, dating back to the 1997–1999 timeframe and the IETF’s SPKI Working Group (e.g. https://theworld.com/~cme/examples.txt) which resulted in RFCs 2692 & 2693, as well as some great drafts (https://people.csail.mit.edu/rivest/Sexp.txt & https://theworld.com/~cme/spki.txt) which never progressed because Reasons™.

I still think that the SPKI project got PKI right. Oh well, what might have been!

Just committed changes to use those functions. Didn't remove all the ceremony but saved a few lines of code. Thanks. Credited @agwa in the commit.

> Indeed, here is Go's definition of ed25519.PublicKey […]

Wouldn’t you need some sort of length restriction on that? Or are all byte sequences valid Ed25519 keys — e.g. the empty byte sequence and the byte sequence that comprises this[1] cat GIF?

https://i.pinimg.com/originals/c3/2b/fa/c32bfa16bcf864e478d3...

> Wouldn’t you need some sort of length restriction on that?

https://github.com/golang/go/blob/master/src/crypto/ed25519/...

It seems it would indeed panic with an invalid size.

Yup! From https://cr.yp.to/ecdh.html

> How do I validate Curve25519 public keys? > Don't. The Curve25519 function was carefully designed to allow all 32-byte strings as Diffie-Hellman public keys. (minor exceptions follow)

Public keys are 32 bytes long, and private keys are 64 bytes long. The Ed25519 functions check to make sure the key is the correct length before using them:

https://github.com/golang/go/blob/d4bfe006155ea0d9c2970e22fc...

The types would ideally use fixed-length arrays, but at the time the API was designed, Go didn't allow conversions between slices and array pointers, which would have made the API annoying to use:

https://github.com/golang/go/issues/24376#issuecomment-37278...

Go is now adding support for such conversions, but the Ed25519 API can't be changed due to backwards compatibility guarantees.

All keys are just opaque strings of bytes.

They actually aren't. RSA keys, for instance, are defined as a pair of integers (modulus and exponent)[1] and the algorithms are specified as taking these integers as input. ECDSA is similar. Therefore, to interchange RSA and ECDSA keys, you need some other specification that tells you how to serialize those integers.

It's true that internally Ed25519 keys are also integers, but the encoding/decoding is encapsulated in the Ed25519 algorithm so to users of Ed25519 they are just byte strings. This is quite a bit nicer in practice than the approach taken by RSA and ECDSA.

[1] https://tools.ietf.org/html/rfc8017#section-3

Much fun[1] can be had because RSA keys are integers. If your key would be encoded with a zero most significant byte, some implementations are fine if that byte is omited, and some will give a nonsensical error instead.

[1] Not actually fun, but I got a patch in OpenSSL, so I guess that's neat.

Adding to the fun: some implementations bark at you if you include it!

If you're going there, all information can be opaque strings of bytes.

Came here to say that the example code in the docs for ParsePKIXPublicKey seems to be exactly what the author was looking for.

Wouldnt it make more sense to use base32 instead of base64.

Also whats wrong with simply using the hex string of the public key. It is not much longer than the base64/32 string. I thought the short length was an intentional part of djb's design. Small public keys, short hex strings. Easy to share.

>opaque byte strings

The point was to have something you (an average Joe) can share on the Internet, and that's definitely not byte string.

But the whole PKI thing is indeed very complicated. After such thorough analysis I was expecting some novel and simple solution like memorizable passphrase or QR code to scan.

Reply on news.ycombinator.com