Hacker News Reader

Comment by Vendan | Hacker News Reader

(I'm the author of the repo in question)

Note that this repo has been sidelined, as I have fundamental issues with the protocol SSB is built on. Unless there's changes to how the messages are signed and verified, I'm not planning on putting any serious effort into SSB.

View on HN

I'm curious to hear about the issues. SSB's network transport is supposed to be pretty okay: https://github.com/auditdrivencrypto/secret-handshake

example message:

    {
      "previous": "%26AC+gU0t74jRGVeDY01...MnutGGHM=.sha256",
      "author": "@hxGxqPrplLjRG2vtjQL87...0nNwE=.ed25519",
      "sequence": 216,
      "timestamp": 1442590513298,
      "hash": "sha256",
      "content": {
        "type": "vote",
        "vote": {
          "link": "%WbQ4dq0m/zu5jxll9zUb...KjZ80JvI=.sha256",
          "value": 1
        }
      }
    }

Signed example message:

    {
      "previous": "%26AC+gU0t74jRGVeDY01...MnutGGHM=.sha256",
      "author": "@hxGxqPrplLjRG2vtjQL87...0nNwE=.ed25519",
      "sequence": 216,
      "timestamp": 1442590513298,
      "hash": "sha256",
      "content": {
        "type": "vote",
        "vote": {
          "link": "%WbQ4dq0m/zu5jxll9zUb...KjZ80JvI=.sha256",
          "value": 1
        }
      }
      "signature": "Sjq1C3yiKdmi1TWvNqxI...gmAQ==.sig.ed25519"
    }

The signature covers the whole message, and is then added into the message it signed. The way this works means you have to encode json exactly the same way as the "main" node.js implementation. Emojis, unicode, html literals, EVERYTHING. Or else it will fail to verify. I've gotten most of it working, but there's still edge cases where I gave up trying to get them to work correctly.

I know that I sound like a broken record, but this is exactly the issue which canonical S-expressions were designed for, and which SPKI wrestled with & solved twenty years ago.

The SPKI version of a message would look something like (I've removed the hash property, because I don't think it makes sense for an object to specify the hash to be used to refer to it, but one could add it back in if one wished):

    (message
     (previous (hash sha256 |XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=|))
     (author (public-key (ed25519 |FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=|)))
     (sequence 216)
     (timestamp "2018-04-19T17:53:26Z")
     (content (type vote)
              (link (hash sha256 |DlBH/hCmXfVzks2uY+WIll4aTzxrfBA8m/3GIdX3Vew=|))
              (value 1)))

The transport version would be this (you can Base64-decode it to see the canonical version):

    {KDc6bWVzc2FnZSg4OnByZXZpb3VzKDQ6aGFzaDY6c2hhMjU2MzI6XphMUkWQtomKjXQvFGfsGYpt
    69sgEY7Y4Vou9cEuJhopKSg2OmF1dGhvcigxMDpwdWJsaWMta2V5KDc6ZWQyNTUxOTMyOhQl/7bA
    y6bmwjyinyK8OIHPkkJB3Gg9e7OxiOov84lmKSkpKDg6c2VxdWVuY2UzOjIxNikoOTp0aW1lc3Rh
    bXAyMDoyMDE4LTA0LTE5VDE3OjUzOjI2WikoNzpjb250ZW50KDQ6dHlwZTQ6dm90ZSkoNDpsaW5r
    KDQ6aGFzaDY6c2hhMjU2MzI6DlBH/hCmXfVzks2uY+WIll4aTzxrfBA8m/3GIdX3VewpKSg1OnZh
    bHVlMToxKSkp}

And a signed message might look something like:

    (sequence
     (message
      (previous (hash sha256 |XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=|))
      (author (public-key (ed25519 |FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=|)))
      (sequence 216)
      (timestamp "2018-04-19T17:53:26Z")
      (content (type vote)
               (link (hash sha256 |DlBH/hCmXfVzks2uY+WIll4aTzxrfBA8m/3GIdX3Vew=|))
               (value 1)))
     (signature (hash sha256 |XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=|)
                (hash sha256 |5hHMWc1PqfrwFVfALci5JXCWqW7VC4I4iS4+Utvr44w=|)
                |z7W1ERg9UYZjNfE72ZwEuJF79khG+eOHWFp6iF+KLuSrw8Lqa6IousK4cCn9T5qFa8E14GVek4cAMmMbjqDnAg==|))

Canonical S-expressions already buy you bit-for-bit identity when hashing, and SPKI (as an example) wraps signatures rather than injecting them — the only sane choice.

Why do I bring this up? Obviously it's possible to make JSON be a cryptographically-sound format (either by foregoing objects for arrays, or by rules around object-field ordering, along with other rules about encoding), but using it instead of an already-sound format indicates an unfamiliarity with prior work in the field.

One slight correction: I realised later on that in the current draft spec, unquoted tokens must not start with a digit. Thus the example would be:

    (message
     (previous (hash sha256 |XphMUkWQtomKjXQvFGfsGYpt69sgEY7Y4Vou9cEuJho=|))
     (author (public-key (ed25519 |FCX/tsDLpubCPKKfIrw4gc+SQkHcaD17s7GI6i/ziWY=|)))
     (sequence "216")
     (timestamp "2018-04-19T17:53:26Z")
     (content (type vote)
              (link (hash sha256 |DlBH/hCmXfVzks2uY+WIll4aTzxrfBA8m/3GIdX3Vew=|))
              (value "1")))

Perhaps more to the point, the need for canonicalization in this use case is well understood from not only canonical S-expressions, but similar things done in XML and other cryptographically signed structured data formats. While not using one of the existing formats is not necessarily a bad thing, overlooking the well-established need for canonicalization is quite bad.

Agree completely. I don't know if I'd choose s-expr, but at the bare minimum, wrapping with signatures is the sane thing to do.

Yup, I've got an implementation of http over packet radio I've been working through and came to the same conclusion. Header contains the signature, base64 the message for verification.

Even though it's super low bitrate(1200bps/9600bps) since you have a binary foundation w/ base64 you can send raw binary to get low over-head but then easily verify+decode since base64 is well supported across a wide range of languages.

Which large scale open-source or commercial software is using s-expr today?

WebAssembly text format? https://developer.mozilla.org/en-US/docs/WebAssembly/Underst...

Hacker News. Emacs. Anything Lispy.

Why do you ask?

Wow, that hurts. What solutions have you suggested to the project?

I had expressed interest in changing it, but it would require changing the core data structure the network is built on, and basically be a breaking change that would almost inevitably result in having to start the entire social web over from scratch. Didn't see huge interest from the devs in doing so.

It sounds like the best path forward might be to document the NodeJS serialization format in perfect detail and create a test suite to verify all known corner cases. Until that's done, alternative implementations will be unreliable.

What are the potential downsides of not changing it?

Makes it difficult to implement the protocol in anything but node.js (and potentially version specific, if node.js ever decides to tweak json serialization stuff)

Objects in JSON are per the spec unordered, while the signature scheme here most likely relies on the order of the object's key-value pairs.

Other data formats that are ordered, and serialize somewhat more deterministically, are CBOR, Protobufs, or s-exp.

This is the same reason I lost interest. I may have stuck with it if the Rust crates weren't AGPL/GPL. (And thus can't be used in Apple's app-store which is an interesting target to me).

Would you be able to elaborate on these issues?

did so in sibling comment

Reply on news.ycombinator.com